Session L17: Machine Learning for Inference and Analysis of Fluid Flows

Fostering Open-source Resources and Practices within Deep Learning of Flow Physics

In the age of big data, open-source platforms and culture offer new resources and approaches for scientific investigations. In this work, we present pathways for integrating open-source techniques involving public datasets and crowd-sourcing into the study of multi-physics flow phenomena. One such pathway involves the Bearable Large Accessible Scientific Training Network-of-Datasets (BLASTNet) - at https://blastnet.github.io/ - which contains terabytes of non-reacting and reacting flow physics data, shared via community involvement on a free cloud computing/storage platform (Kaggle). We demonstrate that this resource can be readily employed for a wide range of data-driven tasks including closure modeling, dimensionality reduction, and super-resolution. We also demonstrate that BLASTNet can be employed for introducing new ideas from other machine learning communities such as ML model benchmarking and data competitions. Results from this work indicate that these open source techniques can help accelerate the development of deep learning techniques within flow physics.   

Overview of a database for reduced-complexity modeling of fluid flows

Reduced-complexity models have proven valuable for analyzing, understanding, and controlling fluid flows.  Often, these models require data as an a priori input or as an a posteriori point of comparison.  This presentation provides an overview of a new publicly accessible database, developed as an initiative of the AIAA Reduced-Complexity Modeling Discussion Group, specifically designed to aid in the conception, training, demonstration, evaluation, and comparison of reduced-complexity models for fluid mechanics [1].  The database contains time-resolved data for six distinct datasets: a large eddy simulation of a turbulent jet, direct numerical simulations of a zero-pressure-gradient turbulent boundary layer, particle-image-velocimetry measurements for the same boundary layer at several Reynolds numbers, direct numerical simulations of laminar stationary and pitching flat-plate airfoils, particle-image-velocimetry and force measurements of an airfoil encountering a gust, and a large eddy simulation of the separated, turbulent flow over an airfoil.  Our vision is that the common testbed provided by the database will aid the fluid mechanics community in clarifying the distinct capabilities of new and existing reduced-complexity modeling methods.

[1] Towne, A., Dawson, S.T.M., Brès, G.A, Lozano-Durán, A., Saxton-Fox, T., Parthasarathy, A., Jones, A.R., Biler, H., Yeh, C.-A. Patel, H.D., Taira, K.  (2023). A Database for Reduced-Complexity Modeling of Fluid Flows. AIAA Journal, 61 (7), 2867-2892.   

Unifying Predictions of Deterministic and Stochastic Physics in Mesh-reduced Space with Sequential Flow Generative Model

Accurately predicting the physical dynamical systems in the unstructured mesh has recently gained much more attention in scientific AI fields. However, due to the complex physics of the underlying system, it is hard to use a unified framework to predict the behavior of both deterministic and stochastic systems. Moreover, there are challenges in predicting the solution in the original high-dimensional space. To bridge these gaps, we propose a framework with the regeneration learning paradigm for accurately predicting/generating fluid dynamics. Specifically, a novel graph auto-encoder is used to represent the full-space physical variables compactly in reduced space by projecting the high dimensional data into lower dimensional intrinsic space. Moreover, an attention-based sequence model is integrated into flow-based deep generative models to predict long time-dependent dynamics. The proposed model can accurately predict/generate several deterministic/stochastic fluid dynamics. Our model outperforms the competitive baseline models for deterministic systems, meanwhile providing a physical spatial-temporal pattern of forward uncertainty estimations. Moreover, our proposed model can generate different physical realizations of stochastic fluid dynamics systems, and the generated sample has high quality using different evaluation metrics.   

Reduced Order Modelling for Urban UAS Wind Field Estimation: A Neural Galerkin Projection Approach

The inception of Urban Air Mobility (UAM), an emergent aviation ecosystem that leverages compact, unmanned aerial vehicles for transport within low-altitude urban and suburban locales, delineates a subset of the broader Advanced Air Mobility (AAM) concept. The latter aims to interconnect communities underserved by traditional transport systems through diverse intra- and inter-city operations. Notwithstanding the integration of various extant technologies, the safe navigation of Unmanned Aerial Systems (UAS) in urban canopies remains challenging, primarily due to unpredictable external forces such as wind gusts and turbulent wakes.

High-fidelity numerical simulations provide accurate wind predictions, albeit at a prohibitive computational cost for real-time applications. Conversely, Reduced Order Models (ROMs) offer an alternative for generating precise yet computationally economical predictions. In particular, Galerkin Projection (GP)-based ROMs have gained traction due to their innate capacity to incorporate underlying operator forms, ensuring physical and theoretical consistency. Nevertheless, these models suffer from instability and inaccuracies over extended temporal windows.

This study seeks to transcend these limitations by extending the Neural GP ROM framework to accommodate three-dimensional turbulence, characteristic of the flow fields encountered by UAS in urban canopies. By utilizing GP and differentiable programming-based strategies, we propose to learn low-dimensional ROM equations via the parameterization of high-dimensional flow features. Our approach anticipates an enhanced level of interpretability and computational efficiency compared to conventional deep learning-based models.

This research also intends to scrutinize the stability traits of the ROM, perform uncertainty quantification for UAS-relevant scenarios, and discuss the potential applications of our model within diverse fluid dynamics contexts. The results are expected to contribute significantly to the knowledge of efficient and precise prediction models suitable for UAM and similar applications.   

Neural field based sequential networks for parametric spatial-temporal PDEs

Real-world engineering applications often involve complex Partial Differential Equations (PDEs), leading to high-dimensional spatial-temporal discrete systems, which can be computationally intensive. To address this challenge, dimension-reduction technologies are necessary to mitigate the computational burden. Previous methods, including principal component analysis (PCA), generalized discriminant analysis (GDA), and neural networks based autoencoders, are susceptible to issues arising from non-uniform mesh resolutions, parameter dependencies, nonlinearity, etc. In this work, we propose a novel approach that combines Neural Fields (NF) for spatial learning and Sequence Networks for temporal learning to handle the complexities of parametric spatial-temporal PDEs. The Neural Field is employed for nonlinear embedding of physics fields, capturing spatial distributions, while the PDE-informed sequence network captures the temporal dynamics within the latent space. Notably, the learnable parameters within the sequence model are partially conditioned on the PDE parameters, utilizing another NF. This dual NF approach enhances the expressiveness and adaptability of our neural field-based sequence networks for parametric spatial-temporal PDEs.   

Extreme Aerodynamic Manifold: Vortex-Airfoil Interactions

Small-scale air vehicles encounter severe flight conditions in urban areas and turbulent wakes behind large structures during adverse weather.  While understanding interactions between a strong gust and a wing is important, sweeping over the huge parameter space of extreme aerodynamic flows with expensive simulations and experiments is impractical, calling for data-driven approaches. This talk discusses how such complex aerodynamics under the vortex gust-airfoil interaction can be expressed in a low-order manner by leveraging machine learning. We consider wakes over a NACA0012 airfoil at Re = 100 covering a range of angles of attack with a strong disturbance modeled by the Taylor vortex, producing a variety of complex wake patterns due to vortex-airfoil interaction. Such unsteady and violent vortical flows over the parameter space can be compressed into only three variables with a lift-augmented nonlinear autoencoder while capturing the fundamental physics of nonlinear interaction under extreme aerodynamics.  We also show that the present approach can be used for real-time state estimation using sparse sensors.   

Analyzing the relationship between wake flow patterns and design element changes of automobile using machine learning

During the automotive vehicle design process, it is crucial to identify the design elements that influence wake characteristics to improve the vehicle's aerodynamic performance. Traditionally, this analysis involved comparing the wake flow of a reference vehicle shape with that of a new design. However, when multiple design changes are made simultaneously, it can be challenging to assess their individual impact on the wake flow. To overcome this challenge, we have developed artificial intelligence models to accurately detect the design element changes that affect the wake flow. Specifically, we trained a ResNet18 model using two different approaches. The first approach is a multi-label classification model that identifies which design elements have been changed, supported by grad-CAM visualization for better interpretability. The second approach is a multi-target regression model that quantifies the magnitude of the design parameter changes. In our study, we utilized the cosine similarity of gradients of the main flow (Ux) or vorticity fields at a plane perpendicular to the main flow in the wake region as the training data format. The results showed that both models achieved effective detection of design elements and their respective impact on the wake flow.   

Factorized kernel attention for scalable PDE learning

The Transformer architecture has demonstrated state-of-the-art performance across various applications and has emerged as a promising tool for data-driven surrogate modeling of partial differential equations (PDEs). However, the application of attention mechanisms to a large number of grid points can lead to instability and remains computationally expensive, despite attempts to introduce linear-complexity variants. In this study, we propose a novel approach called Factorized Transformer (FactFormer), which is based on an axial factorized kernel integral. Specifically, we introduce a learnable projection operator that decomposes the input function into multiple sub-functions, each defined over a one-dimensional domain. These sub-functions are then utilized to compute the instance-based kernel using an axial factorized scheme. We validate the effectiveness of the proposed model on several challenging fluid dynamic systems, including 2D Kolmogorov flow, 3D isotropic turbulence and 3D smoke buoyancy.   

Global Flow Reconstruction from Local Pressure Data using Dynamic Mode Decomposition

Inspired by the lateral lines of fish, sensing aspects of a fluid flow using measurements on a surface has become a topic of substantial interest. However, current approaches often use analytical methods that are only applicable to steady potential flows, or otherwise use machine learning to estimate specific flow parameters. The recent development of Dynamic Mode Decomposition (DMD) has allowed the parameterization of dynamic features of the entire flow. In this talk, we investigate the application of deep neural networks to infer the DMD modes of the pressure field in a large and unsteady fluid-body interaction problem, using only a time-series of pressure measurements on an obstacle. These modes can then be used to reconstruct the entire flowfield. This work has potential applications in identifying efficient trajectories through unsteady flows and in obstacle sensing.   

In Situ Anomaly Detection in Turbulent Reacting Flows at the Exascale

Anomaly detection is an unsupervised machine learning approach to detect outliers in data. Frequently, principal components analysis is used to flag outliers in data with large deviations from the principal components. This method may miss some data with anomalous behavior, and hence, here we use a methodology that is centered on analyzing fourth-order joint moments (co-kurtosis), particularly focusing on its application in multivariate combustion problems with large numbers of species. An in situ co-kurtosis algorithm is employed as the anomaly detection method, facilitated by a flyweight in situ visualization and analysis infrastructure for multi-physics HPC simulations (Ascent). We apply this algorithm on-the-fly to exascale high-fidelity simulations of reacting flows, performed using an adaptive mesh refinement solver (PeleC). We demonstrate the ability of the method to detect and identify the onset of low and high temperature ignition which is used for computational steering, as chemical and combustion anomalies occur intermittently at spatio-temporal locations unknown a priori. Furthermore, through a scalability analysis, we show that the relative computational cost of this in-situ anomaly detection algorithm compared to an iteration of the reacting flow solver is small.   

Analyzing the Dynamics of Discrete Gust Encounters with Persistent Homology

When subjected to strong, discrete gusts, aerodynamic bodies are known to exhibit massive flow separation, often resulting in high levels of unsteadiness. Such flows can be challenging to characterize in a low-order fashion, due to both the nonlinearity inherent to vortex shedding, and the discrete, aperiodic nature of the disturbance. In this talk, we take a topological approach to discrete gust encounters, viewing each gust encounter as a cycle in state space. We posit that because the temporal influence of the gust is finite, these cycles should exhibit a fairly simple topology, which can be leveraged to identify appropriate low-order coordinate systems. To demonstrate this idea, we consider flowfield measurements of a discrete gust encounter. For each case, we characterize the topology of the dynamics using persistent homology, a tool that identifies "holes" in point cloud data. We then transform the dynamics to a low-order space using a nonlinear autoencoder, which we constrain such that it preserves the features identified by persistent homology. With this method, we are able to transform a family of gust encounters to a three-dimensional latent space, in which each gust encounter reduces to a simple circle, and from which the original flow can be accurately reconstructed.   

ChatGPT for Programming Numerical Problems of Fluid Mechanics

ChatGPT is a large language model produced by OpenAI. We investigate the performance of ChatGPT for generating numerical and machine learning codes for executing fluid dynamics problems. Specifically, we consider the diffusion equation, the incompressible Navier-Stokes equations, the Euler equations for compressible inviscid flow, etc. Moreover, we look ChatGPT's capabilities for coding physics-informed neural networks and convolutional neural networks for fluid flow predictions. We discuss aspects of both the successes and failures of ChatGPT. Generating singular matrices and arrays with incompatible sizes are examples of the malfunction of ChatGPT. All in all, we conclude that although ChatGPT is a promising tool for generating codes in the area of scientific computing, it requires a significant improvement for generating numerical programs for challenging and serious large-scale fluid mechanics problems.