Session L12: Low-Order Modeling and Machine Learning in Fluid Dynamics: General II

Latent Diffusion Models for Partial Differential Equations Modeling

Recent advances in deep learning have inspired numerous works on data-driven solutions to fluids problems. These data-driven PDE solvers can often be much faster than their numerical counterparts; however, they present unique limitations and must balance training cost, numerical accuracy, and ease of applicability to different problem setups. To address these limitations, we frame solving time-dependent PDEs as a generative problem and apply latent diffusion models to sample PDE solution trajectories from a learned conditional distribution. In particular, we investigate conditioning on an initial solution as well as conditioning solely on a text prompt, paving the way for more usable and accessible physics solvers. Additionally, we leverage a learned latent space to accelerate training by predicting physics in this reduced space. Through experiments on regular and mesh-based physics problems, we show that this approach is competitive with current data-driven PDE solvers. By extending latent diffusion models to PDE problems, we hope that insights from prior work on generative modeling and diffusion can inspire powerful, fast, and widely applicable physics solvers.   

Pretraining a Neural Operator in Lower Dimensions

There has recently been increasing attention towards developing foundational neural Partial Differential Equation (PDE) solvers and neural operators through large-scale pertaining. However, unlike vision and language models that make use of abundant and inexpensive (unlabeled) data for pretraining, these neural solvers usually rely on simulated PDE data, which can be costly to obtain, especially for high-dimensional PDEs. In this work, we aim to Pretrain neural PDE solvers on Lower Dimensional PDEs (PreLowD) where data collection is the least expensive. We evaluated the effectiveness of this pretraining strategy in similar PDEs in higher dimensions. We use the Factorized Fourier Neural Operator (FFNO) due to having the necessary flexibility to be applied to PDE data of arbitrary spatial dimensions and reuse trained parameters in lower dimensions. In addition, our work sheds light on the effect of the fine-tuning configuration to make the most of this pretraining strategy.   

A Multi-Modal Implicit Neural Representation Method for Dimension Reduction of Spatiotemporal Flow Data

Estimating large-scale flow field data in modern engineering can be computationally prohibitive. Reduced Order Models (ROMs) accelerate flow simulations by compressing data into lower-dimensional spaces. Traditional dimensional reduction methods, such as Proper Orthogonal Decomposition (POD), Dynamic Mode Decomposition (DMD), and Spectral POD (SPOD), have proven effective but struggle with convective-dominant chaotic flows and instability. Machine learning approaches like Convolutional Auto-Encoders (CAEs) offer nonlinear encoding capabilities but fall short in predicting coherent flow structures and lack compatibility with complex irregular domains. Recently, Implicit Neural Representation (INR) has shown promise, with high accuracy and compression ratios, but still faces challenges with chaotic/turbulent flow predictions. To address these issues, we propose an innovative multi-modal INR-based data compression model. This model constructs multiple essential modes for comprehensive flow space coverage and adaptively changes modes according to the current time step, enhancing prediction accuracy for unprecedented flow features. We compare our model with state-of-the-art encoders, including POD, CAE, and INR, across several cases to demonstrate its effectiveness.   

Physics-Constrained Coupled Neural Differential Equations: Application to 1D Blood Flow Modeling

Coupled partial differential equations (PDEs) are crucial for modeling complex multi-physics phenomena, such as cardiovascular flows, which have significant real-world implications. Yet, accurately and efficiently modeling such phenomena poses considerable challenges. Here, we tackle the problem of blood flow in a stenosed artery with deformable walls, where the geometry is idealized but the problem is made reasonably complex by a realistic pulsatile inlet flow rate waveform and wall movement. We develop a low-dimensional model using physics-constrained coupled neural differential equations (PCNDEs), inspired by the 1D blood flow equations, to bridge the gap between 1D and 3D finite element simulations. Although data-driven approaches have shown promise in cardiovascular applications, neural PDE methods have not yet been applied to blood flow problems. Our innovative approach reformulates neural PDEs in space rather than the traditional temporal formulation, significantly improving the stability of the trained coupled PDEs. Temporal periodicity is explicitly enforced in the continuity equation using Fourier-series. This novel framework accurately captures flow rate and area variations, even when extrapolating to unseen inlet flow waveforms and stenosis blockage ratios. Our results offer a new perspective on using neural PDEs to model coupled PDEs with time-periodic boundary conditions.   

Machine Learning-Driven Inverse Fluid-Structure Interaction (FSI) Simulations of the Heart

Current heart fluid-structure interaction (FSI) simulations rely on ex vivo properties, which do not reflect in vivo conditions. To address this, we propose a machine learning-driven approach to solve the inverse FSI problem by identifying properties that match echo images. Our method integrates forward FSI simulations with flow, structural, and electrophysiology solvers. For the inverse problem, we use adjoint methods and convolutional neural networks (CNNs) to minimize discrepancies, validated using elastography-like techniques on a thin rectangular clamped plate with a linear elasticity variation.

We will use a simple multi-layer perceptron (MLP) with two inputs (coordinates) and one output (elasticity), followed by a single neuron with a custom activation function and a constant weight of one to compute the residuals L=R²=g(E(x,y))² . The gradient descent algorithm will optimize the MLP weights and biases to minimize residuals, with the objective​. Using the chain rule,w_i,j=α ∂L/∂w_i,j; ∂L/∂w_i,j=∂L/∂R*∂R/∂E*∂E/∂w_i,j​. The second derivative will be calculated numerically, and the last term will be computed through backpropagation.   

Diff-FlowFSI: A GPU-accelerated, JAX-based Differentiable CFD Solver for Turbulent Flow and Fluid-Structure Interactions

Recent advances in deep learning (DL) have paved the way to develop neural models by integrating physics and deep learning techniques in a hybrid framework. A promising approach is to merge DL with established physics-based numerical solvers in a more integrated manner using differentiable programming, directly enhancing the solver's capability to handle complex dynamics by introducing DL-derived insights into the simulation process. Such an approach requires a solver capable of harnessing automatic differentiation. In this paper, we introduce a highly efficient GPU-accelerated JAX-based differentiable solver, Diff-FlowFSI, for simulating large-scale turbulent flows and flow-structure interactions. Using differentiable programming, finite volume method and immersed boundary method, Diff-FlowFSI offers three key features: efficient forward simulations with GPU accelerations, differentiability for space and time derivative extraction for inverse problems, and seamless integration with DL architectures. Through various benchmarks, we demonstrate the efficacy of Diff-FlowFSI in facilitating computational mechanics research at the DL-CFD interface.   

Box model for colliding turbidity currents via equation discovery methods

Box models have been widely used in the study of singular turbidity currents since the 1990s and have been popular in the more general gravity current community long before that. As a system of ordinary differential equations the box model is easily interpretable and computationally expedient. To achieve this model all horizontal variations are neglected and therefore the classical box model is unable to capture the dynamics when distinct turbidity currents collide. With the prospect of deep-sea mining for polymetallic nodules on the rise it is becoming increasingly important to consider the interaction of neighboring turbidity currents. Using numerical data obtained by solving the shallow-water equatons for turbidity currents, we take advantage of modern equation discovery techniques (SINDy) to generate a system of ordinary differential equations as a box model for colliding turbidity currents. A comparison of the resulting sediment deposition patterns from the shallow-water equations versus the discovered box model system is provided as these patterns give the clearest interpretation of the environmental impact after the suspended sediment settles.   

Real-time prediction of turbulent flow using physics-informed neural networks with coarse spatiotemporal data

The Encoder-based DeepONet (E-DeepONet) framework developed in this study demonstrates robust predictive capabilities even with coarse spatiotemporal data. Building upon the success of the Latent DeepONet framework (Kontolati et al., Nat. Commun., 2024) in predicting complex physical systems in real-time, our research aims to enhance this approach by incorporating physical equations into the learning process. We achieve this by learning the latent space of coarse spatiotemporal data using an autoencoder, which then serves as input to the E-DeepONet for integrating physical equation learning. Instantaneous high-fidelity flow datasets are obtained from direct numerical simulation (DNS) of turbulent channel flow, providing both coarse training data and validation benchmarks. Once trained, E-DeepONet can infer continuous high-fidelity spatiotemporal flow fields in real-time from arbitrary coarse spatiotemporal flow fields. Our results indicate that the physics-informed E-DeepONet significantly outperforms models without physics information, particularly when dealing with increasingly coarse data, thus demonstrating its potential for robust and physically consistent predictions in various complex systems.   

Conjugate Gradient Greedy Identification of Latent Dynamics from Parametric Flow Data.

In this talkl, we introduce an improved regression technique tailored for uncovering quadratic parametric reduced-order dynamical systems from empirical data. Our method, referred to as Conjugate Gradient Greedy Identification of Latent Dynamics (CG-GILD), builds upon the foundation laid by the GILD method introduced in the previous work [1]. We demonstrate a strategic organization of quadratic model coefficients that facilitates an elegant reformulation of the minimum-residual problem, leveraging the Frobenius norm. This reformulation yields a generalized Sylvester equation, efficiently solvable through an adapted conjugate gradient method. Through a meticulous comparative study, we illustrate that the enhanced version CG-GILD, exhibits superior convergence for quadratic model coefficients compared to the conventional GILD employing steepest gradient descent. Notably, this advancement translates into significantly fewer iterations, thereby reducing computational complexity substantially. To underscore the efficacy of our approach, we subject it to rigorous testing on the intricate task of analyzing Ahmed body flow with variable rear slant angle, showcasing its ability to tackle real-world dynamical system challenges.

[1] M. Oulghelou, A. Ammar, R. Ayoub,  Greedy Identification of Latent Dynamics from Parametric Flow Data (arxiv 2024).