Session R29: Modeling Methods I: Closure Models, Automated Discovery of Equations, and Prediction

Closure modeling through the lens of multifidelity operator learning

Projection-based reduced order models (PROMs) have shown promise in representing the behavior of multiscale systems using a small set of generalized (or latent) variables. Despite their success, PROMs can be susceptible to inaccuracies, even instabilities, due to the improper accounting of the interaction between the resolved and unresolved scales of the multiscale system (known as the closure problem). In this talk, we interpret closure as a multifidelity learning task and use a multifidelity deep operator network framework to address it. In addition, to enhance the stability and/or accuracy of the multifidelity-based closure, we employ the recently developed "in-the-loop" training approach from the literature on coupling physics and machine learning models. Numerical experiments, using advection-dominated flow problems, show significant improvement of the predictive ability of the closure-corrected PROM over the un-corrected one both in the interpolative and the extrapolative regimes.   

A Closed Machine Learning Parametric Reduced Order Model Approach - Application to Turbulent Flows

Generally, reduced order models for fluid flows are intrusively built by projecting the governing equations onto a subspace often generated by the Proper Orthogonal Decomposition (POD). In this talk, we introduce a non-intrusive paradigm based on Machine Learning to build Closed Parametric Reduced Order Models (ML-CPROM) relevant to fluid dynamics. This method is purely data-driven as it operates on data regardless of their origin (DNS, RANS, or experiment). The key idea to building such models is to assimilate the derivatives of the temporal POD modes to a quadratic polynomial with a closure term. The closure term predicted by a Long-Short-Term-Memory neural network is added to account for errors that may stem from data noise, POD truncation, and time integration schemes. To address parameter variations, the model is updated by interpolation onto the quotient manifold of the set of maximal-rank matrices by the orthogonal group. The potential of the ML-CPROM method is assessed on examples of flow past a cylinder with variable Reynolds number, and the flow past an Ahmed-body with a variable rear slant angle. We show that the ML-CPORM succeeds in recovering the dynamics with good accuracy, even for parameter values on which it was not previously trained.   

Discovery of viscoelastic constitutive models with complexity-penalized sparse regression

Identifying fluid mechanical constitutive models that are simple, rooted in physics and computationally tractable has been historically challenging. Although data-driven approaches have become increasingly popular, many of these methods result in models that feature impressive accuracy, but degrees of complexity that make them unlikely to represent a `true' solution. In this talk, we present an alternate methodology to formulate compact, algebraic constitutive models for viscoelastic fluids. In this method, sparse regression is applied to 'trusted data' to determine a minimal set of basis tensors required to capture relevant physics. The coefficients for each of the tensor bases are postulated through a mathematical classifier and the ideal model is selected by minimizing a cost functional that penalizes both model error and model complexity; here, complexity is measured by a standardized computational cost of the mathematical operations in each model. The methodology is first demonstrated on two flow classes with known analytical solutions for the polymeric stress tensor: steady pipe flow and start-up Poiseuille flow of Oldroyd-B fluids. These validation cases are chosen due to their increasing level of complexity--statistically one- and two-dimensional, respectively. Finally, the methodology is applied to three-dimensional direct numerical simulations of a viscoelastic jet of a FENE-P fluid; the resulting, learned constitutive model is demonstrated in a forward solve and compared with the original training data.   

Neural Operator for Modeling Dynamical Systems with Trajectories and Statistics Matching

Closure models are widely used in simulating complex multiscale dynamical systems such as turbulence and Earth’s climate, for which direct numerical simulation that resolves all scales is often too expensive. In recent years, neural operator and neural ODE provide spatially and temporally continuous frameworks that are independent of discretization for learning an unknown dynamical system. In this talk, we present a data-driven modeling framework for constructing continuous closure models that can efficiently match both short-term trajectories and long-term statistics for complex dynamical systems, leading to more predictive and stable data-driven closure models. Specifically, neural operator with a hybrid learning method will be demonstrated by a few canonical examples. We also show how different types of regularization can be imposed to improve the performance of the learned closure models. The results show that the proposed methodology provides an efficient and robust framework for constructing generalizable data-driven closure models for dynamical systems.   

Bayesian Identification of Nonlinear Dynamics (BINDy)

The Sparse Identification of Nonlinear Dynamics (SINDy) framework has been shown to be effective in learning interpretable and parsimonious models directly from data. However, existing SINDy derivatives can be computationally expensive and may struggle to learn the correct model equations from noisy and small datasets.

We propose a Bayesian extension to SINDy for learning sparse equations from data. Our method shows more robust capability in learning the correct model in the low-data limit, as it sparsifies the model based on both the value and the distribution of the parameters during regression, and uses the marginal likelihood (evidence) to rank and select the candidate models. The proposed method uses Laplace's method to approximate the Bayesian likelihood and evidence, avoiding the need for computationally expensive Markov chain Monte Carlo (MCMC) sampling. This results in a significant speedup in computation compared to other Bayesian SINDy methods, while still achieving comparable or better performance than existing methods such as Ensemble-SINDy.

We demonstrate the effectiveness of the proposed method on a variety of problems, including learning the Lotka-Volterra equations from 21 experimental data points of the Hudson Bay Lynx-Hare population dataset, and the Lorenz system using tens of noisy data points. We also apply the method to a real-life dataset in fluids, such as the measurements of the quasi-2D Kolmogorov-like flow, showing how it can learn both the high-order equations and low-order dynamics with just a tiny subset of the data.   

Application of Denoising Diffusion Probabilistic Models to Turbulence Prediction

Over the past few years, denoising diffusion probabilistic models (DDPMs), an advancement of the diffusion probabilistic model (DPM), have garnered attention for their comparable ability to state-of-the-art models like generative adversarial networks (GANs). DDPMs have achieved both flexibility and tractability; however, there is still ample room for improvement, as they have not been explored as extensively as GANs in terms of model architecture and hyperparameters. Particularly, applications of DDPMs to turbulence data, more generally to fluid dynamics data, are scarce, necessitating extensive analysis and research to assess the model's feasibility and performance concerning flow physics and turbulence statistics. In this presentation, we introduce an application of the simplest unconditional DDPM to turbulence generation using 2D isotropic turbulence, which provides a relatively simple and analyzable context. Additionally, we extend our investigation to turbulence prediction by utilizing the flow field from the previous time point as a condition for the backbone DDPM. Through a thorough analysis and comparison of the results by the conditional DDPM with a high-performance prediction model based on GANs, we assess the model's potential and identify whether it requires further improvements. This study can contribute to understanding DDPMs' capabilities in handling turbulence data and offer insights into their potential applications in fluid dynamics research.   

Data-driven observable discovery for reduced-order modeling of turbulence based on the Mori-Zwanzig formalism

Full-resolution simulation of turbulent flow is often computationally prohibitive, necessitating the development of reduced-order models (ROM) in real-world scientific and engineering applications. However, accurate ROM for turbulent flow is challenging, as the unresolved information of the full system influences the flow substantially. The Mori-Zwanzig (MZ) formalism provides a strategy to approach ROMs through a mathematically exact evolution of a reduced-order set of observables, in which the effects of the unresolved dynamics are captured via memory kernels and orthogonal dynamics. In our previous work [Tian et al. PoF 33(12), 2021], we presented a data-driven framework that extracts MZ kernels from Direct Numerical Simulation data, where we highlight the importance of observable choices. In this work, we aim to identify observables that can improve the learning and predictability of the learned MZ-based turbulence models. To accomplish this, we formulate a joint-learning problem by combining the learning of MZ operators with the discovery of observables from a diverse set of physics-inspired governing equations and neural network-based functions. Results show that selecting a more suitable set of observables can significantly enhance the MZ-based turbulence model's ability to predict turbulence structures and statistics at the resolved-scale.   

An improved likelihood-weighted sequential sampling method for extreme events statistics

In this work, we aim to improve the sequential sampling method which estimates the extreme-event statistics in response of a system subject to probabilistic input. The central part of sequential sampling is the acquisition based on which the next sample is selected. Among various kinds of acquisitions, the likelihood-weighted one (Blanchard & Spasis 2021) is among the most successful developments and has been applied to quantify extreme-event statistics in different contexts (e.g., extreme ship motion in waves, pandemic burst, along with other applications in Bayesian optimization, UAV path planning, and multi-arm bandit). This acquisition, however, assumes that the predicted output is sufficiently close to the ground truth. With only a limited number of samples available due to high evaluation costs, this condition can hardly be satisfied. Considering that, we improve the likelihood-weight acquisition by remedying the potential discrepancy between the prediction and ground truth. The new acquisition demonstrates significant improvements in a large number of synthetic cases with varying response functions (dimensions, variations, continuity) and a real-world application for quantifying extreme ship statistics.