Multi-agent reinforcement learning for subgrid-scale modeling of environmental turbulence*

The accuracy of large-eddy simulations relies on closures that model the unresolved subgrid effects. Traditionally, such closure models are based on physical models of the structure of the subgrid-scale stress or the energy/enstrophy transfer and self-similarity.

More recently, supervised learning approaches have been extensively investigated as an alternative to traditional closure models. These approaches

learn the subgrid-scale closures from high-fidelity snapshots of the flow. Therefore, they require a large amount of data, which can be prohibitive to acquire, or non-existing, e.g., the direct numerical simulation of environmental flows of the atmosphere or the oceans.

We learn closure models using multi-agent deep reinforcement learning. This approach relies on statistics that can be calculated from a few system snapshots. The local invariants of the flow at sparsely distributed agents represent the state to match the expected long-term statistics of the system.

We demonstrate that the closure model accurately predicts probability distributions of forced two-dimensional and β-plane flows. We also evaluate the generalizability of the trained model to predict extreme events and other system parameters (e.g., at higher Re). 

Additionally, we draw interpretable statistical conclusions between the state invariant and the interscale enstrophy/energy transfers of the learned closure.