NekRS: A Spectral Element Code for Exascale*

We describe NekRS, which is an open source spectral element thermal/fluids code designed for exascale platforms.  NekRS supports both incompressible and low-Mach formulations, ALE-based moving meshes, uRANS, and numerous boundary conditions of relevance to engineering flows.   Second- and third-order timesteppers are provided, using either standard (CFL-limited) semi-implicit methods or characteristics-based subcycling for advection.  A variety of scalable multilevel preconditioners for the pressure Poisson problem have been implemented to ensure optimal performance across a large application space.  Several simulation results are presented, including turbulent flow in the full core of a pebble bed reactor that features 352,000 spherical pebbles in an annular domain.  The mesh features 98.8 million elements of order N=8, for a total of n=50.5 billion gridpoints.  Using 27648 NVIDIA V100s on the OLCF supercomputer, Summit, the wall-clock time for this case is 0.25 seconds per step, corresponding to 6 hours for an entire flow-through time.