GPU-Enabled LICA Fluid Dynamic Solver for Large Scale Semiconductor Fabrication Plant Flow Simulation*

We present the GPU-LICA solver, GPU-enabled scalable solver for largescale flow simulation in complex geometries for industrial applications. Our flow solver is parallelized based on a hybrid approach of utilizing MPI, CUDA, and OpenACC for multi-GPU computational environments. Core components of the flow solver are matrix solvers for solution of the momentum and pressure equations and are parallelized by utilizing MPI and CUDA. The momentum equation is solved by a highly parallel and scalable tridiagonal matrix solver, PaScaL TDMA, which requires significantly lower overheads of all-to-all communications in comparison with other existing tridiagonal solvers. The pressure equation is solved by a newly developed Discrete Cosine Transformation solver, which significantly accelerates all-to-all communications by optimizing MPI topologies for taking advantage of high-bandwidth NVLinks. The GPU-LICA solver demonstrates excellent scalability, achieving 96% parallel efficiency for the 8.5 billion Degrees of Freedom (DoF) case in strong scalability and 97.5% parallel efficiency for the 134 million DoF per GPU case in weak scalability. The GPU-LICA solver demonstrates its ability to simulate flow with complex geometry and facilities within a large-scale semiconductor fabrication plant at Re=66700. The strong scalability results demonstrate an 85.5% parallel efficiency with 128 GPUs, which is practically important as it enables the completion of a flow circulation simulation within a day.