Enhancing Physics-Informed Neural Networks with Constrained Optimization: A Novel Adaptive Augmented Lagrangian Method for Solving Complex PDEs*

Various methods have been developed to solve partial differential equations (PDEs) using physics-informed neural networks (PINNs), with varying levels of accuracy and success across different types of PDE problems. Among these, Physics and Equality Constrained Artificial Neural Networks (PECANNs) stand out by initially adopting a constrained optimization framework. In the PECANN approach, the residual form of the PDE loss is bounded by the residuals of the boundary conditions and any additional constraints relevant to the PDE. This setup employs the Augmented Lagrangian Method (ALM) to transform the constrained optimization problem into an unconstrained one in a systematic manner. Utilizing Lagrange multipliers and penalty parameters, ALM dynamically adjusts the weights of each term in the objective function according to a specific update strategy. In this study, we introduce a novel update strategy inspired by the RMSProp algorithm, demonstrating its effectiveness in enhancing the optimization process. We apply our method to model the reversible advection of a passive scalar by a vortex and to solve the Helmholtz equation with varying degrees of complexity. Our results are compared with those obtained from finite-basis PINNs and physics-informed Kolmogorov-Arnold network approaches, showcasing the potential advantages of our method.