Incorporating Gas Puffing Delays into Density Control Synthesis in Tokamak Reactors By Combining Optimal Control and Reinforcement Learning Techniques*

Gas puffing is one of the main actuation systems available for regulating particle density in tokamaks. In next-generation tokamaks like ITER, the distance between the gas valves and the vacuum vessel is expected to delay the response of gas-puffing actuation systems. If not appropriately accounted for during control synthesis, such delayed response from the actuators can lead to performance degradation or destabilization of the density control system. This work presents a systematic approach to designing a model-based density controller that inherently accounts for input delays in the gas puffing systems. The proposed control synthesis relies on formulating an infinite-horizon optimal control problem and solving it using reinforcement learning algorithms with an actor-critic architecture. The resulting solution is then reformulated to account for input delays. In addition to control synthesis, systematic methods to handle changes in plasma operating conditions, which can lead to substantial uncertainties in the density model, are also proposed in this work. Finally, the effectiveness of the proposed density controller is illustrated using nonlinear numerical simulations.