Controlling Plasma Profiles in a Learned Model via Reinforcement Learning*

Plasmas in a tokamak are well-specified by their profiles alongside parameters describing the shape and magnetic field observed. Many scalar quantities of interest such as $\beta_N$ are derivable from the profiles as well. Therefore, controlling profiles to be close to specified target values well-known and highly useful capability in tokamak experiments.

 

In this work we contribute to profile control by first learning in supervised fashion a dynamical model of profiles from data taken from deuterium shots on DIII-D then assuming this model is the system of interest and using soft actor-critic algorithms to train a controller. We ensure the controls are realistic by modifying the action space and are able to achieve successful realistic looking profile control in our simulated environment.

 

We hope to next learn a conservative controller directly from data using offline reinforcement learning techniques that is well suited to the real tokamak, which will build on results from this study on actuator constraints, feature representation, and identification of successful learning algorithms to be applied to a new setting.