Magnetic control of tokamak plasmas through deep reinforcement learning

Nuclear fusion using magnetic confinement, in particular in the tokamak configuration, is a promising path towards sustainable energy. A core challenge is to shape and maintain a high-temperature plasma within the tokamak vessel. This requires high-dimensional, high-frequency, closed-loop control using magnetic actuator coils, further complicated by the diverse requirements across a wide range of plasma configurations. Traditionally, shape and position control have been handled by linear controllers designed using control engineering methods. In this work [1], we demonstrate the first successful application of Deep Reinforcement Learning (RL) to the magnetic control of tokamaks. The Deep RL algorithm learns magnetic controllers for a target TCV plasma equilibrium solely by interacting with a free-boundary evolution model, which simulates the plasma equilibrium evolution coupled to the circuit equations for external conductors. These controllers generate and maintain a plasma of the desired shape by receiving only magnetic measurements, without the need for equilibrium reconstruction, and actuating directly the coil power supply voltages in real-time. A variety of controllers have been implemented and successfully tested in TCV experiments, achieving several different target shapes including elongated plasmas, negative triangularity and snowflake configurations. This control design method has the potential to reduce the effort required to obtain a new target shape in a tokamak. Moreover, this work represents the first use of reinforcement learning for feedback control on a tokamak and paves the way for combining physics models and machine learning for improving other aspects of control of fusion plasmas.

 

[1] J. Degrave, F. Felici, J. Buchli et al. Magnetic control of tokamak plasmas through deep reinforcement learning. Nature 602, 414–419 (2022). https://doi.org/10.1038/s41586-021-04301-9