Hamiltonian Engineering of Dipolar Coupled Spin Systems Using Reinforcement Learning*

Hamiltonian engineering of quantum many-body systems is central to many quantum simulation and sensing protocols. Average Hamiltonian theory (AHT) has historically been the methodology used to both design pulses sequences to engineer effective Hamiltonians in solid-state spin systems, as well as to characterize their sensitivity to error.  Recently, reinforcement learning (RL) has emerged as another avenue to engineer effective Hamiltonians.  RL algorithms treat the system's dynamics as a black box with the potential to provide robustness to experimental imperfections.  However, unconstrained RL algorithms have not outperformed conventional methods to date.  There are multiple ways in which additional physical constraints can be imposed on the RL algorithm.  Here, we use theoretical insights into the quantum dynamics of the interacting spin systems to constrain the action space of the RL algorithm.  For the problem of decoupling magnetic dipolar interactions in solid-state spin systems, this approach allows us to find multiple pulse sequences that perform at a similar level to conventional sequences, and at a significantly higher level than unconstrained RL algorithms.