Neural loop algorithm for square ice model*

We discuss how to apply a reinforcement learning framework on the square spin ice model. Spin ice is a frustrated magnetic system with a strong topological constraint on the low-energy configurations called the ice rule. The conventional single spin-flip Monte Carlo update breaks this constraint. We exploit a reinforcement learning method that parameterizes the transition operator with neural networks. By extending the Markov chain to a Markov decision process, the algorithm can adaptively search for a global update policy through its interactions with the physical model. We find that the global loop update emerges without the explicit knowledge of the ice rule. This method might serve a general framework to search for efficient update policies in other constrained systems.