Reinforcement learning for pursuit and evasion of microswimmers at low Reynolds number*

Most aquatic organisms can exploit hydrodynamic information to navigate, locate their preys and escape from predators. Abstracting away from specific biological mechanisms, we study a model of two competing microswimmers engaged in a pursue-evasion (zero-sum) game while immersed in a low-Reynolds-number environment. The microswimming agents have access to limited information via the hydrodynamic disturbances generated by their opponent, which provide some cues about its swimming direction and position. They can only perform simple manoeuvres: turn left, right or go straight. The goal of the predator/pursuer is to capture the evader/prey in the shortest possible time. Conversely, the prey aims at avoiding capture or delaying it as much as possible. We let the agents discover their strategies by means of an actor-critic Reinforcement Learning algorithm. We show that the agents are able to find efficient and a-posteriori physically explainable strategies which non-trivially exploit both the dynamics and the signals provided by the hydrodynamic environment. Our study provides a proof-of-concept for the use of Reinforcement Learning to rationalize prey-predator strategies in aquatic environments, with potential applications to underwater robotics.