Emergent behavior of autonomous group swimmers using multi-agent deep reinforcement learning*

Various fish species can utilize the velocity field generated in the wakes of obstacles, and in the wakes of other swimmers, to reduce their energy expenditure. Here, we explore the hydrodynamic benefits of group swimming using two-dimensional numerical simulations of self-propelled anguilliform swimmers, coupled with multi-agent reinforcement learning. These artificial swimmers utilize a sensory input system that allows them to detect the velocity field and pressure on the surface of their body, which is similar to the lateral line sensing system. Deep reinforcement learning is used as a tool to discover optimal swimming patterns at the group level, as well as at the individual level, as a response to different objectives and flow fields. This can be useful in distinguishing various swimming patterns and their role in achieving higher speeds or efficiency, which are desirable objectives in different scenarios. The adaptations in response to changes in the surrounding flow field are also examined by training the swimmers in stationary flow, as well as uniform flow. These flow fields are representative of conditions encountered by fish in lakes and oceans (stationary flow), as well as during long-distance migration and in rivers (uniform flow). The physical mechanisms revealed can be helpful in understanding the motivation behind different swimming behaviors from a hydrodynamic and energetics standpoint.