Learning egocentric navigation against adversarial flows*

Underwater navigation in the presence of ever-changing flows is an essential yet challenging task for underwater autonomous vehicles. In nature, fish have evolved sensory feedback control strategies to take advantage of ambient flow structures. However, it is non-trivial for engineers to design equivalent control laws. Here we focus on two types of challenges in sensing: First, the vehicle has only access to flow information in its immediate surroundings, which discounts a path planning beforehand. Second, sensory cues are measured in the body frame of reference (egocentric) and they change with the vehicle's translation and rotation. I will present our progress in obtaining control policies via deep reinforcement learning for a swimmer to reach a target against the unsteady background flow. While ignorance of the geocentric reference frame makes it harder for the swimmer to identify the wake structures, this difficulty can be overcome by adding velocity gradients to the sensory cues. The successful egocentric policy performs better than the geocentric policy in terms of time efficiency and generalizability to untrained situations. I will conclude with discussions on how the control policies could be further improved with different sensing strategies.