Session M13: Biological Fluid Dynamics: Locomotion, High Reynolds, Number Swimming II

Deep reinforcement learning for fish fin ray control

For ray-finned fishes, the ray-fin structure is a highly sophisticated control system enabling versatile locomotion in complex fluid environments. Although the kinematics and hydrodynamics of fish fin locomotion have been extensively studied, the complex control strategy is still poorly understood. In this work, we develop a deep reinforcement learning (DeepRL) solution coupled with multi-fidelity fluid-structure interaction (FSI) models to decipher the control strategy and understand the underlying mechanism of ray-fin locomotion. In particular, we will leverage state-of-the-art off-policy RL structures, including Twin Delayed Deep Deterministic Policy Gradient (TD3) and Soft Actor Critic (SAC), to learn the complex ray-fin control strategies for different swimming needs. To accelerate the training process, the DeepRL agent interacts with virtual environments built upon the FSI models of different fidelities, where a transfer learning strategy is adopted for efficient learning. We also combine both the model-based DeepRL and model-free fine-tuning methods to improve the sample efficiency and learning performance.   

Deep Learning Model of Flow Interactions in Large Schools

Computations are presented for two to ten foils pitching about their leading edges and aligned side-by-side.  By examining their thrust, lift and efficiency performance optimal arrangements are determined that maximize the performance of individuals within the school.  In contrast, the collective performance is increased with the compactness of the school.   Distinct performance differences between interior and exterior swimmers are observed.  Given this insight, interior swimmer and exterior swimmer deep learning models are developed.  The neural networks are trained on nearest-neighbor interactions from a small school and, using the nearest-neighbor formulation, are able to predict the performance of large schools.   

A Data-Driven Method for Determining the Hydrodynamic Force Induced by Vortices – Force Partitioning Applied to PIV Data for a Caudal Fin Model

The force partitioning method (FPM) (Zhang et al, PLOS One, 2015 and Menon & Mittal, JFM 2021) enables the partitioning of pressure-induced forces into distinct physical mechanisms/features such as added mass, viscous diffusion, vortices, and shear layers. Previous studies have applied FPM to data generated from flow simulations. In the current study, we demonstrate the first-ever application of this method using flow data from experiments. The experiments involve a model of a fish with a dynamic caudal fin and employ PIV to generate 4D data (3D in space + time) of the flow and vortices generated by the motion of the caudal fin. The fin generates a set of vortices including leading-edge and trailing-edge vortices. While a number of methods exist to extract/measure the total forces on the control surface, quantifying the forces induced by individual vortices is quite difficult. We combine PIV data from the experiments with the data-driven and physics-based FPM framework to quantify the forces induced by individual vortices and demonstrate the power of this method to dissect the flow physics of vortex-dominated flows.    

Autonomous navigation of simulated swimmers using deep reinforcement learning

Underwater vehicles usually have to rely on a combination of multiple types of navigation sensors such as SONAR for localization and object detection. We explore the feasibility of using hydrodynamic sensors as an alternative system for underwater navigation. This study uses reinforcement learning coupled with 2-dimensional numerical simulations of self propelled swimmers. The artificial swimmers with mechanosensory inputs similar to the lateral line in biological fish are trained using deep reinforcement learning to optimally perform various actions such as obstacle avoidance and autonomous navigation towards a target. The swimmers trained in this manner can then be used to navigate more reliably in previously unmapped areas. By comparing the behavior exhibited by the swimmers in different flows such as uniform flow and Karman vortex fields, we analyze the different optimal strategies for navigation in each scenario. The results can then be used to develop control strategies for unmanned robots and underwater robotic swarms.   

Experimental evidence of stable schooling formations of unconstrained swimmers

Water channel experiments are presented for a pair of NACA 0012 pitching hydrofoils of aspect ratio 3. One foil is fixed, while the other is completely free to move in the horizontal plane. A side-by-side arrangement is found to be two-dimensionally stable to perturbations away from this equilibrium arrangement, and arises naturally from purely hydrodynamic forces. This provides the first experimental evidence of two-dimensional stability and supports the Lighthill conjecture, which is that hydrodynamics forces may be sculpting the structure of fish schools. This minimal schooling arrangement also increases the swimming speed by 20% compared to an isolated swimmer.  These findings are supported by force measurements, trajectory measurements, PIV measurements, and free-swimming simulations of two-dimensional pitching foils. Moreover, previously discovered one-dimensionally stable equilibria driven by wake vortex interactions are shown to be, in fact, two-dimensionally unstable, at least for an out-of-phase synchronization. These newfound schooling performance and stability characteristics suggest that fluid-mediated equilibria may play a role in the control strategies of schooling fish and fish-inspired robots.   

Flow interactions stabilize flapping swimmers at positions that favor power saving

There is growing evidence that schooling fish benefit from flow interactions with their neighbors. Experiments with pairs of robotic flappers indicate that followers obtain hydrodynamic benefits by matching their tailbeat motion with the flow velocity induced by leader's wake, a strategy that freely-swimming goldfish were found to exhibit. However, it is not clear if fish actively seek this strategy or whether it is a passive outcome of the hydrodynamic interactions. Meanwhile, pairs of hydrofoils, positioned in tandem with no means of adjusting their motions, were shown to swim together cohesively due to interactions with the wake of the leader. Are the same hydrodynamic mechanisms at play in these distinct arrangements? To answer this question, we employed a vortex sheet model to capture the flow interactions among flapping swimmers, and we analyzed the free swimming of a pair of in-line and laterally-displaced flapping swimmers. We found that flow interactions stabilize formations of swimmers at positions that favor power saving for any flapping phase and lateral position of the follower. Our results suggest that fish do not need to actively match their tailbeat phase with the local vorticity to save energy and that passive hydrodynamics alone could create these energetically favorable formations.   

Effect of wavelength in seal-whisker inspired undulated cylinders

While hunting for prey, seals are able to use their whiskers for hydrodynamic trail following, a skill partly attributed to the unique shape of the whiskers themselves. The whisker's undulated topography has thus been investigated for its hydrodynamic properties, demonstrating reduced drag and oscillating lift forces when compared to flow over a smooth cylinder however the exact mechanisms are not well understood. The current investigation parameterizes the seal whisker inspired undulated geometry into seven non-dimensional parameters and shows that individual modifications to the geometric parameters can strongly impact the overall flow response. In particular, simulations are performed for flow over five geometries with various undulation wavelengths (λ=1, 2, 3.4, 5, 6.9) and a comparable smooth elliptical cylinder. The effect on wake patterns and spanwise variation is investigated through comparison of Reynolds stresses and turbulent kinetic energy calculations. Changes in the flow response vary nonlinearly with respect to topography wavelength with minimal drag and oscillating lift occurring at a biologically relevant wavelength, λ=3.4. The analysis highlights the redirection of energy along the span and the resulting effect on wake structure and vorticity.   

Scaling laws for propulsive performance of pitching foils with non-uniform flexibility

Aquatic animals swim efficiently in different environments, using a combination of flexible tails and appendages with non-uniform flexibility. Using a simplified pure pitching foil with non-uniform flexibility, we develop a model based on the scaling of the added mass and circulatory forces. The classic linear theory is augmented by additional nonlinearities and modified for non-uniform flexibility by considering the effective flexibility of the foil.  The scaling relation then is verified using inviscid numerical simulations and experiments over a wide range of variables, including the dimensionless amplitude, dimensionless flexion ratio, reduced frequency, and Strouhal number. The developed relations were found to be in excellent agreement with the numerical and experimental data. These scaling laws are used to identify physical mechanisms that influence the thrust generation and efficiency of a non-uniformly flexible propulsor and can be used as a guide for improving performance. These results are relevant in the development of bio-inspired underwater vehicles.   

Fine-tuning a hydrofoil's interactions with a nearby solid boundary using asymmetric pitch kinematics

We discovered that asymmetric pitch kinematics can fine-tune a pitching hydrofoil’s interactions with a nearby solid boundary. In our experiment, a hydrofoil was suspended in a water channel and driven with prescribed asymmetric pitch motions. By varying the bias angle (spatial asymmetry), stroke-speed ratio (temporal asymmetry), and normalized ground distance, we examined how near-ground thrust, lift, and efficiency were affected by asymmetric kinematics. Thrust and lift were measured by a six-axis force-torque sensor, hydrodynamic power was recorded using an encoder, and ground distance was measured by a laser distance sensor. Results show that 1) asymmetries affect lift and shift the equilibrium altitude, and 2) asymmetries provide thrust benefits with little to no penalty in efficiency. Our work reveals how spatial and temporal asymmetric pitch kinematics interact with a nearby solid boundary and how they can be applied to near-ground propulsion.   

Flow Structures Generated by Robotic Sea Lion Flippers of Varying Elasticity and Angular Velocity

The California sea lion produces thrust by clapping its foreflippers to its body, followed by a streamlined glide. This maneuver produces wake structures that are significantly different from those created by other biological swimmers that rely mostly on body/caudal fin (BCF) locomotion. Through the creation of a passively actuated sea lion foreflipper, we simulate the clapping motion against a plate to investigate the flow structures that result using particle image velocimetry (PIV). Previous research has examined the resulting flow structures of a singular flipper rotating at a constant angular velocity. It was found that suction side vortices in addition to the presence of two jets contribute to thrust production. The structures observed reveal similarities to those of impulsively starting flat plates. To expand on this and understand the effects of elasticity and angular velocity on the resulting flow structures, we investigate three flippers of different young’s modulus at constant angular velocities and altering angular velocity for each of the three flippers.