journal article·2025-11-28

Reinforcement learning for robust navigation of fish-like agents in various fluid environments

Jin Zhang, Xiaolong Chen, Bochao Cao

Bioinspiration & Biomimetics· Volume 20· Issue 6
Publisher page

Abstract

Abstract Achieving robust and energy-efficient navigation in unknown fluid environments remains a key challenge for bioinspired underwater robots. In this study, we develop a reinforcement learning-based control framework that enables a fish-like swimmer to autonomously acquire effective navigation strategies within a high-fidelity computational fluid dynamics environment. By shaping the reward function to favor energy efficiency, the agent spontaneously discovers different locomotion patterns, ranging from continuous bursting to burst-and-coast gaits, all without prior knowledge of fluid mechanics. Although the agent is trained in a quiescent fluid environment, the learned swimming policies are generalized well in various navigation tasks and remain robust under complex flow perturbations, including uniform currents and unsteady vortex wakes. In all test scenarios, the agent achieves a 100% navigation success rate. These findings highlight the potential of integrating physics-based simulation with learning-based control strategy to advance the design of adaptive, efficient, and resilient aquatic robots inspired by biological swimmers.
Source: crossref · Quality: candidate unverified · Source record ↗

Research context

OpenAlex topics: Underwater Vehicles and Communication Systems · Reinforcement Learning in Robotics · Biomimetic flight and propulsion mechanisms

SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.

References (31)

Show source-provided references
1. Hydrodynamics of fishlike swimming10.1146/annurev.fluid.32.1.33 ↗
2. Exploration of underwater life with an acoustically controlled soft robotic fish10.1126/scirobotics.aar3449 ↗
3. Self-powered soft robot in the mariana trench10.1038/s41586-020-03153-z ↗
4. Fish-inspired robots: design, sensing, actuation and autonomy—a review of research10.1088/1748-3190/11/3/031001 ↗
5. Recent progress in modeling and control of bio-inspired fish robots10.3390/jmse10060773 ↗
6. Design and control of an embedded vision guided robotic fish with multiple control surfaces10.1155/2014/631296 ↗
7. CPG network optimization for a biomimetic robotic fish via PSO10.1109/TNNLS.2015.2459913 ↗
8. Flow-relative control of an underwater robot10.1098/rspa.2012.0671 ↗
9. Flow-aided path following of an underwater robot
10. Machine learning for fluid mechanics10.1146/annurev-fluid-010719-060214 ↗
11. Recent advances in applying deep reinforcement learning for flow control: perspectives and future directions10.1063/5.0143913 ↗
12. Reinforcement learning and wavelet adapted vortex methods for simulations of self-propelled swimmers10.1137/130943078 ↗
13. Efficient collective swimming by harnessing vortices through deep reinforcement learning10.1073/pnas.1800923115 ↗
14. Learning to school in dense configurations with multi-agent deep reinforcement learning10.1088/1748-3190/ac9fb5 ↗
15. Fish can save energy via proprioceptive sensing10.1088/1748-3190/ac165e ↗
16. A numerical study of fish adaption behaviors in complex environments with a deep reinforcement learning and immersed boundary–Lattice Boltzmann method10.1038/s41598-021-81124-8 ↗
17. Learning efficient navigation in vortical flow fields10.1038/s41467-021-27015-y ↗
18. Learning obstacle avoidance and predation in complex reef environments with deep reinforcement learning10.1088/1748-3190/ad6544 ↗
19. Intermittent swimmers optimize energy expenditure with flick-to-flick motor control10.1017/jfm.2025.45 ↗
20. Simulating fish autonomous swimming behaviours using deep reinforcement learning based on Kolmogorov–Arnold networks10.1088/1748-3190/ada59c ↗
21. A unified mathematical framework and an adaptive numerical method for fluid–structure interaction with rigid, deforming and elastic bodies10.1016/j.jcp.2013.04.033 ↗
22. The Kármán gait: novel body kinematics of rainbow trout swimming in a vortex street10.1242/jeb.00209 ↗
23. The role of the lateral line and vision on body kinematics and hydrodynamic preference of rainbow trout in turbulent flow10.1242/jeb.02487 ↗
24. An immersed boundary method for smoothed particle hydrodynamics of self-propelled swimmers10.1016/j.jcp.2008.06.017 ↗
25. Simulation of self-propelled anguilliform swimming by local domain-free discretization method10.1002/fld.2670 ↗
26. Tailbeat perturbations improve swimming efficiency in self-propelled flapping foils10.1017/jfm.2024.262 ↗
27. Hydrodynamic interactions in schooling fish: prioritizing real fish kinematics over travelling-wavy undulation
28. Immersed boundary methods: historical perspective and future outlook10.1146/annurev-fluid-120720-022129 ↗
29. Proximal policy optimization algorithms
30. Deep reinforcement learning for robotics: a survey of real-world successes10.1146/annurev-control-030323-022510 ↗
31. Structure of the zebrafish locomotor repertoire revealed with unsupervised behavioral clustering10.1016/j.cub.2017.12.002 ↗