Emergence of Longitudinal Queues in Group Navigation: An Interpretable Approach via Projective Simulation
Source abstract
Abstract
The formation of longitudinal queues is critical for biological and artificial swarm systems to achieve efficient long-distance navigation. However, the “black-box” nature of conventional deep reinforcement learning models often obscures the microscopic rules driving the emergence of such ordered behaviors. To address this challenge, this paper proposes an interpretable computational model of collective behavior based on Projective Simulation and Episodic Compositional Memory, which enables individuals to learn decision-making strategies within a transparent state–action network. Simulation results demonstrate that the swarm can self-organize into stable and highly elongated longitudinal queues. Crucially, through visualization of microscopic strategies, we reveal a deterministic target-priority mechanism: when local neighbor alignment conflicts with global target orientation, individuals learn to strictly prioritize the target direction, serving as the key driving force for queue formation. Further parametric analysis indicates that the action space granularity exerts a nonlinear impact on stability, identifying moderate control precision as the optimal choice. This study not only provides a transparent computational explanation for the self-organization mechanism of queues in collective motion but also offers a theoretical foundation for designing interpretable swarm navigation systems.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Reinforcement Learning in Robotics · Distributed Control Multi-Agent Systems · Slime Mold and Myxomycetes Research
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
College of Mechanical and Electrical Engineering, Harbin Engineering University, Harbin 150001, China
References (52)
Show source-provided references
1. Collective motion10.1016/j.physrep.2012.03.004 ↗
2. Self-organization and collective behavior in vertebrates10.1016/S0065-3454(03)01001-5 ↗
3. Stochastic modelling of bird flocks: Accounting for the cohesiveness of collective motion10.1098/rsif.2021.0745 ↗
4. Emergence of splits and collective turns in pigeon flocks under predation10.1098/rsos.211898 ↗
5. Scale-invariant correlations in dynamic bacterial clusters10.1103/PhysRevLett.108.148101 ↗
6. Large-scale vortex lattice emerging from collectively moving microtubules10.1038/nature10874 ↗
7. Dynamics phenotyping across length and time scales in collective cell migration10.1016/j.semcdb.2018.10.010 ↗
8. From Disorder to Order in Marching Locusts10.1126/science.1125142 ↗
9. From behavioural analyses to models of collective motion in fish schools10.1098/rsfs.2012.0033 ↗
10. Inferring the rules of interaction of shoaling fish10.1073/pnas.1109355108 ↗
11. Causes of variation of darkness in flocks of starlings, a computational model10.1007/s11721-021-00207-4 ↗
12. Swarm robotics: A review from the swarm engineering perspective10.1007/s11721-012-0075-2 ↗
13. Emergent Sensing of Complex Environments by Mobile Animal Groups10.1126/science.1225883 ↗
14. Function of flow wakes for queuing trilobites: Positioning rather than drag reduction—Criteria for drag force assessment in palaeontological CFD simulations10.1016/j.palaeo.2024.112239 ↗
15. Flow interactions lead to self-organized flight formations disrupted by self-amplifying waves10.1038/s41467-024-47525-9 ↗
16. Queuing Behavior of Spiny Lobsters10.1126/science.164.3886.1425 ↗
17. Drag Reduction by Formation Movement in Spiny Lobsters10.1126/science.193.4258.1146 ↗
18. Self-organized lane formation and optimized traffic flow in army ants10.1098/rspb.2002.2210 ↗
19. Collective Decision-Making and Foraging Patterns in Ants and Honeybees10.1016/S0065-2806(08)00002-7 ↗
20. Hydromechanics of Fish Schooling10.1038/241290a0 ↗
21. Energy saving mechanisms, collective behavior and the variation range hypothesis in biological systems: A review10.1016/j.biosystems.2016.05.010 ↗
22. Novel type of phase transition in a system of self-driven particles10.1103/PhysRevLett.75.1226 ↗
23. Flocks, herds and schools: A distributed behavioral model10.1145/37402.37406 ↗
24. Collective memory and spatial sorting in animal groups10.1006/jtbi.2002.3065 ↗
25. Flocking without Alignment Interactions in Attractive Active Brownian Particles10.1103/PhysRevLett.130.148202 ↗
26. Non-reciprocal multifarious self-organization10.1038/s41565-022-01258-2 ↗
27. Entropy production of nonreciprocal interactions10.1103/PhysRevResearch.5.L022033 ↗
28. Modeling Collective Behavior for Fish School with Deep Q-Networks10.1109/ACCESS.2023.3263237 ↗
29. Human-level control through deep reinforcement learning10.1038/nature14236 ↗
30. A reinforcement learning-based predator-prey model10.1016/j.ecocom.2020.100815 ↗
31. Learning to flock through reinforcement10.1103/PhysRevE.102.012601 ↗
32. Costa, T., Laan, A., Heras, F.J.H., and de Polavieja, G.G. (2020). Automated Discovery of Local Rules for Desired Collective-Level Behavior Through Reinforcement Learning. Front. Phys., 8.10.3389/fphy.2020.00200 ↗
33. Modeling collective motion for fish schooling via multi-agent reinforcement learning10.1016/j.ecolmodel.2022.110259 ↗
34. López-Incera, A., Ried, K., Müller, T., and Briegel, H.J. (2020). Development of swarm behavior in artificial learning agents that adapt to different foraging environments. PLoS ONE, 15.10.1371/journal.pone.0243628 ↗
35. Explainable Artificial Intelligence (XAI): Concepts, taxonomies, opportunities and challenges toward responsible AI10.1016/j.inffus.2019.12.012 ↗
36. Explainable Reinforcement Learning: A Survey and Comparative Review10.1145/3616864 ↗
37. Projective simulation for artificial intelligence10.1038/srep00400 ↗
38. Projective simulation with generalization10.1038/s41598-017-14740-y ↗
39. Ried, K., Müller, T., and Briegel, H.J. (2019). Modelling collective motion based on the principle of agency: General framework and the case of marching locusts. PLoS ONE, 14.10.1371/journal.pone.0212044 ↗
40. Categorical Representation of Visual Stimuli in the Primate Prefrontal Cortex10.1126/science.291.5502.312 ↗
41. A biomimetic vision-based hovercraft accounts for bees’ complex behaviour in various corridors10.1088/1748-3182/9/3/036003 ↗
42. Visual Navigation in Flying Insects10.1016/S0074-7742(08)60738-2 ↗
43. Interaction ruling animal collective behavior depends on topological rather than metric distance: Evidence from a field study10.1073/pnas.0711437105 ↗
44. Why neurons mix: High dimensionality for higher cognition10.1016/j.conb.2016.01.010 ↗
45. A mathematical theory of communication10.1002/j.1538-7305.1948.tb01338.x ↗
46. Emergent collective behavior of cohesive, aligning particles10.1140/epje/s10189-025-00482-7 ↗
47. A half-cooperative strategy based Deep Q-network algorithm for multi-agent dynamic target hunting10.1016/j.robot.2026.105368 ↗
48. Adaptive risk-aware reinforcement learning for safe navigation of unmanned systems10.1016/j.patcog.2025.112900 ↗
49. Are Bigger Brains Better?10.1016/j.cub.2009.08.023 ↗
50. Energy limitation as a selective pressure on the evolution of sensory systems10.1242/jeb.017574 ↗
51. Collective detection based on visual information in animal groups10.1098/rsif.2021.0142 ↗
52. Collective animal navigation and migratory culture: From theoretical models to empirical evidence10.1098/rstb.2017.0009 ↗