cs.ROMar 17, 2026

Controlling Fish Schools via Reinforcement Learning of Virtual Fish Movement

Authors: Yusuke Nishii, Hiroaki Kawashima

Organizations: Faculty of Engineering, Kyoto University, Kyoto, Japan · Graduate School of Informatics, Kyoto University, Kyoto, Japan

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

This study investigates a method to guide and control fish schools using virtual fish trained with reinforcement learning. We utilize 2D virtual fish displayed on a screen to overcome technical challenges such as durability and movement constraints inherent in physical robotic agents. To address the lack of detailed behavioral models for real fish, we adopt a model-free reinforcement learning approach. First, simulation results show that reinforcement learning can acquire effective movement policies even when simulated real fish frequently ignore the virtual stimulus. Second, real-world experiments with live fish confirm that the learned policy successfully guides fish schools toward specified target directions. Statistical analysis reveals that the proposed method significantly outperforms baseline conditions, including the absence of stimulus and a heuristic "stay-at-edge" strategy. This study provides an early demonstration of how reinforcement learning can be used to influence collective animal behavior through artificial agents.

Figures & tables

Figure 1: Overview of the interaction system between a fish school and virtual fish.
Figure 2: Image and arrangement of the virtual fish, including the background color.
Figure 3: Example of a captured image. Due to the light from the display, the real fish appear dark as silhouettes.
Figure 4: Interaction between the agent and the environment in reinforcement learning.
Figure 5: Definition of cells. When the number of divisions is WW , the cell at the target end is defined as W−1W-1 , and the cell at the opposite end is defined as 00 .
ParameterDescriptionValue
γ\gammaDiscount factor (Q-learning)0.9
α\alphaStep size (Learning rate) (Q-learning)0.1
WWNumber of cells10
Δwmax\Delta{w}_{\mathrm{max}}Max cell displacement per step (virtual fish)2
1/τvirtual1/\tau_{\mathrm{virtual}}Inverse time constant (virtual fish)3 s−13\text{\,}\mathrm{s}^{-1}
Δtmax\Delta{t}_{\mathrm{max}}Max duration for target switching (real fish)3 s3\text{\,}\mathrm{s}
Table 1: Simulation parameters. θ\theta and δxmax\delta{x}_{\mathrm{max}} are relative to the viewport width, and δymax\delta{y}_{\mathrm{max}} is to the viewport height.
Figure 6: Average reward RR in simulation. pp represents the probability of the simulated real fish ignoring the virtual fish. The baseline denotes the reward obtained using π^\hat{\pi} as defined in Eq. ( 31 ).
Figure 7: Ratio of the average reward of the learned policy to that of the baseline policy in simulation. pp represents the probability of the simulated real fish ignoring the virtual fish.
ParameterDescriptionValue
γ\gammaDiscount factor (Q-learning)0.9
α\alphaStep size (Learning rate) (Q-learning)0.1
WWNumber of cell divisions10
Δwmax\Delta w_{\mathrm{max}}Max cell displacement per step (virtual fish)2
1/τvirtual1/\tau_{\mathrm{virtual}}Inverse time constant (virtual fish)1 s−11\text{\,}\mathrm{s}^{-1}
Table 2: Parameters for RL and control in the real-world environment.
Figure 8: Time series of the xx -coordinate of the real fish centroid during real-world control.
Figure 9: Histograms of real fish xx -coordinates (Proposed Method). Orange bins indicate the mean value’s interval.
Figure 10: Histograms of real fish xx -coordinates (Baseline 1: stay at edge).
Figure 11: Histograms of real fish xx -coordinates (Baseline 2: w/o stimulus).
Condition(a) Target: Left(b) Target: RightDifference (b-a)
Proposed Method0.3580.3580.5250.5250.1660.166
Baseline 1 (stay at edge)0.4610.4610.5620.5620.1010.101
Baseline 2 (w/o stimulus)0.5010.5010.4950.495−0.006-0.006
Table 3: Mean real fish xx -coordinates for each guidance direction and the difference between them for each control method.
Conditionpp -value
Proposed Method1.46\times10−511.46\text{\times}{10}^{-51}
Baseline 1 (stay at edge)8.48\times10−158.48\text{\times}{10}^{-15}
Baseline 2 (w/o stimulus)5.80\times10−15.80\text{\times}{10}^{-1}
Table 4: Results of the significance tests for the difference in the mean xx -coordinates of real fish based on the target guidance direction. A two-tailed Welch’s tt -test was performed for each virtual fish control method to evaluate the statistical significance of the difference.
Figure 12: Distances between the xx -coordinate distributions of real fish for leftward and rightward guidance. For each pair of histograms in Figs. 9 to 11 , the effectiveness of the guidance was quantified by calculating the Bhattacharyya distance between the distributions resulting from the two target directions.

Explore similar work

CardsList

  1. A Deep Reinforcement Learning Framework for Closed-loop Guidance of Fish Schools via Virtual Agents

    Mar 30, 2026Takato Shibayama, Hiroaki KawashimaFeedback ControlReinforcement Learning

    Graduate School of Information Science, University of Hyogo, Kobe, Japan

  2. Robots that learn to evaluate models of collective behavior

    Apr 8, 2026Mathis Hocke, Andreas Gerken, David Bierbach +2Animal Behavior Analysis

    Department of computer science, Freie Universität Berlin, 14195 Berlin, Germany. · SCIoI Excellence Cluster, Technische Universität Berlin, 10587, Berlin, Germany. · Faculty of Life Sciences, Humboldt-Universität zu Berlin, 10117, Berlin, Germany. +1

  3. Positional choice and robust collective behavior in fish schools: biohybrid experiments and modeling

    Sep 28, 2026Vahagn Grigoryan, Donato Romano, Cesare Stefanini +1Agent-Based ModelingAnimal Behavior Analysis

    Sorbonne University Abu Dhabi, Abu Dhabi, UAE · BioRobotics Institute, Sant’Anna School of Advanced Studies, Pisa, Italy · Division of Computing and Mathematical Science, MBZUAI, Abu Dhabi, UAE