Monitoring fish growth behavior provides relevant information about fish health in aquaculture and home aquariums. Yet, monitoring fish sizes poses different challenges, as fish are small and subject to strong refractive distortions in aquarium environments. Image-based measurement offers a practical, non-invasive alternative that allows frequent monitoring without disturbing the fish. In this paper, we propose a non-invasive refraction-aware stereo vision method to estimate fish length in aquariums. Our approach uses a YOLOv11-Pose network to detect fish and predict anatomical keypoints on the fish in each stereo image. A refraction-aware epipolar constraint accounting for the air-glass-water interfaces enables robust matching, and unreliable detections are removed using a learned quality score. A subsequent refraction-aware 3D triangulation recovers 3D keypoints, from which fish length is measured. We validate our approach on a new stereo dataset of endangered Sulawesi ricefish captured under aquarium-like conditions and demonstrate that filtering low-quality detections is essential for accurate length estimation. The proposed system offers a simple and practical solution for non-invasive growth monitoring and can be easily applied in home aquariums.
Figures & tables
Figure 1: Example from our dataset. This image shows the upper left quadrant of a frame from our dataset. The fish only cover a small area of the image and partly hide between plants.
Figure 2: Example of annotated Sulawesi ricefish from our dataset. The annotation includes a bounding box, five keypoints, and a quality class. The fish at the bottom is labeled high quality (Q3) because its keypoints are clearly visible, while the fish at the top (green bounding box) is labeled medium quality (Q2) because its keypoints are hard to recognize due to motion blur. The fish with the pink bounding box is labeled as low quality (Q1) because its keypoints are barely visible due to its low contrast.
# images:
104
# annotated fish:
4,331
keypoints per fish:
5
(mouth, eye, dorsal fin
ventral fin, caudal fin)
label per fish:
quality of visibility
(high, medium, low)
Table 1: Summary of the Sulawesi Ricefish Stereo Dataset
Figure 3: Annotated examples frames of the test-only scenes. In one scene (right), the back of the aquarium was covered with white paper to simulate an aquarium close to a wall.
Figure 4: Our approach applies a YOLOv11-Pose separately on the two images of the stereo pair. The detected fish instances are then matched and different components are applied to improve keypoint positions and remove poorly estimated predictions. The individual improvement and quality assessment components are introduces in Section 4.2 . In brackets are the components’ abbreviations used in the results section.
ground-truth
predicted quality
quality
low
medium
high
backbone: nano
low
0.870
0.497
0.161
medium
0.129
0.440
0.435
high
0
0.063
0.403
backbone: small
Table 2: Confusion matrix for quality level prediction of different YOLO backbones
Used
Com-
Yolo-Backbone
ponents
Qu
Te
Di
n
s
m
l
x
11.6
17.7
19.8
13.9
16.2
X
4.6
1.5
2.4
2.8
1.1
X
5.4
10.7
11.7
7.3
9.2
Table 3: Percentage of Bad Matches [%]. Best performing settings are marked in bold. The components’ abbreviations Qu , Te , and Di refer to Quality Head, Template Matching, and Swimming Direction, respectively. See also Figure 4 and Section 4.2 .
Used
Components
Scene noBG
Scene wBG
Qu
Te
Di
n
s
m
l
x
n
s
m
l
x
22.62
18.97
17.78
16.08
18.57
20.14
18.32
18.92
19.57
18.19
X
17.87
18.75
14.11
12.11
14.16
18.78
16.84
15.86
16.38
15.19
X
20.39
16.82
21.21
17.51
20.06
19.29
17.68
18.33
17.59
17.95
X
X
17.85
15.09
20.79
16.36
19.56
16.22
15.54
11.78
11.75
12.64
Table 4: Root Mean Squared Error of length estimation for different configurations in millimeters. Best numbers per scene are marked in bold. The components’ abbreviations Qu , Te , and Di refer to Quality Head, Template Matching, and Swimming Direction, respectively. See also Figure 4 and Section 4.2 .
The aquaculture industry needs to address several challenges to secure sustainable seafood production that can serve an increasing global demand. One major challenge is to ensure good fish health and acceptable welfare during production since the improvement of fish welfare is of vital importance in current and future production systems. In this study, this is addressed by developing and implementing methods to identify fish behaviors in response to intrusive objects both on individual and on a group basis. A novel approach for detecting, tracking, and estimating the 3D position of individual fish has thus been developed, and specifically designed to track the caudal fins of farmed fish in industrial sea cages. The tracking data was subjected to a novel stereo-vision method adapted to estimate fish positions, velocities, accelerations, and turning and pitch angles. Datasets obtained from industrial-scale fish farms were then analyzed to identify the impact of structures of varying shapes, sizes, and colors on fish behavior. The method was trained using manually labeled caudal fins, and used YOLOv8 with ByteTrack as an object detector and tracker, SuperGlue for matching detections in the left and right frames, and triangulation to reconstruct the 3D positions of the fish. Different image pre-processing and augmentation methods for enhancing object detection accuracy were tested and their performance compared, while RAFT-Stereo was tested for depth estimation purposes. The obtained results both validate the method's performance against previous research efforts, and demonstrate the novelty and potential of this method in providing more insight into behavioral dynamics in sea-cages.
Hanne-Grete Alvheim, Stian Mjelde Jakobsen, Martin Føre +1
Department of Engineering Cybernetics, NTNU · Department of Aquaculture, SINTEF Ocean AS
Monitoring the health conditions of fish is essential, as it enables the early detection of disease, safeguards animal welfare, and contributes to sustainable aquaculture practices. Physiological and pathological conditions of cultivated fish can be inferred by analyzing locomotion activities. In this paper, we present a system that estimates the locomotion activities from videos using multi object tracking. The core of our approach is a YOLOv11 detector embedded in a tracking-by-detection framework. We investigate various configurations of the YOLOv11-architecture as well as extensions that incorporate multiple frames to improve detection accuracy. Our system is evaluated on a manually annotated dataset of Sulawesi ricefish recorded in a home-aquarium-like setup, demonstrating its ability to reliably measure swimming direction and speed for fish health monitoring. The dataset will be made publicly available upon publication.
Timon Palm, Clemens Seibold, Anna Hilsmann +1
Fraunhofer HHI, Berlin, Germany · Humboldt University of Berlin, Berlin, Germany
Precision aquaculture faces a "phenotyping bottleneck" in tracking high-resolution behavioral traits, as conventional methods cannot quantify instantaneous three-dimensional (3D) physical exertion. To address this, we present a high-throughput 3D behavioral phenotyping framework integrating deep learning object detection with binocular stereo vision for real-time monitoring of juvenile tilapia in high-density environments. The system automates non-contact body length estimation and reconstructs 3D swimming trajectories from absolute spatial coordinates. By eliminating 2D perspective distortions, this approach precisely quantifies 3D velocity and acceleration, marking the first estimation of true physical swimming speeds in free-roaming juveniles. Results show the framework successfully establishes circadian locomotor baselines, serving as an early warning system for physiological stress and providing an objective metric for fish vitality.
AI Research Center, National Taiwan Ocean Univ., Keelung City 20224, Taiwan. · Dept. of Aquaculture, National Taiwan Ocean Univ., Keelung City 20224, Taiwan. · Center of Excellence for the Oceans, National Taiwan Ocean University, Keelung City 20224, Taiwan.