cs.CVOct 15, 2026

CFC26: Building Evaluations for Deployment in Sonar-Based Fish Counting

Authors: Madison Van Horn, Suzanne Stathatos, Sevan Brodjian, Justin Kay, Kai Van Brunt, Erik Young, Sara Beery, Pietro Perona, 

Organizations: California Institute of Technology · Massachusetts Institute of Technology · California Institute of Technology Volunteer

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Accurate counts of fish populations are essential for monitoring ecosystems and making decisions for conservation, harvest limits, and river management. In many field deployments, these counts are obtained from sonar videos that experts must inspect manually, making large-scale monitoring slow, expensive, and difficult. Automated counting systems have been proposed as a scalable alternative, but so far this promise has been difficult to realize in deployment. Developing evaluations that better reflect real-world deployment conditions will help close this gap and drive improvements in counting performance in practice. To achieve this, we propose Caltech Fish Counting 2026 (CFC26), a new dataset with improved evaluation protocols that simulate deployment to seen and novel rivers, testing in-distribution and out-of-distribution performance. We expand the CFC22 dataset to eleven deployment locations across nine Pacific salmon river systems from Alaska to Northern California, capturing a broader geographical and ecological range of data. We additionally introduce two new evaluation metrics for counting, nMNE and nMANE, that mirror the operational workflow that experts use and help surface errors that frame-level detection metrics do not capture. Using these metrics, we reveal previously-concealed directional bias in upstream vs downstream counts, which would have direct and significant ecological implications if undetected. The dataset and code are publicly available on CaltechDATA and GitHub.

Figures & tables

Figure 1: CFC26 data is diverse geographically and visually. Sonar sensors are deployed across nine distinct rivers spanning Alaska, British Columbia, Washington, and California, across eleven deployment locations. Blue markers indicate locations included in CFC22, where we have added additional data and annotations; orange markers indicate new CFC26 locations: (1) Chemainus, (2) Dungeness, (3) Eel, (4) Elwha, (5) Kenai-Channel, (6) Kenai-Leftbank, (7) Kenai-Rightbank, (8) Klamath, (9) Mad, (10) Nanaimo, (11) Nushagak.
Figure 2: CFC26 expands coverage with existing and newly added locations. The left chart shows the number of bounding boxes per location, highlighting each river’s contribution to the dataset. Elwha and Kenai-Channel include additional annotations not present in CFC22. The right plot shows how locations differ: CFC22 locations generally have higher densities and larger bounding boxes than newly added locations. Kenai locations are shown by stratum rather than aggregated, exposing variation across sonar range configurations within the same deployment site.
Figure 3: Example of temporal distribution of annotated clips for Kenai-Leftbank Stratum1. Data splits were constructed by identifying gaps in coverage to define temporally separated train, validation and test splits, reducing any potential data leakage. This was applied across all locations.
Location# Clips# Frames# TracksTrack Coverage (%)# BoxesAvg Track Length
Chemainus2216,28744941.828,48822.63
Dungeness63,118895.483,273493.00
Eel272,90010448.313,13635.09
Elwha264107,23845031.124,470392.38
Kenai-Channel21741,9291,42178.31179,789106.65
Kenai-Leftbank546192,6521,97041.31150,599101.94
Table 1: CFC26 scales and diversifies CFC22. Our dataset includes 2.5 ×\times as many clips, 2 ×\times as many frames, and 2 ×\times as many bounding boxes than CFC22, and 6 additional locations that expand OOD evaluation to more realistically reflect deployment needs.
Figure 4: Counting errors per location across training conditions. Balanced Sampling (ID) achieves the lowest nMAE at most locations, while holding out the target river degrades performance substantially at harder sites such as Eel and Dungeness. Directional errors (nMANE) are negative across nearly all conditions, indicating consistent undercounting of the dominant movement direction; Chemainus and Eel show the largest directional bias. Note, the nMNE value for Chemainus under Balanced Sampling (OOD) is a large outlier (3.67) and is truncated from the figure for readability.
Appendix figures & tables8 assets

Supplementary material from the paper’s appendix.

Appendix

 YOLOv5YOLOv26
MetricSmallMediumSmallMedium
Val [email protected]0.67800.69260.63100.6610
Val nMAE0.07290.07200.18570.1601
Val nMNE-0.0325-0.0355-0.2441-0.2003
Val nMANE-0.0372-0.0338-0.1595-0.1338
Test [email protected]0.73460.75460.65550.6854
Appendix
Table 2: Model performance comparison
ArchitectureVal mAP50Val nMAEVal nMNEVal nMANETest mAP50Test nMAETest nMNETest nMANE
YOLOv5m0.69260.0720-0.0355-0.03380.75460.0802-0.0397-0.0506
RT-DETRv2-S0.68920.0725-0.0317-0.03160.74800.0906-0.0433-0.0501
Appendix
Table 3: Detector architecture robustness. Detection and counting performance for YOLOv5m and RT-DETRv2-S on the validation and test sets.
LocationEpoch
Chemainus173
Dungeness516
Eel616
Elwha42
Kenai-Channel43
Kenai-Left20
Appendix
Table 4: Epoch Scaling for Target-Location Training
ConfigurationFramesEpochsUpsamplemAP50nMAEnMNEnMANE
10k, upsampling (selected)10k74✓0.740.07-0.04-0.05
10k, no upsampling10k74×\times0.770.11-0.05-0.07
1k, no upsampling1k734×\times0.690.090.08-0.05
Appendix
Table 5: Sampling strategy comparison. 10k no-upsampling achieves higher mAP50 performance, but 10k-upsampling has lower counting errors.
Val LocationmAP50
Leftbank0.80929
Rightbank0.90179
Appendix
Table 6: Train on Kenai-Leftbank Model performance comparison
 nMAEnMANEnMNE
RiverIDOODTargetIDOODTargetIDOODTarget
Nushagak0.0280.0350.014-0.0280.028-0.014-0.0280.028-0.014
Kenai-Right0.0290.0250.027-0.021-0.017-0.015-0.021-0.017-0.015
Kenai-Left0.0480.0690.052-0.033-0.036-0.048-0.034-0.041-0.048
Kenai-Channel0.1430.2860.1430.0300.0000.0300.032-0.1290.032
Elwha0.1470.2650.206-0.094-0.219-0.094-0.115-0.2690.038
Appendix
Table 7: Per-river counting performance under balanced sampling for in-distribution (ID), balanced sampling out-of-distribution (OOD) evaluation, and on the target river only.
Figure 5: Counting error (nMAE) per location across training conditions. Balanced Sampling (ID), which includes target-location data, achieves the lowest error for most locations, while holding out the target river (OOD) degrades performance substantially at harder locations such as Eel and Dungeness. Training on CFC22 locations alone fails catastrophically on Eel with nMAE ≈\approx 1.0, demonstrating that in-distribution benchmark scores do not predict deployment performance.
Figure 6: Example echogram. A raw sonar clip (left) and its corresponding echogram (right). While the five fish are barely visible in the noisy, raw imagery, the echogram clearly reveals the tracks over time.
More in this topic

Object Counting

Object counting estimates the number and locations of target instances in images or video, from dense crowds to fine-grained categories.

See all 61 papers

Explore similar work

CardsList
Oct 16, 2026

A Paired RGB-Sonar Dataset for Fish Monitoring with Consumer Sonar

Imaging sonar enables underwater observation in conditions where optical imaging is ineffective due to darkness and turbid water. However, imaging-sonar systems previously used for ecological monitoring have typically been expensive, limiting their accessibility. Consumer forward-looking sonar has the potential to make fish monitoring more accessible, but its noisy imagery makes annotation and the development of automated methods challenging. We introduce the Huntington and Caltech Koi RGB and Sonar dataset (HaCKRS), which pairs Garmin LiveScope imaging-sonar recordings with top-down RGB video of koi carp. The dataset contains more than 105,000 temporally matched frame pairs, spanning approximately 3.3 hours of recordings across four range and orientation configurations. We provide frame-wise metadata, sonar geometry and coordinate information, synchronization data for all four configurations, and planar registration for the three horizontal configurations. The release focuses on paired observations and calibration rather than task-specific fish annotations. We intend HaCKRS to support annotation and the development of fish detection, tracking, and counting methods, and investigations into the potential of consumer sonar for ecological monitoring.

Mourad Derbedrosian, Michael Hobley, Pietro Perona

California Institute of Technology · UC Santa Barbara

Multimodal DatasetUnderwater Imaging
AskPDF
Aug 21, 2026

WildFin: An In-the-Wild Dataset for Fish Behavioral Recognition

Recent advances in field technology have led to a massive influx of in-the-wild video data for ecological science. The primary bottleneck in leveraging this data is the high cost of expert annotation. While computer vision offers a potential solution, current models frequently fail when deployed in complex marine environments. To characterize these failures, we introduce WildFin, a novel benchmark for fish behavior recognition collected and annotated by ecologists. WildFin spans two critical real-world paradigms: stationary cameras monitoring groups of fish and dynamic divers following individual subjects. The dataset represents a massive curation effort, involving 1,350 hours of fieldwork and 600 hours of expert annotation to produce 9 hours of behavioral data with over 2 million frame-by-frame labels. We benchmark modern vision foundation models and quantify tradeoffs between static and spatiotemporal architectures, revealing the substantial gap that remains between current model capabilities and the demands of real-world underwater behavioral analysis. Project website: https://team-wildfin.github.io/.

Abigail G. Grassick, Jerome Tze-Hou Hsu, Ethan Lin +10

Cornell University, 14850 Ithaca, NY · University of Colorado Boulder, 80309 Boulder, CO · HHMI Janelia Research Campus, 20147, Ashburn VA

Vision Foundation ModelsVideo Action RecognitionAnimal Behavior Analysis
AskPDF
May 11, 2026

Automated high-frequency quantification of fish communities and biomass using computer vision

Quantifying fish community structure is essential for understanding biodiversity and ecosystem responses in a changing environment, yet existing survey methods provide limited high-frequency, quantitative observations. Conventional approaches, including catch-based methods, underwater visual censuses, and environmental DNA metabarcoding, either require intensive labor or lack reliable estimates of abundance and biomass. Here, we develop an automated framework for quantifying fish communities from underwater video using computer vision. Using videos acquired with a custom-made stereo camera system, the framework integrates deep learning-based fish identification, multi-object tracking, and 3D reconstruction to estimate species-level abundance and biomass. We applied the approach to a reef fish community over a 20-day period with hourly daytime observations, revealing dynamic fluctuations in species richness, abundance, and biomass associated with changes in species composition. By comparing fish communities estimated from visual census and environmental DNA surveys, we demonstrate that our method provides complementary strengths for continuous, non-invasive, and quantitative monitoring of consistently observed species. This approach provides a scalable foundation for long-term monitoring and advances the capacity to resolve fine-scale temporal dynamics in fish communities.

Kota Ishikawa, Takuma Masui, Keita Koeda +3

Graduate School of Life Sciences, Tohoku University, Sendai, Japan · Advanced Institute for Marine Ecosystem Change (WPI-AIMEC), Tohoku University, Sendai, Japan · Graduate School of Science and Engineering, University of the Ryukyus, Okinawa, Japan +1

Wildlife Monitoring
AskPDF