Organizations: School of Artificial Intelligence, Sun Yat-sen University, Zhuhai, China · School of Electronic Information, Central South University, Hunan, China · Guangxi Key Laboratory of Embedded Technology and Intelligent System, Guilin University of Technology, Guilin, China · The Technology Innovation Center for Collaborative Applications of Natural Resources Data in GBA, MNR, Guangzhou, China
Underwater imagery records rich ecological information and supports applications such as science communication, fishery monitoring, ecological conservation, and oceanographic research. Beyond general scene understanding, these applications require models to recognize fine-grained biological traits and reason over spatially grounded evidence. However, existing underwater datasets often provide coarse category labels, generic captions, or total-count supervision, which limits their use for organism-level analysis and ecological reasoning. We introduce Nereus, a large-scale dataset for Underwater Fine-grained Perception that jointly supports Fine-Grained Object Attribute Understanding and Grounded Counting Perception. Nereus contains 88K images and 2.68M question-answer pairs, including annotations for 136K marine organism objects across 2,690 species and 1,515 genera, as well as 65K point annotations and 67K grounded counting QA pairs. Experiments with representative multimodal models show that Nereus improves downstream adaptation over existing underwater-domain data and provides a foundation for future research on fine-grained marine perception.
Figures & tables
Figure 1: Overview of Nereus. The dataset connects fine-grained organism attributes with grounded counting cues and degradation-aware underwater perception.
Datasets
Venue
Img
Annotation Modality
Supported tasks
Object-level Attr. Annotation
QA Pair
Instance Desc.
Detection
VQA
Caption
Fine-Grained Attr. Understanding
Counting Perc.
MarineInst20M [ 1 ]
ECCV 2024
2.42M
×
×
✓
✓
×
✓
×
×
NautData [ 2 ]
NeurIPS 2025
158K
×
✓
✓
✓
✓
✓
×
×
Fish-Vista [ 3 ]
CVPR 2025
69.3K
✓
×
×
×
×
×
✓
×
CoralVQA [ 4 ]
NeurIPS 2025
12.8K
✓
✓
×
×
✓
×
✓
×
UWBench [ 5 ]
arXiv 2025
15.0K
×
✓
✓
×
✓
✓
×
×
Table 1: Comparison between Nereus and existing underwater understanding datasets. Nereus contains 88K images, 2.68M question-answer pairs, 136K objects, and 65K point annotations, and mainly supports two tasks highly related to marine downstream applications: Fine-Grained Object Attribute Understanding and Grounded Counting Perception.
Figure 2: Motivation of Nereus. The gap between underwater application needs and existing resources motivates a unified dataset for fine-grained object understanding, grounded counting, and degradation-aware analysis.
Figure 3: Overview statistics of Nereus. Data-source composition and biological diversity for fine-grained object understanding, alongside semantic-region and camouflage-mechanism distributions for grounded counting.
Figure 4: Construction pipeline of Nereus. (a) Fine-grained annotation pipeline: Multi-model discussion and arbitration generate structured biological attributes using object crops, spatial context, and biological priors. (b) Counting perception data pipeline: Point annotations are transformed into region-aware, camouflage-aware, and density-grounded QA pairs.
Model
Dataset
Identity
Struct. Morph.
Appear. Surf.
Eco. Health
Pose Inter.
Overall
Acc ↑
Acc ↑
Sim ↑
Acc ↑
Sim ↑
Acc ↑
Sim ↑
Acc ↑
Sim ↑
Acc ↑
Sim ↑
BioCLIP2
Zero-Shot
22.40
6.70
19.01
5.96
10.75
12.96
13.52
3.05
23.36
10.69
22.40
GPT-5
Zero-Shot
18.55
20.21
53.90
31.68
50.98
60.27
81.74
25.27
49.85
30.14
59.59
Gemini 2.0 Flash
Zero-Shot
13.89
19.97
56.26
25.32
48.36
13.47
69.97
18.08
49.41
18.22
58.60
Gemini 3.1 Pro
Zero-Shot
32.22
28.88
61.77
34.26
55.52
64.90
88.02
33.99
53.01
37.99
65.42
LLaVA-OneVision-7B
Zero-Shot
7.07
8.16
40.81
20.06
41.32
4.97
78.41
7.84
52.08
9.28
48.39
Table 2: Evaluation results on Fine-Grained Object Attribute Understanding across five attribute categories. Struct. Morph., Appear. Surf., Eco. Health, and Pose Inter. denote Structural Morphology, Appearance and Surface, Ecological Health, and Pose and Interaction, respectively. Nereus fine-tuning uses the structured attribute set as the supervision signal. Best values are shown in bold within each backbone.
Track A: RGB counting perception
Backbone
Setting
MAE ↓
Exact ↑
Acc@ ± 5% ↑
Cell ↑
Camou. F1 ↑
CountGD [ 48 ]
ZS
32.33
6.64
14.35
67.67
—
CLIP-Count [ 49 ]
ZS
81.20
1.06
3.90
33.22
—
Gemini 2.0 Flash
ZS
52.80
9.08
11.12
28.79
18.94
P-ZS
50.50
8.86
13.29
15.50
17.09
Full-Prompt
55.50
9.04
12.68
7.62
12.73
Table 3: Nereus counting and density-evidence benchmark. Results are grouped by evaluation track, and best values within each track are shown in bold. Acc@ ± 5% and Acc@ ± 10% use relative counting-error tolerances of 5% and 10%, respectively. For trainable VLMs, QAS combines Track-A counting, spatial, and camouflage supervision, whereas Full retains all QAS components and adds Track-B density, hotspot, and ecological-grounding supervision (Appendix B.6 ). All models use the same Nereus-specific split; evaluation under the official IOCfish5K protocol and the split mapping are provided in Appendix C.1 .
Supervisory Signal
Identity
Struct. Morph.
Appear. Surf.
Eco. Health
Pose Inter.
Overall
Acc ↑
Acc ↑
Sim ↑
Acc ↑
Sim ↑
Acc ↑
Sim ↑
Acc ↑
Sim ↑
Acc ↑
Sim ↑
QA Pairs
10.98
33.64
49.80
33.38
52.00
77.90
82.00
52.13
70.80
39.08
55.20
Object-level Caption
20.99
17.18
57.70
27.87
55.50
3.94
64.60
11.93
68.00
16.78
61.60
Structured Attribute Set
58.06
30.84
64.00
50.00
67.10
39.92
87.60
34.41
72.10
41.55
73.10
Table 4: Ablation study on different supervisory signals for Fine-Grained Object Attribute Understanding using Qwen2.5-VL-7B. Object-level Caption denotes natural-language object descriptions. Acc is averaged over all attribute questions within each category.
Figure 5: Degradation examples.
Appendix figures & tables18 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 6: Examples of Nereus applications, covering object-level attribute understanding, region-aware counting, camouflage-aware counting, density-hotspot reasoning, and underwater degradation annotation.
Figure 7: Overview of the Nereus attribute taxonomy. The taxonomy organizes the 136-dimensional structured attribute set into identity, structural morphology, appearance and surface, condition and environment, and pose, visibility, and interaction categories.
Score
Criterion and descriptive requirement
10
Perfectly accurate, complete, internally consistent, and highly aligned with image evidence.
9
Accurate overall, with only trivial imperfections that do not affect the core understanding.
8
Mostly accurate; minor issues, such as slight terminology bias, but no critical errors.
7
Mostly correct; some secondary attributes show deviations, but core understanding is intact.
6
Partially correct; the core category is right, but notable errors or omissions affect fine-grained details.
5
Borderline usable; only basic attributes are trustworthy, with frequent deviations.
Appendix
Table 5: 10-point scale for automated annotation quality assessment.
Grade
Mapping score
Qualitative criterion
A
10
Accurate: reflects visual features well, is consistent with the image, and has only minor flaws.
B
7
Acceptable: generally correct, captures main features, and supports basic understanding.
C
4
Deficient: partially correct but contains obvious errors and has limited reference value.
D
1
Failed: inconsistent with the image, with severe errors or internal conflicts.
Appendix
Table 6: Four-level scale for manual annotation accuracy and mapping.
Score range
Semantic alignment criterion
90–100
Semantically equivalent, synonymous, or fully correct.
70–89
Correct core meaning with minor differences in granularity or phrasing.
50–69
Correct direction or intent but imprecise or slightly vague.
30–49
Related to the topic but factually or logically clearly wrong.
10–29
Nearly irrelevant; contains only marginal keywords related to the domain.
0–9
Completely wrong, empty, or nonsensical.
Appendix
Table 7: 100-point scale for SIM-based semantic similarity assessment.
Setting
Identity
Struct. Morph.
Appear. Surf.
Eco. Health
Pose Inter.
Overall
Acc ↑
Acc ↑
Sim ↑
Acc ↑
Sim ↑
Acc ↑
Sim ↑
Acc ↑
Sim ↑
Acc ↑
Sim ↑
Original Img.
58.06
30.84
64.00
50.00
67.10
39.92
87.60
34.41
72.10
41.55
73.10
Degradation Prior + Original Img.
56.30
29.30
66.20
48.16
66.80
40.25
89.50
35.95
76.20
40.51
74.60
Degradation Prior + Enhanced Img.
51.95
25.27
62.30
45.56
66.20
38.18
86.40
36.12
76.40
37.36
71.80
Appendix
Table 8: Ablation study on degradation-aware training strategies for Ultra Fine-Grained Object Attribute Understanding using Qwen2.5-VL-7B.
Attribute
Method
Acc ↑
Balanced Acc ↑
Turbidity
Linear probe
83.5
82.0
Turbidity
KNN probe
79.0
73.5
Color cast
Linear probe
89.0
89.7
Color cast
KNN probe
78.5
79.6
Appendix
Table 9: Exploratory degradation classification using UIQI-derived features. All values are percentages.
Training data / setting
MAE ↓
RMSE ↓
Exact ↑
Acc@ ± 5% ↑
Acc@ ± 10% ↑
Densest Cell ↑
Camou. F1 ↑
Count
Nautilus counting SFT
39.96
158.50
9.65
16.30
24.53
6.11
8.48
1129
Nereus count-only SFT
38.57
89.59
9.92
16.83
26.66
6.11
7.41
1129
Nereus QA+structured-final SFT
31.98
78.25
10.01
18.51
29.85
49.16
52.85
1129
Nereus full SFT
36.19
84.74
10.36
16.56
25.33
47.39
54.06
1129
Appendix
Table 10: Expanded transfer comparison on the Track A counting test split. Nereus variants are compared with Nautilus counting SFT under the same Qwen2.5-VL-7B backbone.
Nereus setting
Δ MAE
Δ RMSE
Δ Acc@ ± 5%
Δ Acc@ ± 10%
Δ Densest Cell
Δ Camou. F1
Count-only SFT
-1.39
-68.91
+0.53
+2.13
+0.00
-1.08
QA+structured-final SFT
-7.98
-80.25
+2.21
+5.31
+43.05
+44.37
Full SFT
-3.77
-73.76
+0.27
+0.80
+41.28
+45.57
Appendix
Table 11: Relative changes over Nautilus counting SFT. Negative values are better for MAE and RMSE; positive values are better for all accuracy and F1 metrics.
Training Setting
MAE ↓
Exact Acc. ↑
Acc@ ± 5% ↑
Densest Cell Acc. ↑
Camouflage F1 ↑
Zero-shot
87.38
2.22
2.22
22.85
–
Prompted zero-shot
167.55
0.00
1.16
20.73
0.00
Count-only SFT
31.72
10.01
19.04
20.81
–
QA+structured-final SFT
25.99
10.45
20.46
52.70
50.02
Full Count SFT
30.48
9.74
18.95
51.90
52.59
Appendix
Table 12: Image-only ablation on InternVL2.5-8B with RGB input. Best values are shown in bold.
Backbone
Setting
MAE ↓
RMSE ↓
Exact ↑
Acc@ ± 5% ↑
Acc@ ± 10% ↑
Densest Cell ↑
CountGD
ZS
32.33
80.16
6.64
14.35
25.69
67.67
CLIP-Count
ZS
81.20
149.04
1.06
3.90
7.79
33.22
Appendix
Table 13: Specialized counting baselines on the Track A test split. Best values among the two specialized baselines are shown in bold.
Figure 8: Qualitative examples for total-count and grid-count QA. The examples compare model outputs with grounded counting annotations, including total fish count and densest-region prediction.
Figure 9: Qualitative examples for camouflage visibility and density-hotspot QA. The examples show visibility-state counting and hotspot reasoning from RGB images and density evidence.
Figure 10: Qualitative examples for density-response interpretation. The examples cover ecological inference from low-response heatmap regions and cross-image reasoning over high-response density cues.
Metric
Track
Computation
Interpretation
Count MAE ↓
A
N1∑i∣c^i−ci∣
Average absolute counting error; the primary RGB counting metric.
Count RMSE ↓
A
N1∑i(c^i−ci)2
Count error with stronger penalty for severe over-counting or under-counting.
Exact Acc ↑
A
N1∑i1[c^i=ci]
Strict exact-count accuracy; any count deviation is incorrect.
Acc@ ± 5% ↑
A
N1∑i1[∣c^i−ci∣≤0.05ci]
Count accuracy within ± 5% of the ground-truth count.
Acc@ ± 10% ↑
A
N1∑i1[∣c^i−ci∣≤0.10ci]
Count accuracy within ± 10% of the ground-truth count.
Densest Cell Acc ↑
A
N1∑i1[r^i=ri⋆]
Accuracy of predicting the densest cell in a 3×3 grid; evaluates spatial counting.
Appendix
Table 14: Metrics used in the counting and density-evidence benchmark.
Ground-truth count
Samples
Proportion
0–50
526
46.59%
51–100
204
18.07%
101–200
183
16.21%
>200
216
19.13%
Appendix
Table 15: Ground-truth count distribution of the 1,129 Track-A evaluation samples.
Setting
Component
Track
Records
Ratio within setting
QAS
Region/grid counting QA
A
15,780
33.33%
QAS
Camouflage QA
A
15,780
33.33%
QAS
Structured total count
A
3,945
8.33%
QAS
Structured grid count
A
3,945
8.33%
QAS
Structured camouflage
A
3,945
8.33%
QAS
Structured final summary
A
3,945
8.33%
Appendix
Table 16: Task composition and record-level proportions of QAS and Full.
Model
Setting
MAE ↓
RMSE ↓
Exact ↑
Acc@ ± 5% ↑
Acc@ ± 10% ↑
Failure ↓
IOCFormer
Public checkpoint
17.52
43.13
7.45
22.95
39.95
0
InternVL2.5-8B QAS
Official train/validation split
31.16
82.31
10.65
19.50
30.45
0
Appendix
Table 17: Comparison under the official IOCfish5K protocol.
Accurate segmentation of marine organisms is vital for biodiversity monitoring and ecological assessment, yet existing datasets and models remain largely limited to terrestrial scenes. To bridge this gap, we introduce AquaOV255, the first large-scale and fine-grained underwater segmentation dataset containing 255 categories and over 20K images, covering diverse categories for open-vocabulary (OV) evaluation. Furthermore, we establish the first underwater OV segmentation benchmark, UOVSBench, by integrating AquaOV255 with five additional underwater datasets to enable comprehensive evaluation. Alongside, we present Earth2Ocean, a training-free OV segmentation framework that transfers terrestrial vision--language models (VLMs) to underwater domains without any additional underwater training. Earth2Ocean consists of two core components: a Geometric-guided Visual Mask Generator (GMG) that refines visual features via self-similarity geometric priors for local structure perception, and a Category-visual Semantic Alignment (CSA) module that enhances text embeddings through multimodal large language model reasoning and scene-aware template construction. Extensive experiments on the UOVSBench benchmark demonstrate that Earth2Ocean achieves significant performance improvement on average while maintaining efficient inference.
Bingyu Li, Tao Huo, Da Zhang +3
Institute of Artificial Intelligence (TeleAI), China Telecom, China · University of Science and Technology of China, China · Northwestern Polytechnical University, China
Fine-grained visual understanding and high-level reasoning in real-world open-water environments remain under-explored due to the lack of dedicated benchmarks. We introduce MARINER, a comprehensive benchmark built under the novel Entity-Environment-Event (3E) paradigm. MARINER contains 16,629 multi-source maritime images with 63 fine-grained vessel categories, diverse adverse environments, and 5 typical dynamic maritime incidents, covering fine-grained classification, object detection, and visual question answering tasks. We conduct extensive evaluations on mainstream Multimodal Large language models (MLLMs) and establish baselines, revealing that even advanced models struggle with fine-grained discrimination and causal reasoning in complex marine scenes. As a dedicated maritime benchmark, MARINER fills the gap of realistic and cognitive-level evaluation for maritime multimodal understanding, and promotes future research on robust vision-language models for open-water applications. Appendix and supplementary materials are available at https://lxixim.github.io/MARINER.
Xingming Liao, Ning Chen, Muying Shu +5
Guangdong University of Technology, Guangzhou, China
Robust perception is essential for the deployment of autonomous underwater robots. However, optical cameras become unreliable under poor illumination and backscatter. Forward looking (2D) acoustic sensors remain effective under these conditions, but they measure range and bearing while leaving elevation unresolved, creating an ambiguity that prevents individual sonar returns from being localized in three dimensional (3D) space. This complicates the sensor use for 3D scene understanding and precise object detection. We introduce uScenes, a multimodal underwater dataset containing synchronized 3D multibeam sonar point clouds and RGB imagery. The dataset contains 110 scenes and 95,834 synchronized observation, representing 277.6 minutes of data collected across multiple field sessions. uScenes establishes a foundation for underwater sensor fusion, cross modal representation learning and 3D scene understanding. Code and datasets are given at https://github.com/era-research-lab/uScenes.
Trung Tien Dong, Zhenqi Wu, Aditya Penumarti +4
Embodied Robotics and Automation Lab, University of South Florida, Tampa, FL 33610, USA · Mechanical and Aerospace Engineering Department, University of Florida, Gainesville, FL 32611, USA · Naval Architecture and Ocean Engineering at the Seoul National University, Seoul 08826, South Korea