MARINER: A 3E-Driven Benchmark for Fine-Grained Perception and Complex Reasoning in Open-Water Environments

Xingming Liao; Ning Chen; Muying Shu; Yunpeng Yin; Peijian Zeng; Zhuowei Wang; Nankai Lin; Lianglun Cheng

cs.CVApr 9, 2026

MARINER: A 3E-Driven Benchmark for Fine-Grained Perception and Complex Reasoning in Open-Water Environments

Authors: Xingming Liao, Ning Chen, Muying Shu, Yunpeng Yin, Peijian Zeng, Zhuowei Wang, Nankai Lin, Lianglun Cheng

Organizations: Guangdong University of Technology, Guangzhou, China

Abstract

Fine-grained visual understanding and high-level reasoning in real-world open-water environments remain under-explored due to the lack of dedicated benchmarks. We introduce MARINER, a comprehensive benchmark built under the novel Entity-Environment-Event (3E) paradigm. MARINER contains 16,629 multi-source maritime images with 63 fine-grained vessel categories, diverse adverse environments, and 5 typical dynamic maritime incidents, covering fine-grained classification, object detection, and visual question answering tasks. We conduct extensive evaluations on mainstream Multimodal Large language models (MLLMs) and establish baselines, revealing that even advanced models struggle with fine-grained discrimination and causal reasoning in complex marine scenes. As a dedicated maritime benchmark, MARINER fills the gap of realistic and cognitive-level evaluation for maritime multimodal understanding, and promotes future research on robust vision-language models for open-water applications. Appendix and supplementary materials are available at https://lxixim.github.io/MARINER.

Figures & tables

Name	Source	Categories	Environment	Event	Cls.	Det.	VQA	Images	Instances
MS COCO ( Lin et al., 2014 )	Open-source	1	Limited	Static	✗	✓	✗	3,164	11,189
SeaShip ( Shao et al., 2018 )	Shore-based	6	Limited	Static	✓	✓	✗	31,455	40,077
Boat Re-ID ( Spagnolo et al., 2019 )	Shore-based	-	Limited	Static	✓	✗	✗	5,523	5,523
McShips ( Zheng and Zhang, 2020 )	Open-source	13	Diverse	Static	✓	✓	✗	14,709	26,529
ABOships ( Iancu et al., 2021 )	Shipboard	9	Limited	Limited	✓	✓	✗	9,880	41,967
SeaSAw ( Kaur et al., 2022 )	Shipboard	12	Diverse	Dynamic	✓	✓	✗	1.9M	14.6M

Table 1. Comparison of ship-related datasets in terms of source diversity, category scale, environmental coverage, event representation, task coverage, and dataset scale. Cls., Det., and VQA denote classification, detection, and visual question answering, respectively. Multi-source denotes that the dataset is collected from open-source imagery, self-developed electro-optical pod platform and unmanned aerial vehicle.

Figure 1. Normalized performance comparison of MARINER and other models ( Hurst et al., 2024 ; Qwen et al., 2025 ; Li et al., 2024a ; Zhu et al., 2025 ; Comanici et al., 2025 ) across the key metrics of the three benchmark tasks.

Figure 2. Representative examples from the MARINER dataset, covering fine-grained military and civilian ship categories, maritime incident scenarios, and adverse environmental conditions. The number in parentheses after each group name denotes the number of corresponding categories included in that group.

Figure 3. Distribution of the annotated instances within the MARINER, formulated specifically for the fine-grained classification and fine-grained detection tasks.

Figure 4. Example of a partially instantiated VQA sample from MARINER. Although the benchmark is designed with eight question categories, not every image can support all of them. The applicable question types are determined by the visible objects, scene composition, and reasoning cues available in each image. The figure shows several representative question-answer pairs for this image, where the bold underlined option denotes the ground-truth answer.

Model	Scale	Classification
Model	Scale	Accuracy	Macro F1	mAcc	Weighted F1
Human (expert)	-	82.52	-	-	-
GPT-4o	-	80.50	80.89	81.01	80.90
GPT-4.1	-	83.30	84.14	84.50	82.77
Gemini-2.5-Flash	-	82.60	80.36	81.73	81.79
Gemini-2.5-Pro	-	42.10	59.68	59.68	42.10

Table 2. Classification performance comparison of different models. Best performances for open-source models are highlighted in bold.

Model	Scale	Detection
Model	Scale	AP	AP 50	AP 75	AR	AP ${}_{\text{Mil}}$	AP ${}_{\text{Civ}}$
Human (expert)	-	68.30	-	-	-	49.12	87.48
GPT-4o (Civ)	-	1.46	6.06	0.34	3.12	-	3.12
GPT-4.1 (Civ)	-	4.28	12.98	1.75	7.75	-	9.14
Gemini-2.5-Flash	-	16.94	35.67	13.72	24.34	22.91	10.14
Gemini-2.5-Pro	-	59.42	68.46	60.62	72.68	69.04	48.47

Table 3. Detection performance comparison of different models. Best performances for open-source models are highlighted in bold. GPT-series models do not report results on military vessel categories, due to safety constraints related to military-target recognition.

		Attributes	Nav. Equ.	Civ. Equ.	OCR	Pers. Esti.	Spal. Rela.	Env. State	Ope. Stat.
Model	Overall	Perception				Space		Reasoning
Human (expert)	82.63	55.30	44.75	80.45	99.85	99.80	99.72	89.10	92.03
GPT-4o	68.91	82.97	47.64	81.19	78.22	62.70	35.26	85.32	77.97
GPT-4.1	73.77	87.46	51.89	85.07	84.16	72.73	32.37	90.01	86.45
Gemini-2.5-Flash	64.52	75.09	45.28	68.96	58.42	59.67	33.53	90.08	85.13
Gemini-2.5-Pro	73.97	86.02	58.96	84.18	86.39	68.53	31.79	92.38	83.48

Table 4. Results of different models on various task types in the VQA benchmark. The evaluation is reported across three major dimensions, including Perception, Space, and Reasoning. Best performances for open-source models are highlighted in bold.

Explore similar work

MarineEVT: Advancing Event-Centric Marine Video Understanding via Visual Tool Reasoning
Jul 27, 2026Tuan-An To, Yuk-Kwan Wong, Tuan-Anh Vu +2Visual Question Answering Video Understanding
The Hong Kong University of Science and Technology, Hong Kong, China · University of California, Los Angeles CA, USA · University of Electronic Science and Technology of China, China
WorldBench: A Challenging and Visually Diverse Multimodal Reasoning Benchmark
Jun 4, 2026Yida Yin, Harish Krishnakumar, Chung Peng Lee +9Multimodal Large Language Models Multimodal Reasoning
Princeton University · NYU · University of Waterloo +1
VisualWorldBench: A Fine-Grained Multi-Task Benchmark for Evaluating Visual World Knowledge in MLLMs
Oct 14, 2026Chen Chen, Zheming Liang, Jiaqi WangVLM Evaluation Knowledge-Based VQA
University of Science and Technology of China · Shanghai Innovation Institute

Jul 27, 2026J/K move · Enter open · S save

MarineEVT: Advancing Event-Centric Marine Video Understanding via Visual Tool Reasoning

Tuan-An To, Yuk-Kwan Wong, Tuan-Anh Vu, Ziqiang Zheng+1

The Hong Kong University of Science and Technology, Hong Kong, China · University of California, Los Angeles CA, USA · University of Electronic Science and Technology of China, China

Visual Question Answering Video Understanding Tool Use in VLMs VLM Reasoning Temporal Video Understanding

Open paper AskPDF arXiv

Recent Vision-Language Models (VLMs) have achieved remarkable success in visual understanding, driven by the growing availability of high-quality image-text pairs. However, the performance of VLMs often degrades in the video domain due to the essential need for temporal understanding and the scarcity of large-scale annotated video data. In this work, we focus on marine video understanding, which brings further challenges: first, it requires substantial domain expertise; and video VLMs usually struggle with localizing and interpreting critical information from marine videos, as the informative events are typically sparse, unpredictable, and unevenly distributed. To address these challenges, we carefully curate the first event-centric marine video understanding dataset called MarineEVT, which features 20K multi-task, video-level visual question-answering pairs spanning multiple dimensions of marine understanding and analysis. Meanwhile, based on MarineEVT, we decompose marine video understanding as an Event-centric Visual Tool-integrated Reasoning process EVT-R1 for short, where we leverage powerful visual tools to drive the model to localize and interpret critical information aligned with visual questions and human intent. To demonstrate its effectiveness, we compare EVT-R1 against 11 SOTA VLMs in different settings. EVT-R1 outperforms the top open-source and top commercial models by 5.22 and 11.09, respectively. MarineEVT and EVT-R1 lay the foundation for ecological discovery and marine education, fostering the development of VLMs capable of interpreting marine dynamics, reasoning about ecological interactions, and supporting sustainable ocean video understanding and analysis.

Abstract

Explore similar work

MarineEVT: Advancing Event-Centric Marine Video Understanding via Visual Tool Reasoning

WorldBench: A Challenging and Visually Diverse Multimodal Reasoning Benchmark

VisualWorldBench: A Fine-Grained Multi-Task Benchmark for Evaluating Visual World Knowledge in MLLMs