MUOT_3M: A 3 Million Frame Multimodal Underwater Benchmark and the MUTrack Tracking Method

Ahsan Baidar Bakht; Mohamad Alansari; Muhayy Ud Din; Muzammal Naseer; Sajid Javed; Irfan Hussain; Jiri Matas; Arif Mahmood

cs.CVFeb 20, 2026

MUOT_3M: A 3 Million Frame Multimodal Underwater Benchmark and the MUTrack Tracking Method

Authors: Ahsan Baidar Bakht, Mohamad Alansari, Muhayy Ud Din, Muzammal Naseer, Sajid Javed, Irfan Hussain, Jiri Matas, Arif Mahmood

Organizations: Khalifa University, Abu Dhabi, UAE · Czech Technical University, Prague, Czech Republic · Information Technology University, Lahore, Pakistan

Abstract

Underwater Object Tracking (UOT) is crucial for efficient marine robotics, large scale ecological monitoring, and ocean exploration; however, progress has been hindered by the scarcity of large, multimodal, and diverse datasets. Existing benchmarks remain small and RGB only, limiting robustness under severe color distortion, turbidity, and low visibility conditions. We introduce MUOT_3M, the first pseudo multimodal UOT benchmark comprising 3 million frames from 3,030 videos (27.8h) annotated with 32 tracking attributes, 677 fine grained classes, and synchronized RGB, estimated enhanced RGB, estimated depth, and language modalities validated by a marine biologist. Building upon MUOT_3M, we propose MUTrack, a SAM-based multimodal to unimodal tracker featuring visual geometric alignment, vision language fusion, and four level knowledge distillation that transfers multimodal knowledge into a unimodal student model. Extensive evaluations across five UOT benchmarks demonstrate that MUTrack achieves up to 8.40% higher AUC and 7.80% higher precision than the strongest SOTA baselines while running at 24 FPS. MUOT_3M and MUTrack establish a new foundation for scalable, multimodally trained yet practically deployable underwater tracking.

Figures & tables

Figure 2: MUOT-3M dataset diversity in terms of 16 Phylum categories, 124 families, and 677 fine-grained classes. 16 Phylum categories with corresponding representative families are shown. The distribution and labels of all classes are validated by the expert marine biologist . Non-marine species categories in MUOT-3M, i.e., human-related (diver, scuba) and non-biological (robot, ROVs), are not shown.

Figure 3: Performance degradation of SOTA trackers on WebUOT-1M [ 8 ] and MUOT-3M.

Figure 4: MUOT-3M is much larger than existing UOT datasets.

Dataset	Year	Videos	Classes	Attr.	Min Frames	Mean Frames	Max Frames	Total Frames	Total Duration	Depth Maps	Target Description	Train	Test
UOT32	2019	32	-	-	283	758	1,573	24K	0.27 h	✗	✗	✗	✓
UOT100	2022	104	-	3	264	702	1,764	74K	0.68 h	✗	✗	✗	✓
UTB180	2022	180	-	10	40	338	1,226	58K	0.53 h	✗	✗	✓	✓
VMAT	2023	33	17	13	438	2,242	5,550	74K	0.68 h	✗	✗	✗	✓
UVOT400	2023	400	50	17	40	688	3,273	275K	2.6 h	✗	✗	✓	✓
WebUOT-1M	2024	1,500	408	23	49	733	9,985	1.1M	10.5 h	✗	✓	✓	✓

Table 1: Underwater object tracking datasets. For the attributes provided with a specific dataset, see supplementary, Fig. S7.

Figure 5: MUTrack: Schematic of the proposed multimodal SAM-based tracking pipeline. Step 1 shows the pre-training process of visual-geometric and visual-textual alignments. Step 2 shows the proposed multimodal teacher tracker pre-trained on visual, geometric, and language cues, while Step 3 shows the proposed unimodal student tracker distilling knowledge from the multimodal teacher tracker.

Multimodal	Modalities				MUOT-3M	WebUOT-1M
Teacher (MMT)	I	E	D	L	(success Rate)	(success Rate)
Baseline SAM2 [ 38 ]	$\checkmark$	$\times$	$\times$	$\times$	60.12	61.11
Baseline SAM2 [ 38 ]	$\times$	$\checkmark$	$\times$	$\times$	61.20	62.23
Baseline SAM2 [ 38 ]	$\times$	$\times$	$\checkmark$	$\times$	58.55	59.21
MUTrack-MMT 2	$\times$	$\checkmark$	$\checkmark$	$\times$	66.12	65.59
MUTrack-MMT 3	$\times$	$\checkmark$	$\times$	$\checkmark$	65.70	63.66

Table 2: MUTrack teacher-only results with baseline SAM tracker. MMT stands for “ Multimodal Teacher ”.

Unimodal	Input	KD Losses				MUOT-3M	WebUOT-1M
Student (UMS)		$\mathcal{L}_{VG}$	$\mathcal{L}_{A}$	$\mathcal{L}_{VLA}$	$\mathcal{L}_{M}$	(Success Rate)	(Success Rate)
MUTrack-UMS	I	$\checkmark$	$\checkmark$	$\checkmark$	$\checkmark$	66.58	67.10
MUTrack-UMS 1	I	$\checkmark$	$\checkmark$	$\checkmark$	$\times$	64.22	65.31
MUTrack-UMS 2	I	$\checkmark$	$\checkmark$	$\times$	$\checkmark$	64.10	63.96
MUTrack-UMS 3	I	$\checkmark$	$\times$	$\checkmark$	$\checkmark$	64.50	66.09
MUTrack-UMS 4	I	$\times$	$\checkmark$	$\checkmark$	$\checkmark$	63.16	64.77

Table 3: Teacher is fixed and MUTrack student-only results are reported. UMS stands for “ Unimodal Student ”.

Trackers	MUOT-3M			WebUOT-1M			UTB180			UVOT400			UWCOT22
Trackers	S	P	NP	S	P	NP	S	P	NP	S	P	NP	S	P	NP
MUTrack (MMT)	68.10	69.93	86.66	68.79	70.33	88.20	73.77	74.33	81.69	72.10	75.87	80.67	79.40	81.66	83.22
MUTrack (UMS)	66.58	68.16	84.32	67.10	69.33	86.97	72.88	76.61	80.30	70.55	74.51	78.97	77.87	80.10	81.11
UOSTack	49.22	41.92	61.20	53.14	47.76	65.67	52.67	45.87	68.22	44.25	39.67	56.32	45.31	35.95	53.30
DUTrack	62.66	60.20	79.90	63.81	64.27	81.70	64.61	62.50	83.73	64.16	63.96	80.98	62.08	59.23	75.85
ATC	62.32	60.37	76.50	64.66	65.48	79.10	64.16	65.11	79.43	61.58	63.08	74.47	62.58	61.03	73.93

Table 4: Cross-dataset performance comparison of SOTA trackers: MMT stands for “ Multimodal Teacher ”, UMS is “Unimodal Student”, S is “Success rate”, P is “Precision” and NP is “Normalized Precision”.

Trackers	MUOT-3M			WebUOT-1M			UTB180			UVOT400			UWCOT22
Trackers	S	P	NP	S	P	NP	S	P	NP	S	P	NP	S	P	NP
MUTrack (MMT)	68.10	69.93	86.66	68.79	70.33	88.20	73.77	74.33	81.69	72.10	75.87	80.67	79.40	81.66	83.22
MUTrack (UMS)	66.58	68.16	84.32	67.10	69.33	86.97	72.88	76.61	80.30	70.55	74.51	78.97	77.87	80.10	81.11
DAM4SAM [ 69 ]	62.33	64.21	75.12	61.10	62.34	70.23	68.10	70.84	76.66	68.97	69.81	74.44	71.12	73.20	80.12
MUTrack (MMT-DAM4SAM)	70.13	71.23	87.33	69.01	72.43	88.32	74.56	75.14	83.61	74.56	76.44	82.34	80.12	82.08	84.61
MUTrack (UMS-DAM4SAM)	68.69	70.32	85.13	66.10	71.40	86.56	73.65	74.46	82.69	73.65	75.14	81.63	79.21	81.80	82.29

Table 5: MUTrack generalization with SOTA Trackers: Our MMT and UMS are plugged into the recent SOTA SAM-based tracking pipelines. MMT stands for “ Multimodal Teacher ”, UMS is “Unimodal Student”, S is “Success rate”, P is “Precision” and NP is “Normalized Precision”.

Panel of Figure 6 — Figure 6: Performance comparison of MUTrack with SOTA trackers using protocol I and II on MUOT-3M dataset. Norm. stands for “Normalized”.

Appendix figures & tables13 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 7: Overview of the MUOT-3M dataset collection and curation pipeline. The process includes three main stages: large-scale video collection from multiple sources, expert-driven curation to ensure underwater authenticity and visual quality, and annotation with multi-modal data generation. Each stage is standardized to maintain consistency and ensure high-quality tracking sequences across diverse marine environments.

Figure 8: Visualization of the 16 phylum-level classes in the MUOT-3M dataset, showcasing examples from each category alongside their multimodal representations. Each triplet illustrates the RGB frame with bounding box, corresponding depth map, and enhanced image, demonstrating the dataset’s diversity across marine environments and visual conditions. The language annotations describe scene context and species type and behavior, emphasizing the ecological and multimodal richness of MUOT-3M Dataset.

Figure 9: Hierarchical taxonomy of the MUOT-3M Dataset . The diagram illustrates the multi-level organization of MUOT-3M, comprising 16 phylum, 124 families, and 677 fine-grained species. Phylum nodes are shown in green, Family nodes in blue, and fine-grained classes in orange, representing the taxonomic depth of the dataset.

Unimodal	Depth Methods			MUOT-3M	WebUOT-1M
Student (UMS)	MiDaS	Depth Anything	DAC	(success Rate)	(success Rate)
MUTrack-MMT	$\checkmark$	$\times$	$\times$	66.58	67.10
MUTrack-MMT	$\times$	$\checkmark$	$\times$	66.12	66.96
MUTrack-MMT	$\times$	$\times$	$\checkmark$	66.40	67.06

Appendix

Table 6: Influence of depth estimation methods on MUTrack-UMS. The multimodal teacher is fixed, and MUTrack student-only results are reported. UMS stands for “Unimodal Student”

Unimodal	Enhancement Methods			MUOT-3M	WebUOT-1M
Student (UMS)	UTransformer	PixMamba	UVEB	(success Rate)	(success Rate)
MUTrack-MMT	$\checkmark$	$\times$	$\times$	66.58	67.10
MUTrack-MMT	$\times$	$\checkmark$	$\times$	66.50	66.93
MUTrack-MMT	$\times$	$\times$	$\checkmark$	66.52	67.02

Appendix

Table 7: Influence of underwater image enhancement methods on MUTrack-UMS. The multimodal teacher is fixed, and MUTrack student-only results are reported. UMS stands for “Unimodal Student”

Figure 10: Visualization of the 32 tracking attributes defined in the MUOT-3M dataset. Each attribute represents a distinct visual or environmental challenge, covering both generic and underwater-specific conditions. These attributes collectively characterize the full spectrum of appearance and motion variability in underwater tracking scenarios

Figure 11: Attribute-wise tracking performance (1–5)

Figure 12: Attribute-wise tracking performance (6–10)

Figure 13: Attribute-wise tracking performance (11–15)

Figure 14: Attribute-wise tracking performance (16–20)

Figure 15: Attribute-wise tracking performance (21–25)

Figure 16: Attribute-wise tracking performance (26–30)

Figure 17: Attribute-wise tracking performance (31–32)

Visual Object Tracking

Visual object tracking follows objects across video frames, maintaining their locations and identities despite motion, occlusion, and changing appearance.

See all 171 papers

UETrack: A Unified and Efficient Framework for Single Object Tracking
Mar 2, 2026Ben Kang, Jie Zhao, Xin Chen +5Multimodal Visual Object Tracking Visual Object Tracking
Dalian University of Technology · City University of Hong Kong
uScenes: A Multimodal RGB and 3D Sonar Dataset for Underwater Robot Perception
Aug 28, 2026Trung Tien Dong, Zhenqi Wu, Aditya Penumarti +4Underwater Robotics 3D Scene Understanding
Embodied Robotics and Automation Lab, University of South Florida, Tampa, FL 33610, USA · Mechanical and Aerospace Engineering Department, University of Florida, Gainesville, FL 32611, USA · Naval Architecture and Ocean Engineering at the Seoul National University, Seoul 08826, South Korea
Unified Multimodal Visual Tracking with Dual Mixture-of-Experts
May 5, 2026Lingyi Hong, Jinglun Li, Xinyu Zhou +6Cross-Modal Learning Multimodal Visual Object Tracking
Shanghai Key Lab of Intelligent Information Processing, College of Computer Science and Artificial Intelligence, Fudan University

Mar 2, 2026J/K move · Enter open · S save

UETrack: A Unified and Efficient Framework for Single Object Tracking

Ben Kang, Jie Zhao, Xin Chen, Wanting Geng+4

Dalian University of Technology · City University of Hong Kong

Multimodal Visual Object Tracking Visual Object Tracking

Open paper AskPDF arXiv

With growing real-world demands, efficient tracking has received increasing attention. However, most existing methods are limited to RGB inputs and struggle in multi-modal scenarios. Moreover, current multi-modal tracking approaches typically use complex designs, making them too heavy and slow for resource-constrained deployment. To tackle these limitations, we propose UETrack, an efficient framework for single object tracking. UETrack demonstrates high practicality and versatility, efficiently handling multiple modalities including RGB, Depth, Thermal, Event, and Language, and addresses the gap in efficient multi-modal tracking. It introduces two key components: a Token-Pooling-based Mixture-of-Experts mechanism that enhances modeling capacity through feature aggregation and expert specialization, and a Target-aware Adaptive Distillation strategy that selectively performs distillation based on sample characteristics, reducing redundant supervision and improving performance. Extensive experiments on 12 benchmarks across 3 hardware platforms show that UETrack achieves a superior speed-accuracy trade-off compared to previous methods. For instance, UETrack-B achieves 69.2% AUC on LaSOT and runs at 163/56/60 FPS on GPU/CPU/AGX, demonstrating strong practicality and versatility. Code is available at https://github.com/kangben258/UETrack.

Abstract

Visual Object Tracking

Explore similar work

UETrack: A Unified and Efficient Framework for Single Object Tracking

uScenes: A Multimodal RGB and 3D Sonar Dataset for Underwater Robot Perception

Unified Multimodal Visual Tracking with Dual Mixture-of-Experts