Organizations: Zhejiang University · Harbin Engineering University · Nanyang Technological University · Harbin Institute of Technology · University of Auckland
Underwater navigation requires translating visual spatial cues into motion, yet marine vision-language datasets rarely include paired robot trajectories. We present NereidVLA, a vision-language-action model that transfers marine spatial knowledge to language-conditioned 3D navigation. We first train Sea2Know on approximately two million marine examples to learn target grounding, spatial relations, and scene structure without paired robot-motion supervision. We then adapt it using expert trajectories executed under simulated underwater vehicle dynamics. Given RGB histories and a language instruction, NereidVLA predicts discrete motion tokens that decode into local 3D trajectories for closed-loop tracking. Evaluation combines an independent six-dimensional underwater navigation understanding benchmark with offline prediction and closed-loop navigation. Under matched navigation training and execution settings, marine spatial pretraining increases simulation success from 72.4% to 84.4%. Without fine-tuning on the real test environment, NereidVLA achieves 78.75% success in laboratory-pool robot trials, compared with 47.50% for InternVLA-N1. These results show that marine spatial supervision can support downstream motion learning and physical underwater navigation.
Figures & tables
Figure 1: Overview of the two-stage learning framework for NereidVLA. Marine spatial learning produces Sea2Know, which initializes the navigation model. Expert demonstrations executed under simulated vehicle dynamics supervise local 3D trajectory prediction, followed by closed-loop tracking with a low-level feedback controller.
Figure 2: Navigation architecture and trajectory supervision. A Sea2Know-initialized navigation VLA predicts discrete motion tokens from four historical RGB observations and a task instruction. Executed expert trajectories provide token-level supervision, and codebook decoding reconstructs local 3D waypoints for closed-loop tracking.
Figure 3: Underwater Navigation Understanding Benchmark. The six panels show representative examples and question proportions for target grounding, spatial relations, passage structure, traversability, temporal understanding, and local action decisions. Each example pairs visual observations with a question, answer options, and the reference answer.
Method
TGA ↑
SRA ↑
TSA ↑
TPA ↑
TUA ↑
LADA ↑
API-based General Models
DeepSeek V4.1 Flash [ 8 ]
87.06
87.66
87.26
86.82
75.48
34.88
Doubao Seed 2.1 Pro [ 5 ]
87.06
84.34
84.91
84.58
72.36
32.43
GPT-6-Sol [ 17 ]
91.54
82.44
80.90
77.73
69.35
33.24
Claude Opus 5 [ 3 ]
85.07
82.92
84.20
83.15
71.15
26.70
Open-source General VLMs
Table 1: Task-oriented underwater VQA evaluation (%). Bold marks the column best; dark/light blue marks the best/second-best Sea2Know results. Parentheses show point changes from size-matched Qwen3-VL backbones (red/green: gains/drops).
Model
ADE ↓
FDE ↓
Z MAE ↓
ADE P95 ↓
FDE P95 ↓
InternVLA-N1
0.20008
0.39498
0.05735
0.67412
1.26874
Qwen3-VL-4B
0.23568
0.45514
0.06190
0.74359
1.37255
Qwen3-VL-2B
0.28983
0.53348
0.06178
0.91908
1.65784
Qwen3-VL-8B
0.29121
0.55396
0.06181
0.91134
1.66065
Qwen3-VL-32B
0.19766
0.38996
0.05219
0.64042
1.20181
NAUTILUS (7B)
0.21748
0.42234
0.05230
0.74137
1.37125
Table 2: Offline trajectory prediction results. All metrics are lower-is-better. Boldface marks the best score in each column.
Method
Task success rate (%) ↑
Overall performance
Obstacle avoidance
Target search / approach
Pipeline navigation
Cave navigation
SR (%) ↑
Jerk RMS ↓
Grid-SPL (%) ↑
U0 †
5.60
19.20
10.40
13.60
12.20
4.1906
10.71
π0.5 -USIM † [ 20 , 9 ]
0.80
8.00
4.80
0.80
3.60
1.7826
3.09
Qwen3-VL-8B
84.00
52.00
71.20
82.40
72.40
1.1247
71.52
Qwen3-VL-32B
76.00
44.00
59.20
65.60
61.20
1.4338
59.47
SimpleVLN (8B)
74.40
38.40
52.00
41.60
51.60
1.9478
49.82
Table 3: Closed-loop navigation in simulation. Each model is evaluated on 500 trials (125 per task). Overall SR averages the success rates across the four equally sized tasks. Bold marks the best score in each column. † : original checkpoints without navigation adaptation.
Figure 4: Closed-loop navigation examples. (a) Target search and in-pipe navigation in simulation, showing successive frames, onboard views, and predicted and executed trajectories. (b) Real-robot goal approach with obstacles and narrow-passage avoidance, shown through external and onboard views. Time labels indicate stages of each execution.
Method
TA
OA
GA
NP
Overall
Qwen3-VL-8B
35.00
80.00
15.00
10.00
35.00
InternVLA-N1
60.00
85.00
30.00
15.00
47.50
Ours
90.00
95.00
70.00
60.00
78.75
Table 4: Real-robot navigation success rates (%, ↑ ). TA: target approach; OA: obstacle avoidance; GA: goal approach with obstacles; NP: narrow-passage avoidance. Overall is the four-task mean.
National Key Laboratory of Human-Machine Hybrid Augmented Intelligence, National Engineering Research Center for Visual Information and Applications, and Institute of Artificial Intelligence and Robotics, Xi’an Jiaotong University · The State Key Laboratory of Internet of Things for Smart City, Centre for Artificial Intelligence and Robotics, University of Macau · Xiaomi EV +2