Embodied at Sea: Imitation Learning for Multi-Task Underwater Manipulation

Greta Perez-Haiek; Luca Macesanu; Kevin Angers; Mohammed Imam; Timothy Gambin; Christopher Clark

cs.ROOct 28, 2026

Embodied at Sea: Imitation Learning for Multi-Task Underwater Manipulation

Authors: Greta Perez-Haiek, Luca Macesanu, Kevin Angers, Mohammed Imam, Timothy Gambin, Christopher Clark

Organizations: New York University · University of Malta

Abstract

Behavior Cloning has demonstrated promising generalization capabilities for robotic manipulation in terrestrial environments. However, marine visual distortions and dynamic currents create a severe domain shift that remains a key bottleneck in adapting generalist architectures to underwater field manipulation. To address this, 1616 episodes of underwater manipulation data were collected in calm and turbulent waters and were used to train 14 generalist Vision-Language-Action (VLA) models (DynamicVLA) with varying proprioception observation configurations. Across three tasks on 1260 gradable on-hardware rollouts in a pool, 10 models outperformed the baseline, with the highest-performing model achieving a 93.3% overall success rate, including partial successes. Results are also introduced from a case study evaluating the performance of a single-task Action Chunking Transformer (ACT) and a generalist VLA (SmolVLA) policy for two tasks in marine archaeology and conservation: grasping and mooring. Experiments are performed on an ROV on and around the WWII wreck Coralita X127 and at Manoel Island Marina, Malta. Across 748 collected episodes and 508 gradable on-hardware rollouts, the ACT policy substantially outperformed SmolVLA (47.9% vs. 7.9% pooled success rate), earning a 66.7% success rate on the most successful task at the marine archaeological site. To enable further contributions to embodied intelligence in marine field robotics, the BLUEMBODIED KIT used to collect data, train models, and deploy policies on the ROV has been open sourced.

Figures & tables

Panel of Figure 1 — Fig. 2: Pool Deployment: Experimental tasks evaluated in a controlled environment. Data collection in the Pool (Control Environment) was fully illuminated with a split (see Fig. 3 ) between calm and turbulent waters created by a human volunteer (see Fig. 5 ). Rod locations and color order were randomized between episodes. Post location stayed relatively the same, but initial robot and ring configurations varied between episodes as well.

Fig. 3: Pool Deployment: Dataset split. Training data collected in the pool environment incorporates multi-modal observations and human-made turbulent waters. The dataset is broken down into two conditions: a control condition (calm waters) and a turbulent condition.

Panel of Figure 3 — Fig. 4: Marine Deployment: Experimental tasks evaluated across different underwater environments. Data collection in the Pool (Control Environment) was fully illuminated with a split between calm (Control Condition) and turbulent waters. Data collection at the Marina and Coralita X127 shipwreck took place during both day and night, featuring varying disturbances (see Fig. Embodied at Sea: Imitation Learning for Multi-Task Underwater Manipulation ).

Fig. 5: Pool Deployment: calm vs. turbulent conditions. Mean attitude and few choice IMU observations plotted over all Grab the Red Rod episodes (see Fig. 3 ). Since all models (except the baseline) observe roll, pitch, and yaw, these measurements illustrate how turbulence manifests in the observation space.

Fig. 6: Modified DynamicVLA Architecture. The policy outputs a six-dimensional action at each step: four motion commands and two tool commands. The motion commands are sent as MAVLink MANUAL_CONTROL messages to the BlueROV2’s onboard ArduSub autopilot, which converts them into individual thrust commands for its six thrusters. The policy therefore drives the robot through the same interface used to collect the teleoperated demonstrations it was trained on. Image modified from [ 22 ] .

#	Observation Space	Obs. Dim.	succ.	part.	%	% w/ part.	%	%	%	% w/ part.	(ms)
			Grab the rod				Moor on Post	Remove Ring	Overall		Latency
#1	none (images only) $\dagger$	0	8	12	26.7	66.7	60.0	53.3	46.7	60.0	101.7 $\pm$ 1.5
#2	attitude, (control dataset)	3	4	8	13.3	40.0	60.0	93.3	55.6	64.4	101.0 $\pm$ 0.6
#3	attitude	3	10	19	33.3	96.7	86.7	96.7	72.2	93.3	95.2 $\pm$ 5.0
#4	attitude + $\Delta$ tokens (1.2 s)	3	8	14	26.7	73.3	63.3	46.7	45.6	61.1	102.4 $\pm$ 1.2
#5	attitude + $\Delta$ tokens (2.0 s)	3	6	8	20.0	46.7	50.0	93.3	54.4	63.3	100.0 $\pm$ 0.5

TABLE I: Pool Deployment: Success rates and forward-pass latency of multi-task models. A success for the Grab the rod task implies grabbing the right color rod, while a partial is when the model grabs a rod of the wrong color. Including partial successes and across 90 total gradable on-hardware rollouts (30 rollouts per task), 10 different DynamicVLA models outperformed the baseline. Latency is the time to produce one 20-step action chunk (mean over all models:

107.4\pm 11.4

ms). Four models run faster than the baseline despite a larger observation space.

Fig. 7: Pool Deployment: Multi-task models ranked. The median time to completion was calculated for each model over the successful rollouts, (a subste of the 90 total). All models outperformed the baseline in speediness of a successful task completion.

\dagger

Baseline: no proprioception of any kind.

Panel of Figure 8 — Fig. 8: Pool Deployment: Rod color confusion per multi-task model. Perceptive confusion exists strongly in every single multi-task model, and no model stood out to be perceptually more accurate than the other. The most common confusion is between red and green rods. None implies that the episode was failed, and hence, no rods were grabbed. $\dagger$ Baseline: no proprioception of any kind.

Task	Location	Policy	$n$	SR (%)	95% CI
Grab (red rod)	Shipwreck	ACT	150	33.3	[26.3, 41.2]
Grab (red rod)	Marina	ACT	14	42.9	[21.4, 67.4]
Grab (red rod)	Marina	SmolVLA	28	0.0	[0.0, 12.1]
Grab (green rod)	Pool	ACT	22	63.6	[43.0, 80.3]
Moor (ladder)	Marina	ACT	206	56.3	[49.5, 62.9]
Moor (ladder)	Marina	SmolVLA	48	12.5	[5.9, 24.7]

TABLE II: Marine Deployment: Success rate by task and location. ACT consistently outperforms SmolVLA, and Mooring is the best-performing task family for both architectures.

Multi-Task Robotic Manipulation

Multi-task robotic manipulation studies how robots can learn reusable, reliable behaviors across diverse objects, tasks, and environments.

See all 57 papers

WasserMan: Benchmark for Underwater Manipulation Policy Learning
Oct 3, 2026Danil Belov, Artem Erkhov, Sergei Parsegov +1Underwater Robotics Robot Manipulation Benchmarks
Central University, Moscow, Russia · Center for Engineering Systems and Sciences, Moscow, Russia · Sirius University of Science and Technology, Sirius Federal Territory, Russia +1
AquaCap: A Training-Free Underwater Embodied Agent with Code-as-Policy
Sep 19, 2026Xiaoshi Li, Yule Xu, Chunghiu Kong +5Underwater Robotics Autonomous Underwater Navigation
Sun Yat-sen University · Dalian University of Technology · Southern Marine Science and Engineering Guangdong Laboratory
UMI-Underwater: Learning Underwater Manipulation without Underwater Teleoperation
Mar 27, 2026Hao Li, Long Yin Chung, Jack Goler +5Affordance-Based Robotic Manipulation Robotic Grasping
All authors are with Stanford University.

Oct 3, 2026J/K move · Enter open · S save

WasserMan: Benchmark for Underwater Manipulation Policy Learning

Danil Belov, Artem Erkhov, Sergei Parsegov, Pavel Osinenko

Central University, Moscow, Russia · Center for Engineering Systems and Sciences, Moscow, Russia · Sirius University of Science and Technology, Sirius Federal Territory, Russia+1

Underwater Robotics Robot Manipulation Benchmarks Visuomotor Policy Learning Multi-Task Robotic Manipulation Robotic Policy Evaluation

Open paper AskPDF arXiv

Underwater manipulation couples visual decisions, contact forces and a thruster-controlled floating base. We introduce WasserMan, to our knowledge the first multi-task simulation benchmark for visuomotor learning of floating-base underwater contact manipulation. It provides ten expert-solvable tasks, two vehicle-arm platforms, a bimanual configuration and underwater dynamics. Nine tasks have learned-policy evaluations. We compare ACT, diffusion policies (DP) and behavioral cloning on six tasks, with three training runs and equal sampled-window budgets. Pretrained SmolVLA adds evaluations on three tasks. In the tested settings, removing integral action can prevent completion, while changing action interfaces can reduce learned-policy success despite successful expert replay. Currents produce task-dependent changes in completion and actuation effort. Versioned tasks, demonstrations and per-episode evidence support a protocol separating expert feasibility, learned completion and execution effort.

Abstract

Multi-Task Robotic Manipulation

Explore similar work

WasserMan: Benchmark for Underwater Manipulation Policy Learning

AquaCap: A Training-Free Underwater Embodied Agent with Code-as-Policy

UMI-Underwater: Learning Underwater Manipulation without Underwater Teleoperation