cs.ROOct 28, 2026

Embodied at Sea: Imitation Learning for Multi-Task Underwater Manipulation

Authors: Greta Perez-Haiek, Luca Macesanu, Kevin Angers, Mohammed Imam, Timothy Gambin, Christopher Clark

Organizations: New York University · University of Malta

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Behavior Cloning has demonstrated promising generalization capabilities for robotic manipulation in terrestrial environments. However, marine visual distortions and dynamic currents create a severe domain shift that remains a key bottleneck in adapting generalist architectures to underwater field manipulation. To address this, 1616 episodes of underwater manipulation data were collected in calm and turbulent waters and were used to train 14 generalist Vision-Language-Action (VLA) models (DynamicVLA) with varying proprioception observation configurations. Across three tasks on 1260 gradable on-hardware rollouts in a pool, 10 models outperformed the baseline, with the highest-performing model achieving a 93.3% overall success rate, including partial successes. Results are also introduced from a case study evaluating the performance of a single-task Action Chunking Transformer (ACT) and a generalist VLA (SmolVLA) policy for two tasks in marine archaeology and conservation: grasping and mooring. Experiments are performed on an ROV on and around the WWII wreck Coralita X127 and at Manoel Island Marina, Malta. Across 748 collected episodes and 508 gradable on-hardware rollouts, the ACT policy substantially outperformed SmolVLA (47.9% vs. 7.9% pooled success rate), earning a 66.7% success rate on the most successful task at the marine archaeological site. To enable further contributions to embodied intelligence in marine field robotics, the BLUEMBODIED KIT used to collect data, train models, and deploy policies on the ROV has been open sourced.

Figures & tables

Fig. 2: Pool Deployment: Experimental tasks evaluated in a controlled environment. Data collection in the Pool (Control Environment) was fully illuminated with a split (see Fig. 3 ) between calm and turbulent waters created by a human volunteer (see Fig. 5 ). Rod locations and color order were randomized between episodes. Post location stayed relatively the same, but initial robot and ring configurations varied between episodes as well.
Fig. 3: Pool Deployment: Dataset split. Training data collected in the pool environment incorporates multi-modal observations and human-made turbulent waters. The dataset is broken down into two conditions: a control condition (calm waters) and a turbulent condition.
Fig. 4: Marine Deployment: Experimental tasks evaluated across different underwater environments. Data collection in the Pool (Control Environment) was fully illuminated with a split between calm (Control Condition) and turbulent waters. Data collection at the Marina and Coralita X127 shipwreck took place during both day and night, featuring varying disturbances (see Fig. Embodied at Sea: Imitation Learning for Multi-Task Underwater Manipulation ).
Fig. 5: Pool Deployment: calm vs. turbulent conditions. Mean attitude and few choice IMU observations plotted over all Grab the Red Rod episodes (see Fig. 3 ). Since all models (except the baseline) observe roll, pitch, and yaw, these measurements illustrate how turbulence manifests in the observation space.
Fig. 6: Modified DynamicVLA Architecture. The policy outputs a six-dimensional action at each step: four motion commands and two tool commands. The motion commands are sent as MAVLink MANUAL_CONTROL messages to the BlueROV2’s onboard ArduSub autopilot, which converts them into individual thrust commands for its six thrusters. The policy therefore drives the robot through the same interface used to collect the teleoperated demonstrations it was trained on. Image modified from [ 22 ] .
   Grab the rodMoor on PostRemove RingOverallLatency
#Observation SpaceObs. Dim.succ.part.%% w/ part.%%%% w/ part.(ms)
#1none (images only) †\dagger081226.766.760.053.346.760.0101.7 ±\pm 1.5
#2attitude, (control dataset)34813.340.060.093.355.664.4101.0 ±\pm 0.6
#3attitude3101933.396.786.796.772.293.395.2 ±\pm 5.0
#4attitude + Δ\Delta tokens (1.2 s)381426.773.363.346.745.661.1102.4 ±\pm 1.2
#5attitude + Δ\Delta tokens (2.0 s)36820.046.750.093.354.463.3100.0 ±\pm 0.5
TABLE I: Pool Deployment: Success rates and forward-pass latency of multi-task models. A success for the Grab the rod task implies grabbing the right color rod, while a partial is when the model grabs a rod of the wrong color. Including partial successes and across 90 total gradable on-hardware rollouts (30 rollouts per task), 10 different DynamicVLA models outperformed the baseline. Latency is the time to produce one 20-step action chunk (mean over all models: 107.4±11.4107.4\pm 11.4 ms). Four models run faster than the baseline despite a larger observation space.
Fig. 7: Pool Deployment: Multi-task models ranked. The median time to completion was calculated for each model over the successful rollouts, (a subste of the 90 total). All models outperformed the baseline in speediness of a successful task completion. †\dagger Baseline: no proprioception of any kind.
Fig. 8: Pool Deployment: Rod color confusion per multi-task model. Perceptive confusion exists strongly in every single multi-task model, and no model stood out to be perceptually more accurate than the other. The most common confusion is between red and green rods. None implies that the episode was failed, and hence, no rods were grabbed. †\dagger Baseline: no proprioception of any kind.
TaskLocationPolicynnSR (%)95% CI
Grab (red rod)ShipwreckACT15033.3[26.3, 41.2]
Grab (red rod)MarinaACT1442.9[21.4, 67.4]
Grab (red rod)MarinaSmolVLA280.0[0.0, 12.1]
Grab (green rod)PoolACT2263.6[43.0, 80.3]
Moor (ladder)MarinaACT20656.3[49.5, 62.9]
Moor (ladder)MarinaSmolVLA4812.5[5.9, 24.7]
TABLE II: Marine Deployment: Success rate by task and location. ACT consistently outperforms SmolVLA, and Mooring is the best-performing task family for both architectures.
More in this topic

Multi-Task Robotic Manipulation

Multi-task robotic manipulation studies how robots can learn reusable, reliable behaviors across diverse objects, tasks, and environments.

See all 57 papers

Explore similar work

CardsList
Oct 3, 2026

WasserMan: Benchmark for Underwater Manipulation Policy Learning

Underwater manipulation couples visual decisions, contact forces and a thruster-controlled floating base. We introduce WasserMan, to our knowledge the first multi-task simulation benchmark for visuomotor learning of floating-base underwater contact manipulation. It provides ten expert-solvable tasks, two vehicle-arm platforms, a bimanual configuration and underwater dynamics. Nine tasks have learned-policy evaluations. We compare ACT, diffusion policies (DP) and behavioral cloning on six tasks, with three training runs and equal sampled-window budgets. Pretrained SmolVLA adds evaluations on three tasks. In the tested settings, removing integral action can prevent completion, while changing action interfaces can reduce learned-policy success despite successful expert replay. Currents produce task-dependent changes in completion and actuation effort. Versioned tasks, demonstrations and per-episode evidence support a protocol separating expert feasibility, learned completion and execution effort.

Danil Belov, Artem Erkhov, Sergei Parsegov +1

Central University, Moscow, Russia · Center for Engineering Systems and Sciences, Moscow, Russia · Sirius University of Science and Technology, Sirius Federal Territory, Russia +1

Underwater RoboticsRobot Manipulation BenchmarksVisuomotor Policy LearningMulti-Task Robotic ManipulationRobotic Policy Evaluation
AskPDF
Sep 19, 2026

AquaCap: A Training-Free Underwater Embodied Agent with Code-as-Policy

Recent advances in vision-language-action models have stimulated growing interest in underwater embodied intelligence. However, their reliance on large-scale interaction data limits their applicability underwater, where data collection is costly and scarce. To address this challenge, we present AquaCap, a training-free Code-as-Policy framework for autonomous underwater navigation and manipulation. AquaCap employs a dual-layer agent that translates task instructions and environmental observations into condition-aware plans and executable control programs. Structured perception then provides the agent with semantic, geometric, and reliability-aware observations under degraded underwater conditions. A failure-aware memory diagnoses unsuccessful actions and supports closed-loop replanning and code revision. This design enables online adaptation without task-specific training or parameter updates. AquaCap achieves a 66.43% success rate in simulation. Real-world experiments further demonstrate autonomous grasping and object transport with an ROV, including the manipulation of targets displaced by hydrodynamic disturbances.

Xiaoshi Li, Yule Xu, Chunghiu Kong +5

Sun Yat-sen University · Dalian University of Technology · Southern Marine Science and Engineering Guangdong Laboratory

Underwater RoboticsAutonomous Underwater NavigationLanguage Model-Based Control
AskPDF
Mar 27, 2026

UMI-Underwater: Learning Underwater Manipulation without Underwater Teleoperation

Underwater robotic grasping is difficult due to degraded, highly variable imagery and the expense of collecting diverse underwater demonstrations. We introduce a system that (i) autonomously collects successful underwater grasp demonstrations via a self-supervised data collection pipeline and (ii) transfers grasp knowledge from on-land human demonstrations through a depth-based affordance representation that bridges the on-land-to-underwater domain gap and is robust to lighting and color shift. An affordance model trained on on-land handheld demonstrations is deployed underwater zero-shot via geometric alignment, and an affordance-conditioned diffusion policy is then trained on underwater demonstrations to generate control actions. In pool experiments, our approach improves grasping performance and robustness to background shifts, and enables generalization to objects seen only in on-land data, outperforming RGB-only baselines. Code, videos, and additional results are available at https://umi-under-water.github.io.

Hao Li, Long Yin Chung, Jack Goler +5

All authors are with Stanford University.

Affordance-Based Robotic ManipulationRobotic GraspingRobotic ManipulationGeneralization in Robotic ManipulationZero-Shot Robotic Manipulation
AskPDF