Sim2Sea: Sim-to-Real Policy Transfer for Maritime Vessel Navigation in Congested Waters

Xinyu Cui; Xuanfa Jin; Xue Yan; Yongcheng Zeng; Luoyang Sun; Siying Wei; Ruizhi Zhang; Jian Zhao

cs.ROMar 4, 2026

Sim2Sea: Sim-to-Real Policy Transfer for Maritime Vessel Navigation in Congested Waters

Authors: Xinyu Cui, Xuanfa Jin, Xue Yan, Yongcheng Zeng, Luoyang Sun, Siying Wei, Ruizhi Zhang, Jian Zhao,

Organizations: Institute of Automation, CAS School of Artificial Intelligence, UCAS Beijing, China · Institute of Automation, CAS Beijing, China · Zhongguancun Academy Beijing, China · University College London London, United Kingdom

Abstract

Autonomous navigation in congested maritime environments is a critical capability for a wide range of real-world applications. However, it remains an unresolved challenge due to complex vessel interactions and significant environmental uncertainties. Existing methods often fail in practical deployment due to a substantial sim-to-real gap, which stems from imprecise simulation, inadequate situational awareness, and unsafe exploration strategies. To address these, we propose Sim2Sea, a comprehensive framework designed to bridge simulation and real-world execution. Sim2Sea advances in three key aspects. First, we develop a GPU-accelerated parallel simulator for scalable and accurate maritime scenario simulation. Second, we design a dual-stream spatiotemporal policy that handles complex dynamics and multi-modal perception, augmented with a velocity-obstacle-guided action masking mechanism to ensure safe and efficient exploration. Finally, a targeted domain randomization scheme helps bridge the sim-to-real gap. Simulation results demonstrate that our method achieves faster convergence and safer trajectories than established baselines. In addition, our policy trained purely in simulation successfully transfers zero-shot to a 17-ton unmanned vessel operating in real-world congested waters. These results validate the effectiveness of Sim2Sea in achieving reliable sim-to-real transfer for practical autonomous maritime navigation.

Figures & tables

Figure 1. Sim2Sea overview. Left: Parallel maritime simulator with multiple vessel models and interaction simulations. Middle: Spatiotemporal policy with BEV fusion and VO guided active action masking. Right: Zero-shot onboard deployment enabled by domain randomization.

Parallelization	Windows PC		Linux Server
Parallelization	Ryzen 5900X CPU	RTX 5070Ti GPU	EPYC 7742 CPU	A100 GPU
Full	$0.938\pm 0.117$	$0.036\pm 0.002$	$0.504\pm 0.011$	$0.028\pm 0.001$
Per-Environment	$4.632\pm 0.043$	$0.147\pm 0.016$	$9.667\pm 0.037$	$0.264\pm 0.011$
Per-Agent	$8.968\pm 0.028$	$1.399\pm 0.012$	$20.368\pm 0.194$	$2.371\pm 0.013$

Table 1. Average execution time (in seconds) for simulation with 50 control steps across different parallelization strategies and hardware backends. Lower values indicate better performance. Results are averaged over 10 runs.

Figure 2. Mini Coastline and Mini Port scenarios. Pink polygons denote land or restricted terrain, green rectangles indicate the departure regions, yellow stars mark the goals, dark circles are static obstacles, and light circles are moving obstacles. The figure contains two subplots. Left: a coastline with a large landmass and one offshore polygon; a green rectangle at the bottom marks the departure region; a yellow star offshore marks the goal; several dark (static) and light (moving) circular obstacles are scattered along the route. Right: a port enclosed by breakwaters forming narrow fairways and anchorages; a green rectangle near the entrance marks the departure region; a yellow star inside the port marks the goal; multiple static (dark) and moving (light) circular obstacles are placed along the channels. Legend indicates colors and symbols as described above.

Panel of Figure 3 — Figure 3. Learning performance with and without BEV fusion, active action masking and temporal sequence.

Method	MiniCoast			MiniPort
Method	SR(%)	Length	UA	SR(%)	Length	UA
Sim2Sea	$93$	$500\pm 23$	$1.18\pm 0.42$	$90$	$343\pm 25$	$0.70\pm 0.29$
VO-RL	$77$	$558\pm 108$	$2.24\pm 1.43$	$47$	$356\pm 31$	$1.28\pm 0.37$
COLREG-RL	$83$	$587\pm 44$	$1.96\pm 0.32$	$77$	$467\pm 132$	$0.92\pm 0.34$
VO	$70$	$488\pm 22$	$2.53\pm 0.42$	$67$	$314\pm 9$	$1.24\pm 0.34$

Table 2. Performance of different methods on MiniCoast and MiniPort Scenarios. SR is success rate in percent. Length is mean episode length in steps. UA is the average number of unsafe actions per decision step.

Panel of Figure 4 — Figure 4. Learning performance with and without BEV fusion, active action masking and temporal sequence.

Panel of Figure 5 — Figure 5. Onboard interface and camera views. Left: graphical user interface displaying vessel state, mission progress, and chart overlays. Right: forward or side camera view providing situational awareness for safety monitoring.

Panel of Figure 6 — Figure 6. Real-ship sim-to-real trials in two scenarios. Post-convergence trajectories compare Sim2Sea with ablations without domain randomization and without temporal inputs. Obstacles are shown in the figures. The departure is green, the destination is marked with a red cross, and the goal is marked with a star.

Sim-to-Real Transfer

Simulation-to-reality transfer adapts models and policies trained in simulation for reliable performance in physical or otherwise real-world settings.

See all 300 papers

Sim-to-Real RL for ASVs using SysID
Oct 8, 2026Cody Sheltraw, Tsimafei Lazouski, Maani Ghaffari +1Sim-to-Real Transfer Robotics Simulation
University of Michigan, Ann Arbor, MI 48109, USA.
LOTUSim: Multi-Domain Simulator for Marine Robotics
Jul 3, 2026Cédric Buche, Juliette Grosset, Hélène Lechêne +4Multi-Robot Systems Robotics Simulation
CROSSING, IMT Atlantique · CROSSING IRL 2010, CNRS · IMT Atlantique +2
IoT-Enabled Autonomous Maritime Navigation in Smart Ports: A Curriculum-Guided Shared Policy Learning Framework
Aug 12, 2026Yuqing Lin, Rangya Zhang, Kum Fai YuenPartially Observable RL Robot Navigation
School of Civil and Environmental Engineering, Nanyang Technological University, Singapore

Oct 8, 2026J/K move · Enter open · S save

Sim-to-Real RL for ASVs using SysID

Cody Sheltraw, Tsimafei Lazouski, Maani Ghaffari, Alan Papalia

University of Michigan, Ann Arbor, MI 48109, USA.

Sim-to-Real Transfer Robotics Simulation RL Control Robot Navigation

Open paper AskPDF arXiv

Autonomous Surface Vehicles (ASVs) operating in dynamic marine environments require robust control policies for tasks such as path following and station keeping, making reinforcement learning (RL) a promising alternative to classical controllers. However, existing ASV simulators rarely support parallel environments for RL training. Such existing simulators require accurate hydrodynamic modeling from computational fluid dynamics solvers or towing tank tests for setting hydrodynamic parameters to address the sim-to-real gap. To address these challenges, we present an ASV simulator and accompanying pipeline that enables training policies starting from unknown vehicle dynamics. Our framework uses only a CAD model and brief set of open-water field trajectories for approximating and refining both hydrodynamic and thruster parameters. Real-world deployments on a BlueBoat ASV demonstrate successful zero-shot sim-to-real transfer in path following and station-keeping tasks without prior hydrodynamic and propeller information.

Abstract

Sim-to-Real Transfer

Explore similar work

Sim-to-Real RL for ASVs using SysID

LOTUSim: Multi-Domain Simulator for Marine Robotics

IoT-Enabled Autonomous Maritime Navigation in Smart Ports: A Curriculum-Guided Shared Policy Learning Framework