Organizations: Institute of Automation, CAS School of Artificial Intelligence, UCAS Beijing, China · Institute of Automation, CAS Beijing, China · Zhongguancun Academy Beijing, China · University College London London, United Kingdom
Autonomous navigation in congested maritime environments is a critical capability for a wide range of real-world applications. However, it remains an unresolved challenge due to complex vessel interactions and significant environmental uncertainties. Existing methods often fail in practical deployment due to a substantial sim-to-real gap, which stems from imprecise simulation, inadequate situational awareness, and unsafe exploration strategies. To address these, we propose Sim2Sea, a comprehensive framework designed to bridge simulation and real-world execution. Sim2Sea advances in three key aspects. First, we develop a GPU-accelerated parallel simulator for scalable and accurate maritime scenario simulation. Second, we design a dual-stream spatiotemporal policy that handles complex dynamics and multi-modal perception, augmented with a velocity-obstacle-guided action masking mechanism to ensure safe and efficient exploration. Finally, a targeted domain randomization scheme helps bridge the sim-to-real gap. Simulation results demonstrate that our method achieves faster convergence and safer trajectories than established baselines. In addition, our policy trained purely in simulation successfully transfers zero-shot to a 17-ton unmanned vessel operating in real-world congested waters. These results validate the effectiveness of Sim2Sea in achieving reliable sim-to-real transfer for practical autonomous maritime navigation.
Figures & tables
Figure 1. Sim2Sea overview. Left: Parallel maritime simulator with multiple vessel models and interaction simulations. Middle: Spatiotemporal policy with BEV fusion and VO guided active action masking. Right: Zero-shot onboard deployment enabled by domain randomization.
Parallelization
Windows PC
Linux Server
Ryzen 5900X CPU
RTX 5070Ti GPU
EPYC 7742 CPU
A100 GPU
Full
0.938±0.117
0.036±0.002
0.504±0.011
0.028±0.001
Per-Environment
4.632±0.043
0.147±0.016
9.667±0.037
0.264±0.011
Per-Agent
8.968±0.028
1.399±0.012
20.368±0.194
2.371±0.013
Table 1. Average execution time (in seconds) for simulation with 50 control steps across different parallelization strategies and hardware backends. Lower values indicate better performance. Results are averaged over 10 runs.
Figure 2. Mini Coastline and Mini Port scenarios. Pink polygons denote land or restricted terrain, green rectangles indicate the departure regions, yellow stars mark the goals, dark circles are static obstacles, and light circles are moving obstacles. The figure contains two subplots. Left: a coastline with a large landmass and one offshore polygon; a green rectangle at the bottom marks the departure region; a yellow star offshore marks the goal; several dark (static) and light (moving) circular obstacles are scattered along the route. Right: a port enclosed by breakwaters forming narrow fairways and anchorages; a green rectangle near the entrance marks the departure region; a yellow star inside the port marks the goal; multiple static (dark) and moving (light) circular obstacles are placed along the channels. Legend indicates colors and symbols as described above.
Figure 3. Learning performance with and without BEV fusion, active action masking and temporal sequence.
Method
MiniCoast
MiniPort
SR(%)
Length
UA
SR(%)
Length
UA
Sim2Sea
93
500±23
1.18±0.42
90
343±25
0.70±0.29
VO-RL
77
558±108
2.24±1.43
47
356±31
1.28±0.37
COLREG-RL
83
587±44
1.96±0.32
77
467±132
0.92±0.34
VO
70
488±22
2.53±0.42
67
314±9
1.24±0.34
Table 2. Performance of different methods on MiniCoast and MiniPort Scenarios. SR is success rate in percent. Length is mean episode length in steps. UA is the average number of unsafe actions per decision step.
Figure 4. Learning performance with and without BEV fusion, active action masking and temporal sequence.
Figure 5. Onboard interface and camera views. Left: graphical user interface displaying vessel state, mission progress, and chart overlays. Right: forward or side camera view providing situational awareness for safety monitoring.
Figure 6. Real-ship sim-to-real trials in two scenarios. Post-convergence trajectories compare Sim2Sea with ablations without domain randomization and without temporal inputs. Obstacles are shown in the figures. The departure is green, the destination is marked with a red cross, and the goal is marked with a star.
Autonomous Surface Vehicles (ASVs) operating in dynamic marine environments require robust control policies for tasks such as path following and station keeping, making reinforcement learning (RL) a promising alternative to classical controllers. However, existing ASV simulators rarely support parallel environments for RL training. Such existing simulators require accurate hydrodynamic modeling from computational fluid dynamics solvers or towing tank tests for setting hydrodynamic parameters to address the sim-to-real gap. To address these challenges, we present an ASV simulator and accompanying pipeline that enables training policies starting from unknown vehicle dynamics. Our framework uses only a CAD model and brief set of open-water field trajectories for approximating and refining both hydrodynamic and thruster parameters. Real-world deployments on a BlueBoat ASV demonstrate successful zero-shot sim-to-real transfer in path following and station-keeping tasks without prior hydrodynamic and propeller information.
Simulation is essential for maritime robotics, supporting operator training, mission rehearsal, and human-vehicle interaction in environments where real-world testing is costly or hazardous. Existing simulators focus primarily on autonomy systems and often lack human-in-the-loop interaction and realistic environmental physics. This paper introduces LOTUSim, an open-source, real-time maritime simulator supporting multi-user interaction across aerial, surface, and underwater robotic systems for coordinated naval-style operations. The first contribution of this work is enabling real-time interactive performance for users while ensuring scalability to large fleets operating within a shared interactive simulation environment. Validation demonstrates robust human-in-the-loop performance, maintaining strict real-time execution and high visual fidelity while scaling to large heterogeneous maritime drone swarms. The second contribution is a computationally efficient, Ekman-inspired layered, underwater current model that captures wind-driven, depth-dependent flow dynamics with sufficient physical fidelity for large-scale simulations. Validation against ocean reanalysis data demonstrates substantially improved accuracy compared to commonly used stochastic Gauss-Markov current models. These results confirm LOTUSim's suitability as a simulation platform for operatorin-the-loop maritime robotics research.
As smart port infrastructures increasingly rely on autonomous maritime devices enabled by the Internet of Things (IoT), ensuring reliable onboard navigation intelligence has become a critical challenge for safe and scalable operations in congested waterways. This paper investigates onboard autonomous navigation for such IoT devices under partial observability and dense traffic conditions. A curriculum-guided reinforcement learning framework with a shared recurrent policy is developed to enhance temporal reasoning, deployment scalability, and robustness of edge-level decision-making. Centralized training is adopted as an offline design-time strategy, while all navigation actions are executed fully onboard, consistent with IoT edge intelligence paradigms. Extensive simulations in multiple realistic port environments demonstrate that the proposed approach improves navigation reliability, collision avoidance, and training stability compared with standard baseline methods, and generalizes effectively to previously unseen high-density scenarios. The results indicate that curriculum-guided shared learning provides a practical solution for scalable deployment of IoT-enabled autonomous maritime devices in smart port operations.
Yuqing Lin, Rangya Zhang, Kum Fai Yuen
School of Civil and Environmental Engineering, Nanyang Technological University, Singapore