Sim-to-Real RL for ASVs using SysID

Cody Sheltraw; Tsimafei Lazouski; Maani Ghaffari; Alan Papalia

cs.ROOct 8, 2026

Sim-to-Real RL for ASVs using SysID

Authors: Cody Sheltraw, Tsimafei Lazouski, Maani Ghaffari, Alan Papalia

Organizations: University of Michigan, Ann Arbor, MI 48109, USA.

Abstract

Autonomous Surface Vehicles (ASVs) operating in dynamic marine environments require robust control policies for tasks such as path following and station keeping, making reinforcement learning (RL) a promising alternative to classical controllers. However, existing ASV simulators rarely support parallel environments for RL training. Such existing simulators require accurate hydrodynamic modeling from computational fluid dynamics solvers or towing tank tests for setting hydrodynamic parameters to address the sim-to-real gap. To address these challenges, we present an ASV simulator and accompanying pipeline that enables training policies starting from unknown vehicle dynamics. Our framework uses only a CAD model and brief set of open-water field trajectories for approximating and refining both hydrodynamic and thruster parameters. Real-world deployments on a BlueBoat ASV demonstrate successful zero-shot sim-to-real transfer in path following and station-keeping tasks without prior hydrodynamic and propeller information.

Figures & tables

Fig. 1: Overview of our sim-to-real pipeline. Initial hydrodynamic parameters are derived from a hull CAD model via a boundary element method (BEM) solver alongside a set of open-water trajectories from the physical autonomous surface vehicle (ASV). System identification (SysID) is performed in the simulator using the covariance matrix adaptation evolution strategy (CMA-ES) to obtain refined hydrodynamic ( $X_{u},X_{uu},Y_{v},Y_{vv},N_{r},N_{rr},I_{z}$ ) and thruster ( $c_{u},c_{\text{rev}}$ ) parameters, grounding the simulator to the real ASV. Reinforcement learning (RL) policies are trained in the grounded simulator, enabling zero-shot transfer for path following and station-keeping tasks.

Parameter	Port	Starboard	Units
Reverse Thrust Limit ( $f_{\text{min},i}$ )	-10.0	-10.0	N
Forward Thrust Limit ( $f_{\text{max},i}$ )	20.0	20.0	N
Motor Force Deadband ( $f_{\text{dead}}$ )	5.0		N
Motor Spool-up Lag ( $\tau_{\text{lag}}$ )	0.15		s
Thrust advance loss ( $c_{u}$ )	0.6016		—
Reverse thrust coefficient ( $c_{\text{rev}}$ )	0.4356		—

TABLE I: Nominal Action Space Parameter Values

Reward Component	$w_{k}$	Hyperparameters
Cross-Track Error ( $r^{\text{cross}}_{t}$ )	$2.0$	$\sigma_{\text{cross}}=0.3$
PF Heading ( $r^{\text{PF,head}}_{t}$ )	$1.0$	$\sigma_{\text{PF,head}}=0.3$
Path Progress ( $r^{\text{prog}}_{t}$ )	$1.5$	$\sigma_{\text{prog}}=0.25,\ u_{\text{target}}=0.5$
PF Term Penalty ( $r^{\text{PF,term}}_{t}$ )	$-10.0$	$d^{\text{PF}}_{\text{term}}=2.0$
Target Distance ( $r^{\text{dist}}_{t}$ )	$2.0$	$\sigma_{\text{dist}}=0.3$
SK Heading ( $r^{\text{SK,head}}_{t}$ )	$1.0$	$\sigma_{\text{SK,head}}=0.3,\ \epsilon_{\text{SK,head}}=2^{\circ}$

TABLE II: Reward Function Weights and Hyperparameters

Parameter	Distribution
Hydrodynamic multiplier	$\mathcal{U}(0.9,1.1)$
Advance loss ( $c_{u}$ ) multiplier	$\mathcal{U}(0.9,1.1)$
Reverse efficiency ( $c_{u}$ ) multiplier	$\mathcal{U}(0.9,1.1)$
Motor spool-up ( $\tau_{\text{lag}}$ )	$\mathcal{U}(0.10,0.20)$
Thrust asymmetry ( $c_{\text{stbd}}$ )	$\mathcal{U}(0.95,1.05)$
PF wind speed (m/s)	Curriculum: $\mathcal{U}(0,0)\rightarrow\mathcal{U}(0,5)$

TABLE III: Domain Randomization Parameters

Parameter	Path Following (PF)	Station Keeping (SK)
Episode Duration	20 s	30 s
Rollout Steps	128	256
Discount Factor ( $\gamma$ )	0.99	0.9975

TABLE IV: Task-Specific Training Hyperparameters

Fig. 2: Real-world path following trajectory evaluations for each configuration: Real-World (top-left), Proposed (top-right), Zigzag (bottom-left), and CMA-ES DR (bottom-right). Proposed achieves the lowest average cross-track RMSE and single-trajectory grounding suffers from divergent dynamics.

Parameter	Real-World	Proposed	Zigzag	Units
$I_{z}$	4.86	6.31	4.96	kg $\cdot$ m 2
$X_{u}$	-0.46	-0.59	-0.59	kg/s
$X_{uu}$	-6.83	-6.98	-8.87	kg/m
$Y_{v}$	-54.89	-71.26	-71.28	kg/s
$Y_{vv}$	-114.30	-148.37	-148.42	kg/m
$N_{r}$	-6.38	-4.48	-8.29	kg $\cdot$ m 2 /s

TABLE V: SysID Parameters Across Experimental Configurations

Fig. 3: Real-world station-keeping evaluations lasting 45 s for each configuration: Real-World (top-left), Proposed (top-right), Zigzag (bottom-left), and CMA-ES DR (bottom-right). Grounded policies all similarly outperform Real-World, with Zigzag notably outperforming CMA-ES DR.

Evaluation Task	Real-World	Proposed	Zigzag	CMA-ES DR
PF Cross-Track RMSE	0.423 m	0.417 m	2.909 m	0.549 m
SK Position RMSE	1.074 m	0.388 m	0.569 m	0.643 m
SK Heading MAE	17.30 ∘	10.37 ∘	12.70 ∘	10.68 ∘

TABLE VI: Real-World Evaluation RMSE and MAE

Explore similar work

Sim-to-Real Transfer and Robustness Evaluation of Reinforcement Learning Control with Integrated Perception on an ASV for Floating Waste Capture
May 4, 2026Luis F. W. Batista, Stéphanie Aravecchia, Cédric PradalierRL Control Sim-to-Real Transfer
GeorgiaTech Europe - IRL2958 GT-CNRS, Metz, France
Sim2Swim: Zero-Shot Velocity Control for Agile AUV Maneuvering in 3 Minutes
Dec 9, 2025Lauritz Rismark Fosso, Herman Biørn Amundsen, Marios Xanthidis +1RL Control Sim-to-Real Transfer
SINTEF Ocean, Trondheim, Norway
Cross-Platform Control for Autonomous Surface Vehicles via Adaptive Reinforcement Learning
Jul 2, 2026Ruiheng Jiang, Thomas Bi, Raffaello D'Andrea +1Robotic Control Robot Trajectory Tracking
Institute for Dynamic Systems and Control, ETH Zurich, Switzerland.

May 4, 2026, cs.ROJ/K move · Enter open · S save

Sim-to-Real Transfer and Robustness Evaluation of Reinforcement Learning Control with Integrated Perception on an ASV for Floating Waste Capture

Luis F. W. Batista, Stéphanie Aravecchia, Cédric Pradalier

GeorgiaTech Europe - IRL2958 GT-CNRS, Metz, France

RL Control Sim-to-Real Transfer

Open paper AskPDF arXiv

Autonomous surface vessels for floating-waste removal operate under varying hydrodynamics, external disturbances, and challenging water-surface perception. We present a field-validated system that combines camera-based polarimetric perception with a lightweight DRL-based controller for floating-waste detection and capture. Camera detections are converted into water-surface target points and tracked by a controller trained entirely in simulation and deployed directly on a retrofitted ASV platform. Our main contribution is a sim-to-real testing methodology that combines a two-stage simulation protocol with a perception abstraction module designed to mimic real camera behavior, enabling reproducible field trials and explicit evaluation of the sim-to-real gap. We apply this framework in matched simulation and field experiments across 14 disturbance regimes to expose failure modes and evaluate robustness. The results show centimeter-level terminal accuracy and indicate robust control performance under the evaluated perturbation regimes. The main source of degradation is insufficient actuation-model fidelity. We also demonstrate the system in a search-and-capture application using real camera detections in real-world conditions over areas of up to

450~m^2

. The study distills practical lessons for reliable transfer, including improved actuation-model fidelity, targeted domain randomization, and careful management of latency and timestamps across modules, while highlighting remaining challenges.

Abstract

Explore similar work

Sim-to-Real Transfer and Robustness Evaluation of Reinforcement Learning Control with Integrated Perception on an ASV for Floating Waste Capture

Sim2Swim: Zero-Shot Velocity Control for Agile AUV Maneuvering in 3 Minutes

Cross-Platform Control for Autonomous Surface Vehicles via Adaptive Reinforcement Learning