ManeuverNet: A Soft Actor-Critic Framework for Precise Maneuvering of Double-Ackermann-Steering Robots with Optimized Reward Functions

Kohio Deflesselle; Mélodie Daniel; Aly Magassouba; Miguel Aranda; Olivier Ly

cs.ROFeb 16, 2026

ManeuverNet: A Soft Actor-Critic Framework for Precise Maneuvering of Double-Ackermann-Steering Robots with Optimized Reward Functions

Authors: Kohio Deflesselle, Mélodie Daniel, Aly Magassouba, Miguel Aranda, Olivier Ly

Organizations: Univ. Bordeaux, CNRS, Bordeaux INP, LaBRI, UMR 5800, F-33400 Talence, France. · School of Computer Science, University of Nottingham, UK. · Instituto de Investigación en Ingeniería de Aragón (I3A), Universidad de Zaragoza, 50018 Zaragoza, Spain.

Abstract

Autonomous control of double-Ackermann-steering robots is essential in agricultural applications, where robots must execute precise and complex maneuvers within a limited space. Classical methods, such as the Timed Elastic Band (TEB) planner, can address this problem, but they rely on parameter tuning, making them highly sensitive to changes in robot configuration or environment and impractical to deploy without constant recalibration. At the same time, end-to-end deep reinforcement learning (DRL) methods often fail due to unsuitable reward functions for non-holonomic constraints, resulting in sub-optimal policies and poor generalization. To address these challenges, this paper presents ManeuverNet, a DRL framework tailored for double-Ackermann systems, combining Soft Actor-Critic with CrossQ. Furthermore, ManeuverNet introduces four specifically designed reward functions to support maneuver learning. Unlike prior work, ManeuverNet does not depend on expert data or handcrafted guidance. We extensively evaluate ManeuverNet against both state-of-the-art DRL baselines and the TEB planner. Experimental results demonstrate that our framework substantially improves maneuverability and success rates, achieving more than a 40% gain over DRL baselines. Moreover, ManeuverNet effectively mitigates the strong parameter sensitivity observed in the TEB planner. In real-world trials, ManeuverNet achieved up to a 90% increase in maneuvering trajectory efficiency, highlighting its robustness and practical applicability.

Figures & tables

Fig. 1 : Maneuver handling (left figure) with 4WS robots in DRL is challenging because it requires current reward loss (circled in red in the right figure), making classical approaches sub-optimal.

Fig. 2 : DASMR rotating around an instantaneous center of rotation (ICR).

Panel of Figure 3 — Fig. 3 : Comparison between the shape of our reward functions (top) vs. classic state-of-the-art reward functions (bottom). X is the longitudinal axis, Y is the lateral axis, and the robot’s center position is at the origin. Each point on the heatmaps represents the reward for a $\boldsymbol{X_{d}}$ in the same position. Thus, the robot reaches $\boldsymbol{X_{d}}$ when $\|\boldsymbol{X_{d}}\|<d_{\text{th}}$ .

Rewards	Formulation
$\mathcal{R}_{\text{HS}}$	$-\sqrt{(\Delta x)^{2}+\left(c\cdot(\Delta y\pm\max\left\{0,\|\Delta y\|-\|\Delta x\|\right\})\right)^{2}}$
$\mathcal{R}_{\text{ES}}$	$-\sqrt{(\Delta x)^{2}+(c\Delta y)^{2}}$
$\mathcal{R}_{\text{Ch}}$	$-\max\left\{\|\Delta x\|,\|\Delta y\|\right\}$
$\mathcal{R}_{\text{Cl}}$	$\begin{cases}\tan^{-1}\left(\frac{\Delta y}{\Delta x}\right)\times ce^{d_{\text{th}}-d}&\text{ if$ \Delta y>d_{\text{th}} $}\\ -d&\text{otherwise}\\ \end{cases}$

TABLE I : DASMR reward functions where

\Delta x

and

\Delta y

are the X and Y components of

\left(\boldsymbol{X_{d}}-\boldsymbol{X_{c}}\right)

and

c

a weighting parameter.

Parameter	Value
Nb. layers	2
Actor Hidden size	256
Critic Hidden size	1024
$\alpha_{A}=\alpha_{C}$	1e-3
Replay buffer size	1,000,000
Batch size $N$	256

TABLE II : SAC and CrossQ parameters used on SBX during training and testing.

Panel of Figure 6 — Fig. 4 : Environment setup in simulation and real-world settings. The red square denotes the goal space. The blue square represents the robot’s workspace. In real-world settings, both spaces coincide to impose stricter constraints on navigation and positioning.

Panel of Figure 7 — Fig. 5 : An example of a maneuver executed by ManeuverNet in both simulation and real-world settings is shown in the first two rows. Another example, showcasing multi-terrain performance, is presented in the final row of figures. The red dot represents $\boldsymbol{X}_{d}$ .

Approach	Approach	Test Unseen: seed = 10
Type	Approach	SR $\uparrow$	AE( $\boldsymbol{\sigma}$ ) $\downarrow$	SPL $\uparrow$
Analytical	TEB [ 12 ]	100	0.05 (0.05)	0.79
DRL	Standard DRL [ 32 ]	45	0.25 (0.17)	0.35
	FastRLap [ 22 ]	00	3.79 (2.11)	0.00
	ManeuverNet (ours)	85	0.15 (0.04)	0.70

TABLE III : Benchmarking results for an unseen seed and

d_{\text{th}}=15

cm, with the success rate (SR) in %, the average error (AE) (the standard deviation

\sigma

) in m, and the average SPL.

Rew.	$\boldsymbol{d_{\text{th}}}$	Seen			Unseen 1			Unseen 2
Rew.	(cm)	SR $\uparrow$	AE( $\boldsymbol{\sigma}$ ) $\downarrow$	SPL $\uparrow$	SR $\uparrow$	AE( $\boldsymbol{\sigma}$ ) $\downarrow$	SPL $\uparrow$	SR $\uparrow$	AE( $\boldsymbol{\sigma}$ ) $\downarrow$	SPL $\uparrow$
$\mathcal{R}_{\text{Car}}$	15	02	0.19(0.62)	0.00	02	0.17(0.75)	0.02	02	0.17(0.62)	0.02
$\mathcal{R}_{\text{Car}}$	10	02	0.19(0.63)	0.00	02	0.18(0.76)	0.02	02	0.17(0.63)	0.02
$\mathcal{R}_{\text{Exp}}$	15	04	0.53(0.20)	0.03	04	0.55(0.25)	0.03	06	0.55(0.26)	0.04
$\mathcal{R}_{\text{Exp}}$	10	02	0.53(0.24)	0.01	02	0.56(0.24)	0.01	00	0.57(0.24)	0.00
$\mathcal{R}_{\text{HER}}$	15	55	0.25(0.21)	0.47	50	0.28(0.25)	0.40	55	0.30(0.26)	0.45

TABLE IV : ManeuverNet simulation results for different reward functions on seen, unseen 1, and unseen 2 seeds, with the success rate (SR) in %, the average error (AE) (the standard deviation

\sigma

) in m, and the average SPL.

Approach	Target ID						#Param.
	1	2	3	4	5	6
	SPL $\uparrow$ [Time $\downarrow$ ]
TEB [ 12 ]	0.21 [9]	0.31 [4]	0.63 [7]	0.93 [4]	0.28 [7]	0.37 [5]	$>10$
ManeuverNet	0.40 [4]	0.32 [4]	1.00 [3]	0.99 [3]	0.41 [5]	0.37 [5]	0
$\boldsymbol{\Delta}^{+}$ ( $\%$ )	90 [56]	03 [00]	60 [57]	06 [25]	46 [29]	00 [00]	$-$

TABLE V : Real-world comparison of SPL and [navigation time] (in seconds) across six target positions. The final column reports the number of parameters requiring manual tuning for real-world deployment.

\boldsymbol{\Delta}^{+}

denotes the performance improvement of ManeuverNet relative to the TEB planner.

Robotic RL

Robotic reinforcement learning uses trial-and-error learning to help robots control, navigate, and manipulate objects.

See all 299 papers

DRL-Based Pose Control for Double-Ackermann Robots Under Actuation Uncertainties
May 29, 2026Oussama Zaim, Mélodie Daniel, Aly Magassouba +2Robotic Control Robotics Simulation
Univ. Bordeaux, CNRS, Bordeaux INP, LaBRI, UMR 5800, F-33400 Talence, France. · School of Computer Science, University of Nottingham, UK. · Instituto de Investigación en Ingeniería de Aragón (I3A), Universidad de Zaragoza, 50018 Zaragoza, Spain.
Trajectory Planning and Control near the Limits: an Open Experimental Benchmark on the RoboRacer Platform
May 19, 2026Mattia Piccinini, Patrick Zambiasi, Aniello Mungiello +3Autonomous Driving Benchmarks Robot Trajectory Tracking
Professorship of Autonomous Vehicle Systems, Technical University of Munich, 85748 Garching, Germany; Munich Institute of Robotics and Machine Intelligence (MIRMI). · Avilus GmbH, Germany. · Department of Information Technology and Electrical Engineering (DIETI), University of Naples Federico II, Naples 80125, Italy. +1
Meta-Adaptive Beam Search Planning for Transformer-Based Reinforcement Learning Control of UAVs with Overhead Manipulators under Flight Disturbances
Mar 27, 2026Hazim Alzorgan, Sayed Pedram Haeri Boroujeni, Abolfazl RaziAerial Manipulation Robot Trajectory Tracking
School of Computing, Clemson University, Clemson, SC 29632, USA

May 29, 2026J/K move · Enter open · S save

DRL-Based Pose Control for Double-Ackermann Robots Under Actuation Uncertainties

Oussama Zaim, Mélodie Daniel, Aly Magassouba, Miguel Aranda+1

Univ. Bordeaux, CNRS, Bordeaux INP, LaBRI, UMR 5800, F-33400 Talence, France. · School of Computer Science, University of Nottingham, UK. · Instituto de Investigación en Ingeniería de Aragón (I3A), Universidad de Zaragoza, 50018 Zaragoza, Spain.

Robotic Control Robotics Simulation Sim-to-Real Transfer Robust RL

Open paper AskPDF arXiv

Robust deployment of deep reinforcement learning (DRL) policies on real robots remains challenging due to discrepancies between simulation and real-world dynamics. We address this issue in the context of maneuvering with double-Ackermann-steering mobile robots, which introduce additional constraints due to their non-holonomic nature. Building upon the DRL framework ManeuverNet, we extend its objective from position control to full pose control, resulting in a more challenging task. We further investigate the impact of actuation-related uncertainties on policy transfer. The use of simplified actuation models during training of the extended policy can lead to poor generalization, shown by a success rate drop from 100% in PyBullet to 25% in Gazebo under stricter evaluation conditions. To address this limitation, we adopt a sim-to-sim-to-real approach, where actuation effects observed in Gazebo are incorporated into the PyBullet training environment. Using multi-environment DRL with SAC and CrossQ, we learn policies that remain robust despite modeling inaccuracies. This approach can significantly reduce the performance gap across simulators, achieving up to 92% success rate in Gazebo and maintaining 69% under stricter thresholds, with successful transfer to a real robot without additional tuning.

Abstract

Robotic RL

Explore similar work

DRL-Based Pose Control for Double-Ackermann Robots Under Actuation Uncertainties

Trajectory Planning and Control near the Limits: an Open Experimental Benchmark on the RoboRacer Platform

Meta-Adaptive Beam Search Planning for Transformer-Based Reinforcement Learning Control of UAVs with Overhead Manipulators under Flight Disturbances