State-Dependent Objective Coordination for End-to-End Balance and Navigation

Xuhao Huang; Xitao Xu; Yuxin Zheng; Yunfan Ren; Zheng Gong

cs.ROOct 21, 2026

State-Dependent Objective Coordination for End-to-End Balance and Navigation

Authors: Xuhao Huang, Xitao Xu, Yuxin Zheng, Yunfan Ren, Zheng Gong

Abstract

Traditional robotic navigation systems separate mapping, planning, and low-level control through cascaded modules, introducing computational overhead and accumulated latency. End-to-end reinforcement learning offers a compact unified policy that maps onboard observations directly to control actions. However, on inherently unstable platforms such as two-wheeled inverted-pendulum robots, navigation progress and obstacle avoidance can conflict with attitude stabilization in a strongly state-dependent manner. We propose a lightweight state-dependent objective coordination method for integrated navigation and control. A pitch-conditioned scalarization coefficient $β(s)\in[0,1]$ adjusts the relative contributions of balance stabilization and navigation objectives during training, without extra control hierarchies or auxiliary attitude-stabilization controllers, such as proportional-integral-derivative (PID) or linear-quadratic regulator (LQR) controllers. To support zero-shot deployment, simulation training models execution delay, action hold, asynchronous and noisy observations, and dynamics uncertainty. Gradient analysis reveals persistent directional interference and state-dependent variation in the objectives' relative scales. Comparisons with alternative reward-weighting, constraint, training, and control-structure strategies show that PPO+ $β(s)$ , where PPO denotes proximal policy optimization, achieves 98% and 95% success rates with small attitude excursions on the fixed obstacle map and the randomly generated high-obstacle-density map, respectively. Zero-shot hardware experiments further demonstrate bounded attitude behavior and consistent obstacle avoidance using only onboard observations. Training and evaluation code will be made publicly available upon acceptance.

Figures & tables

Fig. 1 : Zero-shot simulation-to-real-world transfer of the learned policy using ultra-wideband (UWB), light detection and ranging (LiDAR), and an inertial measurement unit (IMU). Left: simulation. Right: real-world deployment.

Fig. 2 : Framework for state-dependent objective coordination and deployment-oriented training. During simulation training, $\beta(s_{t})$ performs reward-level state-conditioned scalarization of balance and navigation, while safety regularization is separated. The rollout model considers execution delay, action hold, asynchronous and noisy observations, and dynamics uncertainty. The trained actor is extracted from the policy network and deployed using onboard observations only.

Fig. 3 : State-dependent objective coordination on the balancebot. A pitch-based attitude-state indicator determines the scalarization coefficient $\beta(s_{t})$ , which adjusts the relative contributions of the balance-stabilization and navigation-progress objectives. The coefficient increases with pitch magnitude according to Eq. ( 6 ).

Pitch state	Mean Cos	Conflict ratio	Norm ratio
Low pitch	0.191	32.4%	36.4 $\times$
Medium pitch	0.235	27.9%	1.2 $\times$
High pitch	0.208	30.8%	1.3 $\times$

TABLE I : Gradient interaction under different pitch-related state regimes.

Fig. 4 : Training success rates of representative coordination strategies over normalized training progress. Curves and shaded regions denote the mean and standard deviation across repeated training runs, respectively.

Configuration	Fixed obstacle map				Hardest map
	SR $\uparrow$	AvgTime $\downarrow$ (s)	$\mathrm{NFD}_{\mathit{fail}}\downarrow$	$\overline{\theta}_{\mathit{max}}\downarrow$ (rad)	SR $\uparrow$	AvgTime $\downarrow$ (s)	$\mathrm{NFD}_{\mathit{fail}}\downarrow$	$\overline{\theta}_{\mathit{max}}\downarrow$ (rad)
(a) Alternative $\beta$ coordination strategies under the deployment-oriented constraint setting
Training-adaptive $\beta$	Fail (Timeout)		0.73 $\pm$ 0.16	0.30 $\pm$ 0.24	Fail (Timeout)		0.82 $\pm$ 0.06	0.28 $\pm$ 0.04
Learnable $\beta$	Fail (Timeout)		0.99 $\pm$ 0.02	0.32 $\pm$ 0.22	Fail (Timeout)		1.00 $\pm$ 0.01	0.26 $\pm$ 0.13
State-conditioned learnable $\beta$	57%	145.71 $\pm$ 53.34	0.34 $\pm$ 0.28	0.26 $\pm$ 0.04	74%	112.06 $\pm$ 31.85	0.35 $\pm$ 0.29	0.24 $\pm$ 0.13
(b) Reduced-constraint setting as an alternative

TABLE II : Comparison of alternative coordination strategies, constraint settings, and training/control structures on the fixed obstacle map and the hardest map.

Fig. 5 : Two-dimensional trajectories of representative coordination strategies across six test maps. Solid curves show representative trajectories, and shaded regions indicate trajectory distributions across repeated rollouts. Gray blocks denote obstacles; circles, stars, and crosses mark starts, goals, and failure endpoints, respectively.

SR	AvgTime (s)	$\mathrm{mean}(\|\theta\|)$ (rad)	$\mathrm{mean}(\|\dot{\theta}\|)$ (rad/s)
$95\%\pm 5\%$	$17.09\pm 9.04$	$0.05\pm 0.03$	$0.19\pm 0.09$

TABLE III : Sim-to-real performance of PPO+

\beta(s)

Panel of Figure 9 — Fig. 6 : Matched-layout simulation-to-real-world comparison: (a)–(d) simulation; (e)–(h) hardware deployment for the corresponding four test cases.

Panel of Figure 10 — Fig. 7 : Sim-to-real attitude signals of PPO+ $\beta(s)$ . Curves compare pitch and pitch angular velocity between simulation and hardware; shaded regions indicate variability across repeated hardware trials.

Robotic Control

Robotic control studies how machines use sensing and computation to produce reliable movement and interaction in changing environments.

See all 587 papers

End-to-End Safe Social Navigation via Multi-Task Reinforcement Learning and Probabilistic Perception
Oct 5, 2026Tommaso Van Der Meer, Andrea Garulli, Antonio Giannitrapani +3Robotic RL Safe Robot Navigation
Dipartimento di Ingegneria dell’Informazione e Scienze Matematiche, Universit`a di Siena, Siena, Italy · Uninettuno University, Rome, Italy · Visual Intelligence for Transportation (VITA) Lab, ´Ecole Polytechnique F´ed´erale de Lausanne (EPFL), Lausanne, Switzerland
GUIDE: Goal-Initialized Directional Understanding for End-to-End Legged Navigation
Jun 9, 2026Liang Wang, Jin Jin, KanZhong Yao +6Robot Navigation RL for Legged Locomotion
Institute of Cyber-Systems and Control, Zhejiang University · Institute of Artificial Intelligence (TeleAI), China Telecom · Oxford Robotics Institute, University of Oxford +2
Predictive Training with Latent Imagination for Visual Quadruped Navigation
Jul 20, 2026Yancheng Zhu, Wanli Ma, Chen Han +3Robot Navigation RL for Legged Locomotion
Anker Spatial Perception Lab

Oct 5, 2026J/K move · Enter open · S save

End-to-End Safe Social Navigation via Multi-Task Reinforcement Learning and Probabilistic Perception

Tommaso Van Der Meer, Andrea Garulli, Antonio Giannitrapani, Renato Quartullo+2

Dipartimento di Ingegneria dell’Informazione e Scienze Matematiche, Universit`a di Siena, Siena, Italy · Uninettuno University, Rome, Italy · Visual Intelligence for Transportation (VITA) Lab, ´Ecole Polytechnique F´ed´erale de Lausanne (EPFL), Lausanne, Switzerland

Robotic RL Safe Robot Navigation Social Robot Navigation Robot Safety Multi-Task RL Robotic Perception

Open paper AskPDF arXiv

Autonomous social navigation requires balancing efficiency, physical safety, and social compliance. Reinforcement Learning (RL) methods provide a viable and effective solution but often rely on unrealistic assumptions, such as the knowledge of humans' position and velocity. In this paper, we introduce JESSI (JAX-based E2E Safe Social Interpretable navigation), a lightweight end-to-end RL framework that maps raw LiDAR scans directly to kinematically feasible control commands. JESSI enhances safety via Dirichlet-parameterized continuous action spaces and deterministic bounding, while an integrated attention-based perception module extracts probabilistic human states for interpretable, socially aware decision-making. Through extensive simulations and real-world deployment on a differential-drive robot, we demonstrate that jointly optimizing the RL policy with a supervised perception signal in a multi-task paradigm enhances social behavior. Ultimately, JESSI is able to balance high navigation success rates and superior social behaviors compared to state-of-the-art baselines.

Abstract

Robotic Control

Explore similar work

End-to-End Safe Social Navigation via Multi-Task Reinforcement Learning and Probabilistic Perception

GUIDE: Goal-Initialized Directional Understanding for End-to-End Legged Navigation

Predictive Training with Latent Imagination for Visual Quadruped Navigation