cs.ROOct 21, 2026

State-Dependent Objective Coordination for End-to-End Balance and Navigation

Authors: Xuhao Huang, Xitao Xu, Yuxin Zheng, Yunfan Ren, Zheng Gong

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Traditional robotic navigation systems separate mapping, planning, and low-level control through cascaded modules, introducing computational overhead and accumulated latency. End-to-end reinforcement learning offers a compact unified policy that maps onboard observations directly to control actions. However, on inherently unstable platforms such as two-wheeled inverted-pendulum robots, navigation progress and obstacle avoidance can conflict with attitude stabilization in a strongly state-dependent manner. We propose a lightweight state-dependent objective coordination method for integrated navigation and control. A pitch-conditioned scalarization coefficient β(s)∈[0,1]β(s)\in[0,1] adjusts the relative contributions of balance stabilization and navigation objectives during training, without extra control hierarchies or auxiliary attitude-stabilization controllers, such as proportional-integral-derivative (PID) or linear-quadratic regulator (LQR) controllers. To support zero-shot deployment, simulation training models execution delay, action hold, asynchronous and noisy observations, and dynamics uncertainty. Gradient analysis reveals persistent directional interference and state-dependent variation in the objectives' relative scales. Comparisons with alternative reward-weighting, constraint, training, and control-structure strategies show that PPO+β(s)β(s), where PPO denotes proximal policy optimization, achieves 98% and 95% success rates with small attitude excursions on the fixed obstacle map and the randomly generated high-obstacle-density map, respectively. Zero-shot hardware experiments further demonstrate bounded attitude behavior and consistent obstacle avoidance using only onboard observations. Training and evaluation code will be made publicly available upon acceptance.

Figures & tables

Fig. 1 : Zero-shot simulation-to-real-world transfer of the learned policy using ultra-wideband (UWB), light detection and ranging (LiDAR), and an inertial measurement unit (IMU). Left: simulation. Right: real-world deployment.
Fig. 2 : Framework for state-dependent objective coordination and deployment-oriented training. During simulation training, β(st)\beta(s_{t}) performs reward-level state-conditioned scalarization of balance and navigation, while safety regularization is separated. The rollout model considers execution delay, action hold, asynchronous and noisy observations, and dynamics uncertainty. The trained actor is extracted from the policy network and deployed using onboard observations only.
Fig. 3 : State-dependent objective coordination on the balancebot. A pitch-based attitude-state indicator determines the scalarization coefficient β(st)\beta(s_{t}) , which adjusts the relative contributions of the balance-stabilization and navigation-progress objectives. The coefficient increases with pitch magnitude according to Eq. ( 6 ).
Pitch stateMean CosConflict ratioNorm ratio
Low pitch0.19132.4%36.4 ×\times
Medium pitch0.23527.9%1.2 ×\times
High pitch0.20830.8%1.3 ×\times
TABLE I : Gradient interaction under different pitch-related state regimes.
Fig. 4 : Training success rates of representative coordination strategies over normalized training progress. Curves and shaded regions denote the mean and standard deviation across repeated training runs, respectively.
ConfigurationFixed obstacle mapHardest map
 SR ↑\uparrowAvgTime ↓\downarrow (s)NFDfail↓\mathrm{NFD}_{\mathit{fail}}\downarrowθ‾max↓\overline{\theta}_{\mathit{max}}\downarrow (rad)SR ↑\uparrowAvgTime ↓\downarrow (s)NFDfail↓\mathrm{NFD}_{\mathit{fail}}\downarrowθ‾max↓\overline{\theta}_{\mathit{max}}\downarrow (rad)
(a) Alternative β\beta coordination strategies under the deployment-oriented constraint setting
Training-adaptive β\betaFail (Timeout)0.73 ±\pm 0.160.30 ±\pm 0.24Fail (Timeout)0.82 ±\pm 0.060.28 ±\pm 0.04
Learnable β\betaFail (Timeout)0.99 ±\pm 0.020.32 ±\pm 0.22Fail (Timeout)1.00 ±\pm 0.010.26 ±\pm 0.13
State-conditioned learnable β\beta57%145.71 ±\pm 53.340.34 ±\pm 0.280.26 ±\pm 0.0474%112.06 ±\pm 31.850.35 ±\pm 0.290.24 ±\pm 0.13
(b) Reduced-constraint setting as an alternative
TABLE II : Comparison of alternative coordination strategies, constraint settings, and training/control structures on the fixed obstacle map and the hardest map.
Fig. 5 : Two-dimensional trajectories of representative coordination strategies across six test maps. Solid curves show representative trajectories, and shaded regions indicate trajectory distributions across repeated rollouts. Gray blocks denote obstacles; circles, stars, and crosses mark starts, goals, and failure endpoints, respectively.
SRAvgTime (s)mean(∣θ∣)\mathrm{mean}(|\theta|) (rad)mean(∣θ˙∣)\mathrm{mean}(|\dot{\theta}|) (rad/s)
95%±5%95\%\pm 5\%17.09±9.0417.09\pm 9.040.05±0.030.05\pm 0.030.19±0.090.19\pm 0.09
TABLE III : Sim-to-real performance of PPO+ β(s)\beta(s) .
Fig. 6 : Matched-layout simulation-to-real-world comparison: (a)–(d) simulation; (e)–(h) hardware deployment for the corresponding four test cases.
Fig. 7 : Sim-to-real attitude signals of PPO+ β(s)\beta(s) . Curves compare pitch and pitch angular velocity between simulation and hardware; shaded regions indicate variability across repeated hardware trials.
More in this topic

Robotic Control

Robotic control studies how machines use sensing and computation to produce reliable movement and interaction in changing environments.

See all 587 papers

Explore similar work

CardsList
Oct 5, 2026

End-to-End Safe Social Navigation via Multi-Task Reinforcement Learning and Probabilistic Perception

Autonomous social navigation requires balancing efficiency, physical safety, and social compliance. Reinforcement Learning (RL) methods provide a viable and effective solution but often rely on unrealistic assumptions, such as the knowledge of humans' position and velocity. In this paper, we introduce JESSI (JAX-based E2E Safe Social Interpretable navigation), a lightweight end-to-end RL framework that maps raw LiDAR scans directly to kinematically feasible control commands. JESSI enhances safety via Dirichlet-parameterized continuous action spaces and deterministic bounding, while an integrated attention-based perception module extracts probabilistic human states for interpretable, socially aware decision-making. Through extensive simulations and real-world deployment on a differential-drive robot, we demonstrate that jointly optimizing the RL policy with a supervised perception signal in a multi-task paradigm enhances social behavior. Ultimately, JESSI is able to balance high navigation success rates and superior social behaviors compared to state-of-the-art baselines.

Tommaso Van Der Meer, Andrea Garulli, Antonio Giannitrapani +3

Dipartimento di Ingegneria dell’Informazione e Scienze Matematiche, Universit`a di Siena, Siena, Italy · Uninettuno University, Rome, Italy · Visual Intelligence for Transportation (VITA) Lab, ´Ecole Polytechnique F´ed´erale de Lausanne (EPFL), Lausanne, Switzerland

Robotic RLSafe Robot NavigationSocial Robot NavigationRobot SafetyMulti-Task RLRobotic Perception
AskPDF
Jun 9, 2026

GUIDE: Goal-Initialized Directional Understanding for End-to-End Legged Navigation

End-to-end reinforcement learning (RL) has shown strong potential for legged robot navigation, yet existing approaches commonly rely on continuously updated robot-to-goal states from external state estimation modules, leaving part of the navigation problem outside the learned policy. In this work, we seek to push the limits of end-to-end sim-to-real RL navigation by investigating whether a legged robot can internally maintain the spatial context required for long-horizon navigation. To this end, we study goal-initialized navigation, where the goal is provided only once at the beginning of an episode, with no subsequent external relative-goal updates. We present GUIDE, an end-to-end RL framework that jointly learns navigation and internal directional awareness from onboard observations. GUIDE leverages multi-frequency proprioceptive history to capture egomotion and auxiliary spatial-anchor prediction to maintain task-relevant spatial states, while temporal depth observations provide local environmental geometry. The navigation policy is trained entirely in simulation and transferred zero-shot to the real world. Across cluttered environments and structured mazes, GUIDE reliably avoids obstacles, escapes dead ends, and reaches distant goals using only onboard sensing. These results demonstrate that robust sim-to-real legged navigation can be achieved without continuously providing external robot-to-goal estimation, opening a promising direction for future research on more self-contained end-to-end navigation.

Liang Wang, Jin Jin, KanZhong Yao +6

Institute of Cyber-Systems and Control, Zhejiang University · Institute of Artificial Intelligence (TeleAI), China Telecom · Oxford Robotics Institute, University of Oxford +2

Robot NavigationRL for Legged Locomotion
AskPDF
Jul 20, 2026

Predictive Training with Latent Imagination for Visual Quadruped Navigation

Reinforcement-learning navigation policies for legged robots select actions reactively from current observations and short-term memory, with limited capacity to anticipate how moving obstacles will evolve in the near future. In dynamic environments, this reactivity causes the robot to respond too late because collision risk depends on short-horizon scene structure rather than on current obstacle positions alone. Lightweight predictive supervision applied to the policy's recurrent state during training can encode anticipatory obstacle dynamics without modifying the inference-time controller. We augment a reactive LSTM-SRU navigation backbone with an auxiliary JEPA-style predictor and SIGReg regularization: during training, the predictor supervises the deterministic hidden state to anticipate its own next state; at inference, it is fully discarded, incurring zero additional computational cost. On simulated and real-world navigation benchmarks with dynamic obstacles, our method substantially improves navigation success while reducing collision rates through the predictive training signal alone, without additional inference-time parameters. Real-robot deployment on a Unitree Go2 demonstrates zero-shot sim-to-real transfer: the controller navigates cluttered indoor and dynamic outdoor environments without fine-tuning, with evasive behavior consistent with the collision reduction observed in simulation.

Yancheng Zhu, Wanli Ma, Chen Han +3

Anker Spatial Perception Lab

Robot NavigationRL for Legged LocomotionPredictive Representation LearningDynamic Obstacle Avoidance
AskPDF