Active Reachability Learning for Humanoid Recoverability Monitoring

Zeyuan Feng; Baljeet Singh; Sarthak Ranjeet Kaingade; Somil Bansal; Bryan Trinh

cs.ROOct 23, 2026

Active Reachability Learning for Humanoid Recoverability Monitoring

Authors: Zeyuan Feng, Baljeet Singh, Sarthak Ranjeet Kaingade, Somil Bansal, Bryan Trinh

Organizations: Stanford University · Amazon

Abstract

Deploying humanoid task policies requires reliable runtime monitors that anticipate unrecoverable falls in time to trigger a fallback. We formulate this task as learning the stochastic, finite-horizon recoverability envelope of a given fallback policy. Synthesizing this monitor is challenging: existing methods assume training data naturally covers the recoverability envelope, yet acquiring such data remains a major bottleneck for contact-rich systems. Consequently, they recover only a fraction of the true envelope, leaving them prone to optimistic hallucinations for out-of-distribution (OOD) inputs. We resolve this via a unified framework that iterates between value learning and active exploration. Data exploration is reframed as a reach-avoid problem, where an uncertainty-guided explorer systematically uncovers unseen states while avoiding confidently unrecoverable regions. This loop is further supported by a combined Temporal Difference and Monte Carlo objective for accurate in-distribution value prediction and reliable uncertainty estimation. By explicitly treating the remaining high-uncertainty regions as unrecoverable, the monitor mitigates optimistic OOD hallucinations. Evaluated on stochastic benchmarks and a 29-DoF Unitree G1 humanoid, our approach drastically improves monitor coverage and accuracy and successfully transfers zero-shot to real-world hardware.

Figures & tables

Figure 2: Value heatmaps and level-set contours in the $x$ - $y$ plane (Left: DI; Right: Dubins). Black dashed boxes denote the initial distribution $\rho_{0}$ projected onto the $x$ - $y$ plane (unshown dimensions are sampled uniformly). Solid circles mark the target set, defined via $g(x)=\tanh(\text{SDF}(x))$ , where SDF denotes the signed distance function to the target set.

Figure 3: Ablation across toy systems (plot rows: DI, Dubins3D, Dubins3D-Exp.). Curves (left) show per-round mean over seeds; the table (right) reports the final-round means for the same metrics. RND uses a single value net, ensemble uses $N_{\text{ens}}$ nets. Our default is RND + reach-avoid .

Figure 4: 3D Dubins: Data coverage evolution over rounds on an originally OOD slice ( $\theta_{\text{yaw}}=-\frac{\pi}{4}$ ). Dashed black: GT recoverable set; Green: learned set; Blue: all datapoints with $\theta_{\text{yaw}}=-\frac{\pi}{4}\pm 0.1$ .

Method	Precision	Recall	$F_{1}$	MSE	Cum. seeds	Wall (min)
RND + reach-avoid (TD+MC)	0.942	0.955	0.949	0.006	53524	71
RND + reach-avoid (TD only)	0.885	0.940	0.912	0.015	72710	57
Ensemble + unbiased (TD+MC)	0.955	0.909	0.932	0.010	86464	48
Ensemble + unbiased (TD only)	0.904	0.615	0.731	0.031	180467	35

Table I: Final-round mean results: TD-only vs TD+MC on DI.

$N_{\mathrm{MC}}$	Coverage	Precision	$F_{1}$	MSE	Cum. seeds	Wall (min)
1	0.490 $\pm$ 0.013	0.833 $\pm$ 0.009	0.838 $\pm$ 0.005	0.032 $\pm$ 0.001	486626	80
10	0.830 $\pm$ 0.006	0.913 $\pm$ 0.003	0.920 $\pm$ 0.001	0.013 $\pm$ 0.003	405432	80
30	0.889 $\pm$ 0.010	0.929 $\pm$ 0.003	0.934 $\pm$ 0.004	0.009 $\pm$ 0.001	164003	78
100	0.935 $\pm$ 0.008	0.918 $\pm$ 0.008	0.936 $\pm$ 0.003	0.008 $\pm$ 0.000	60884	79
300	0.929 $\pm$ 0.004	0.930 $\pm$ 0.011	0.939 $\pm$ 0.001	0.007 $\pm$ 0.001	69518	79
1000	0.941 $\pm$ 0.011	0.942 $\pm$ 0.010	0.949 $\pm$ 0.007	0.006 $\pm$ 0.001	53524	71

Table II: Hyperparameter sensitivity (mean

\pm

std over seeds). Top: DI

N_{\mathrm{MC}}

sweep (RND + reach-avoid). Bottom: 3D Dubins reach-avoid horizon sweep (

N_{\text{MC}}=1000

Figure 5: DI value heatmaps across different $N_{\text{MC}}$ .

Head	Set	Prec.	Recall	$F_{1}$	Coverage
Value $V_{\theta}$	Val	$0.957{\pm}0.005$	$0.956{\pm}0.003$	$0.957{\pm}0.001$	$0.944{\pm}0.003$
Value $V_{\theta}$	OOD	$0.944{\pm}0.010$	$0.893{\pm}0.024$	$0.918{\pm}0.009$	$0.815{\pm}0.034$
$P_{\text{fail}}$	Val	$0.920{\pm}0.006$	$0.921{\pm}0.014$	$0.920{\pm}0.004$	$0.920{\pm}0.014$
$P_{\text{fail}}$	OOD	$0.915{\pm}0.018$	$0.711{\pm}0.049$	$0.799{\pm}0.026$	$0.698{\pm}0.047$

Table III: Humanoid results: mean

\pm

std over 3 seeds.

Panel of Figure 6 — Figure 6: (Left) Per-bin performance on the OOD push set (seed 0). Bins labeled by $\|v\|+0.25\|\omega\|$ . (Right) Closed-loop failure/trigger rates across various maximum push impulse durations. The monitor with our TD+MC objective triggers least often while achieving the smallest failure rates.

$T_{\max}$ (s)	Healthy Policy (%)				Degraded Policy (%)
$T_{\max}$ (s)	Unmon. Fall	Mon. Fall	Rel. Red.	Trigger	Unmon. Fall	Mon. Fall	Rel. Red.	Trigger
0.00	0.00	0.00	–	0.0	35.01	10.55	69.9	93.3
0.05	0.00	0.00	–	0.0	34.23	10.89	68.2	93.1
0.10	0.63	0.44	30.8	2.8	37.65	11.67	69.0	93.8
0.15	2.98	2.59	13.1	9.2	41.21	13.28	67.8	94.4
0.20	5.86	4.59	21.7	13.0	43.60	16.02	63.3	94.1

Table IV: Closed-loop failure (fall) and trigger rates across maximum push durations for the healthy and degraded policies. Unmon.: Unmonitored; Mon.: Monitored; Rel. Red.: relative reduction in failure rate.

Reachability Analysis

Reachability analysis determines which states a system can enter from given initial conditions, actions, and disturbances.

See all 78 papers

RecoverFormer: End-to-End Contact-Aware Recovery for Humanoid Robots
Apr 24, 2026Zihui LiuHumanoid Robotics Humanoid Robot Control
Stanford University
Stubborn: A Streamlined and Unified Reinforcement Learning Framework for Robust Motion Tracking and Fall Recovery for Humanoids
Jun 11, 2026Xiao Ren, Yuhui Yang, Zongbiao Weng +2RL for Robotics Reinforcement Learning
Southern University of Science and Technology, Shenzhen 518055, China
Embedding Classical Balance Control Principles in Reinforcement Learning for Humanoid Recovery
Mar 9, 2026Nehar Poddar, Stephen McCrory, Luigi Penco +3Humanoid Robot Fall Recovery Humanoid Robot Control
Florida Institute for Human and Machine Cognition, Pensacola, FL, USA. · University of West Florida, Pensacola, FL, USA.

Apr 24, 2026J/K move · Enter open · S save

RecoverFormer: End-to-End Contact-Aware Recovery for Humanoid Robots

Zihui Liu

Stanford University

Humanoid Robotics Humanoid Robot Control Robot Failure Recovery Humanoid Robot Locomotion Robot Policy Generalization Robust RL+1

Open paper AskPDF arXiv

Humanoid robots operating in unstructured environments must recover from unexpected disturbances-a capability that remains challenging for end-to-end control policies. We present RECOVERFORMER, a fully end-to-end humanoid recovery policy that learns when and how to switch among recovery behaviors-including compensatory stepping, hand-environment contact, and center-of-mass reshaping-while maintaining robust performance under model mismatch. The architecture combines a causal transformer over a 50-step observation history with two novel heads: a latent recovery mode that enables smooth transitions among distinct recovery strategies, and a contact affordance head that predicts which environmental surfaces (walls, railings, table edges) are beneficial for stabilization. We evaluate RECOVERFORMER on the Unitree G1 humanoid in MuJoCo. Trained only on open floor, RECOVERFORMER transfers zero shot to walled environments, achieving 100% recovery success across 100-300 N pushes and across wall distances from 0.25-1.4m. Under zero-shot dynamics mismatch, RECOVERFORMER reaches 75.5% at plus +25% mass, 89% under 30 ms latency, 91.5% at low friction, and 99% under compound friction, latency and mass perturbation. The learned latent modes specialize across force regimes without mode-level supervision, validated by t-SNE analysis of 300 episodes. Taken together, these results show that a single end-to-end policy can deliver multi-modal, contact aware humanoid recovery that generalizes across perturbation magnitude, contact geometry, and dynamics shift.

Abstract

Reachability Analysis

Explore similar work

RecoverFormer: End-to-End Contact-Aware Recovery for Humanoid Robots

Stubborn: A Streamlined and Unified Reinforcement Learning Framework for Robust Motion Tracking and Fall Recovery for Humanoids

Embedding Classical Balance Control Principles in Reinforcement Learning for Humanoid Recovery