Contact-UAN: Unsupervised Actuator Networks as a Residual World Model for Humanoid Control

Nolan Fey; Khai Nguyen; Yash Prabhu; Gabriel B. Margolis; Pulkit Agrawal

cs.ROOct 13, 2026

Contact-UAN: Unsupervised Actuator Networks as a Residual World Model for Humanoid Control

Authors: Nolan Fey, Khai Nguyen, Yash Prabhu, Gabriel B. Margolis, Pulkit Agrawal

Organizations: Improbable AI Lab, MIT

Abstract

Humanoid policies trained in simulation can transfer qualitatively to hardware while failing to transfer the force interactions learned during training, compromising torque efficiency, load regulation, and robustness to impacts. This failure can arise from actuator sim-to-real gaps because forces depend on actuator torques. We introduce Contact-UAN, a residual world model that preserves a simulator's physics priors while correcting actuator dynamics to align simulated rollouts with recorded hardware rollouts. For humanoid motion with contact, this requires contact-consistent replay: if hardware playback in simulation begins with misaligned contact wrenches, the residual model learns to correct reset artifacts rather than actuator discrepancies. Using only onboard proprioception, Contact-UAN infers the simulator-side stance-foot pose needed to initialize replay with consistent contact forces. We show that a UAN trained on 6.5 minutes of real-world walking data generalizes to tasks that benefit from accurate force and impulse regulation and produces policies that transfer seamlessly to the real world, yielding 20% lower running torque, 43% lower force-tracking error, extreme compliance, accurate impedance control, and dynamic, robust jumps.

Figures & tables

Figure 4: Contact-UAN. a) Train a UAN to add joint-position target corrections to reduce simulation transition error relative to recorded hardware data, with base velocities and contact poses labeled via proprioceptive state estimators. b) Train a task policy in the UAN-augmented simulator with no robot-specific domain randomization. c) Deploy only the learned policy on hardware.

Panel of Figure 5 — Figure 5: Contact estimation and replay accuracy. (A) Contact-height RMSE for estimators trained on both stance and swing (Direct) or stance only (Stance), evaluated in the hard target simulator. (B) 24-step hardware joint-position MSE on WTW training data. (C) Sim-to-sim 24-step joint-position RMSE. The same UAN substantially reduces replay error under both estimated and ground-truth initialization, supporting actuator-model correction beyond compensation for state-estimation errors.

Panel of Figure 6 — Figure 6: Held-out running torque replay. (A, B) Representative left- and right-knee commanded-torque traces under identical relabeled initial states. (C) Aggregate commanded torque-usage relative error decreases by $69.2\%$ with UAN.

Panel of Figure 7 — Figure 7: Improved running and force control sim-to-real transfer. UAN running policy retains tracking ability (A) while improving efficiency (B). The baseline marker at $3.0~\mathrm{m\,s^{-1}}$ denotes protection-triggered safe damping. UAN force policy reduced average force tracking error by $42.8\pm 12.1\%$ (C).

Panel of Figure 8 — Figure 8: Sim-to-sim reset ablation and policy transfer. (A) Reported 20-s survival after retraining UANs and policies with direct, estimated (Est.), component-oracle, or full-oracle (All) resets. Base, Cnt., and Pose replace base velocity, contact state, and stance pose, respectively, with ground truth. (B, C) Velocity tracking and squared torque across target gaps. WTW-trained UAN policies approach expert performance, with little additional benefit from task-specific running data.

Figure 9: Dynamic and robust jumping policies. (A) The UAN policy first tracks a jumping reference motion, (B) then is fine-tuned to maximize jump height; (C) it is also robust to external disturbances.

Figure 10: Impedance position control. Combining a precise force control policy with a proprioceptive state estimator, we develop an impedance controller for closed-loop end-effector position tracking. The controller tracks a circular trajectory reasonably well without any external sensing.

Appendix figures & tables20 assets

Supplementary material from the paper’s appendix.

Appendix

Setting	Base velocity	Contact status	Foot height	Foot proj. gravity
Output dimension	3	2	1 per foot	3 per foot
Hidden layers	$(256,256,256)$	$(128,128)$	$(256,256,256)$	$(256,256,256)$
Loss	MSE	BCE-logits	MSE	MSE

Appendix

Table 1: State-estimator architectures. All estimators use ELU activations, Adam, learning rate

3\times 10^{-4}

, batch size 4096, history length 3, and 200 epochs.

Observation	Noise range
Base angular velocity	$\pm 0.2~\mathrm{rad/s}$
Projected gravity	$\pm 0.05$
Joint position	$\pm 0.01~\mathrm{rad}$
Joint velocity	$\pm 1.5~\mathrm{rad/s}$
Previous target	none

Appendix

Table 2: Observation noise used when collecting simulated data (uniform additive).

Actuator group	Joints
7520_14	Hip pitch, hip yaw, waist yaw
7520_22	Hip roll, knee
5020_2x	Waist roll, waist pitch, ankle pitch, ankle roll
5020	Shoulder pitch, shoulder roll, shoulder yaw, elbow, wrist roll
4010	Wrist pitch, wrist yaw

Appendix

Table 3: Unitree G1 actuator groups used by the shared UAN architecture.

Element	condim	friction	solref	solimp
Foot collision geoms	$3$	$[0.6,\,0.005,\,0.0001]$	$[0.02,\,1]$	$[0.9,\,0.95,\,0.001,\,0.5,\,2]$
Terrain/ground geom	$3$	$[1,\,0.005,\,0.0001]$	$[0.02,\,1]$	$[0.9,\,0.95,\,0.001,\,0.5,\,2]$

Appendix

Table 4: MuJoCo contact parameters used in the contact-pose sensitivity diagnostic.

Optimization
Parameter	UAN	WTW	Running	Force-control	Jumping
Parallel environments	4096	4096	4096	4096	16384
Steps per environment	24	24	24	24	24
Learning epochs	5	5	5	5	5
Mini-batches	4	4	4	4	4
Learning rate	$10^{-3}$ , adaptive (target KL 0.01) for all

Appendix

Table 5: PPO hyperparameters for the UAN, the WTW data-collection policy, and the downstream task policies.

Term	Math	Weight
joint_position_mse	$\frac{1}{n_{q}}\sum_{j}(q^{\mathrm{sim}}_{t,j}-q^{\mathrm{hw}}_{t,j})^{2}$	$-1000$
joint_velocity_mse	$\frac{1}{n_{q}}\sum_{j}(\dot{q}^{\mathrm{sim}}_{t,j}-\dot{q}^{\mathrm{hw}}_{t,j})^{2}$	$-0.1$
residual_rate_l2	$\frac{1}{n_{q}}\sum_{j}(\delta q_{t,j}-\delta q_{t-1,j})^{2}$	$-0.01$

Appendix

Table 6: UAN replay reward weights.

Term	Math	Weight
Task rewards
track_linear_velocity	$\exp(-(\\|v^{\mathrm{cmd}}_{xy}-v^{b}_{xy}\\|^{2}+(v^{b}_{z})^{2})/0.25)$	$2.0$
track_angular_velocity	$\exp(-(\omega^{\mathrm{cmd}}_{z}-\omega^{b}_{z})^{2}/0.25)$	$2.0$
Regularization
upright	$\exp(-\\|g^{b}_{\mathrm{torso},xy}\\|^{2}/0.2)$	$1.0$
pose	$\exp\!\left(-\frac{1}{n_{q}}\sum_{j}\frac{(q_{j}-q^{0}_{j})^{2}}{\sigma_{j}(\\|v^{\mathrm{cmd}}\\|)^{2}}\right)$	$1.0$

Appendix

Table 7: Running policy reward weights.

Term	Math	Weight
Task rewards
track_force_ee	$\exp(-\\|F^{\mathrm{cmd}}-F^{\mathrm{applied}}\\|^{2}/50)$	$3.0$
track_lin_vel_xy_exp	$\exp(-\\|v^{b}\\|^{2}/0.25)$	$2.0$
track_ang_vel_z_exp	$\exp(-\\|\omega^{b}\\|^{2}/0.5)$	$2.0$
Regularization
upright	$\exp(-\\|g^{b}_{\mathrm{torso},xy}\\|^{2}/0.2)$	$1.0$

Appendix

Table 8: Force-control policy reward weights.

Term	Dim.	Noise $n$
Reference joint positions and velocities (current frame)	58	–
Reference height	1	–
Reference projected gravity	3	–
Reference root velocity	6	–
Projected gravity (IMU)	3	0.05
Base angular velocity (IMU)	3	0.2

Appendix

Table 9: Actor observation space for the jumping task. Noise is additive, sampled uniformly from

[-n,n]

per component. The anchor body is the torso link.

Motion tracking
Term	Math	Pretraining	Fine-tuning
motion_global_root_pos	$\exp\!\big(-\\|\hat{\mathbf{p}}_{a}-\mathbf{p}_{a}\\|^{2}/0.3^{2}\big)$	$0.5$	$0.5$
motion_global_root_ori	$\exp\!\big(-(\hat{\mathbf{q}}_{a}\ominus\mathbf{q}_{a})^{2}/0.4^{2}\big)$	$0.5$	$0.5$
motion_body_pos	$\exp\!\big(-\tfrac{1}{\|\mathcal{B}\|}\sum_{b}\\|\hat{\mathbf{p}}_{b}-\mathbf{p}_{b}\\|^{2}/0.3^{2}\big)$	$1.0$	$1.0$
motion_body_ori	$\exp\!\big(-\tfrac{1}{\|\mathcal{B}\|}\sum_{b}(\hat{\mathbf{q}}_{b}\ominus\mathbf{q}_{b})^{2}/0.4^{2}\big)$	$1.0$	$1.0$
motion_body_lin_vel	$\exp\!\big(-\tfrac{1}{\|\mathcal{B}\|}\sum_{b}\\|\hat{\mathbf{v}}_{b}-\mathbf{v}_{b}\\|^{2}/1.0^{2}\big)$	$1.0$	$1.0$

Appendix

Table 10: Reward terms for jumping pretraining and fine-tuning. Motion-tracking and regularization weights remain unchanged during fine-tuning, which adds a ballistic center-of-mass rise bonus. The indicator

\mathbf{1}_{\mathrm{to}}

marks a confirmed two-foot takeoff, rewarded at most once per eligible reference-motion segment. Here

[x]_{+}=\max(x,0)

, and

q_{j}^{\min}

and

q_{j}^{\max}

are the soft joint-position limits. All terms are multiplied by the policy timestep

\Delta t_{\rm policy}

when computing the reward.

Condition	Trigger	Pretraining	Fine-tuning
Episode timeout	$t_{\mathrm{episode}}\geq\theta$	$20$ s	$20$ s
Anchor height error	$\|\hat{z}_{a}-z_{a}\|>\theta$	$0.25$ m	–
Anchor gravity mismatch	$\|\hat{g}_{a,z}-g_{a,z}\|>\theta$	$0.8$	–
End-effector height error	$\max_{b\in\mathcal{E}}\|\hat{z}_{b}-z_{b}\|>\theta$	$0.25$ m	–
Minimum root height	$z_{\mathrm{root}}<\theta$	–	$0.30$ m
Maximum root tilt	$\alpha_{\mathrm{root}}>\theta$	–	$80^{\circ}$

Appendix

Table 11: Termination conditions for jumping pretraining and fine-tuning. Thresholds are fixed within each stage; a dash indicates an inactive condition. The set

\mathcal{E}

contains the tracked feet and wrists,

g_{a,z}

is the vertical component of projected gravity in the anchor frame, and

\alpha_{\mathrm{root}}

is the root tilt angle from upright.

Perturbation	easy	medium	hard
Damping offset	$+2.0$	$+4.0$	$+6.0$
Frictionloss offset	$+0.5$	$+1.0$	$+1.5$
Action delay	$1$ step	$2$ steps	$3$ steps
Encoder bias	$+0.003$ rad	$+0.006$ rad	$+0.009$ rad
Armature multiplier	$1.5\times$	$2.0\times$	$2.5\times$

Appendix

Table 12: Sim-to-sim target perturbations. Each target simulator applies deterministic actuator perturbations to the nominal G1 model. The medium and hard targets scale the same perturbation vector used for easy .

Panel of Figure 11 — Figure 11: Controlled world-model data-efficiency reference. Mean normalized episode length with standard deviation in each method’s training environment and the hard target environment. UAN uses 5 minutes of target data; our best-effort G1 adaptation of RWM-U uses approximately 33 hours. WM initializes the policy randomly, while WM FT fine-tunes the data-collection policy inside the same learned world model.

Architecture
Parameter	Value
Backbone	GRU ensemble
Ensemble size	5
RNN hidden size	256
RNN layers	2
Head MLP (state / ext. / contact / term.)	$[128]$

Appendix

Table 13: World-model architecture and training hyperparameters used for RWM-U on G1.

Rollout collection
Parameter	Baseline PPO	MOPO-PPO (RWM-U)
Parallel envs	4096	4096
Steps per environment	24	24
Rollout source	simulator	learned WM
Outer iterations	20,000	5,000
Optimization

Appendix

Table 14: PPO hyperparameters used for MOPO-PPO policy training inside the RWM-U world model. The baseline PPO column is shown for reference; only the bold rows differ.

Panel of Figure 12 — Figure 12: Fixed-residual initialization sensitivity: joint velocity. 24-step joint-velocity replay RMSE in rad/s in the sim-to-sim experiment. The same residual is either disabled or enabled under ground-truth and estimated initialization. Joint-position results are shown in Figure 5 C.

Panel of Figure 13 — Figure 13: Sensor-corruption retraining experiment. Reported 20-s survival after training with clean data, fixed IMU pitch bias (degrees), or velocity low-pass filtering (Hz).

Model	Replay $q$	Survival	XY error	Yaw error	Power
	(rad)	(%)	(m/s)	(rad/s)	(W)
Position residual	0.00349	97.34	0.391	0.185	69.29
Torque residual	0.00571	98.17	0.387	0.214	69.25
Supervised torque	0.00565	97.56	0.394	0.215	71.90

Appendix

Table 15: Matched actuator-model comparison. While the position residual UAN achieves the tightest fit to target data, all variations achieve similar downstream performance.

Figure 14: Hardware replay knee-position trajectories. Representative knee-position trajectories on held-out running data compare recorded hardware motion with nominal and UAN-augmented simulator replay of identical recorded commands from the same relabeled initial states.

Figure 15: Open-loop rollouts. We initialize the simulator with the estimated state and roll out recorded hardware actions. The estimated simulator reset state is shown in blue. (A) uses a foot-height estimator trained on both stance and swing data; (B) uses our estimator trained only on stance data to align initial contacts; (C) uses our estimator and UAN to reduce the actuation gap.

Humanoid Robot Control

Humanoid robot control develops methods for coordinating bipedal balance, locomotion, and whole-body interaction with people and objects.

See all 202 papers

RecoverFormer: End-to-End Contact-Aware Recovery for Humanoid Robots
Apr 24, 2026Zihui LiuHumanoid Robotics Humanoid Robot Control
Stanford University
SceneBot: Contact-Prompted General Humanoid Whole Body Tracking with Scene-Interaction
Jun 25, 2026Sirui Chen, Shibo Zhao, Zhen Wu +3Robotic Control Humanoid Robot Locomotion
Stanford, Amazon FAR United States · Amazon FAR United States · CMU, Amazon FAR United States
WorldContact: A Contact-Centric World Model for Scalable Robot Learning
Sep 17, 2026Caoliwen Wang, Mengdi Wang, Heng Zhang +7Robot Policy Adaptation World Models for Robotics
University of British Columbia · Westlake University · Style3D Research

Apr 24, 2026J/K move · Enter open · S save

RecoverFormer: End-to-End Contact-Aware Recovery for Humanoid Robots

Zihui Liu

Stanford University

Humanoid Robotics Humanoid Robot Control Robot Failure Recovery Humanoid Robot Locomotion Robot Policy Generalization Robust RL+1

Open paper AskPDF arXiv

Humanoid robots operating in unstructured environments must recover from unexpected disturbances-a capability that remains challenging for end-to-end control policies. We present RECOVERFORMER, a fully end-to-end humanoid recovery policy that learns when and how to switch among recovery behaviors-including compensatory stepping, hand-environment contact, and center-of-mass reshaping-while maintaining robust performance under model mismatch. The architecture combines a causal transformer over a 50-step observation history with two novel heads: a latent recovery mode that enables smooth transitions among distinct recovery strategies, and a contact affordance head that predicts which environmental surfaces (walls, railings, table edges) are beneficial for stabilization. We evaluate RECOVERFORMER on the Unitree G1 humanoid in MuJoCo. Trained only on open floor, RECOVERFORMER transfers zero shot to walled environments, achieving 100% recovery success across 100-300 N pushes and across wall distances from 0.25-1.4m. Under zero-shot dynamics mismatch, RECOVERFORMER reaches 75.5% at plus +25% mass, 89% under 30 ms latency, 91.5% at low friction, and 99% under compound friction, latency and mass perturbation. The learned latent modes specialize across force regimes without mode-level supervision, validated by t-SNE analysis of 300 episodes. Taken together, these results show that a single end-to-end policy can deliver multi-modal, contact aware humanoid recovery that generalizes across perturbation magnitude, contact geometry, and dynamics shift.

Abstract

Humanoid Robot Control

Explore similar work

RecoverFormer: End-to-End Contact-Aware Recovery for Humanoid Robots

SceneBot: Contact-Prompted General Humanoid Whole Body Tracking with Scene-Interaction

WorldContact: A Contact-Centric World Model for Scalable Robot Learning