We test whether Koopman structure helps a differentiable world model used to train a simulated Unitree G1 whole-body tracking policy by backpropagation through imagined rollouts, replacing the MLP world model of an otherwise unchanged pipeline. It does not: return is at best equal (0.823 versus 0.830), one-step error on identical data is 1.3 to 1.6 times the MLP's, and training is about 15% slower. Lifted rollouts diverge or fail to train policies; of the schemes tested, re-lifting at every imagined step trains best. Re-lifting never uses the lifted linear dynamics and leaves a model affine in the action, with state-dependent offset and slope; an action-affine MLP also predicts worse, so the accuracy cost belongs to this class. At a fixed state, the dynamics' curvature in the action explains only part of it. Across states, the policy's actions span a curved region, so an affine model fits policy data only as the tangent at the policy's action, the operating point: on G1, the MLP's tangent there is as accurate as the MLP; at the mean action, 31 times worse. The offset must encode, from the state alone, where the policy acts, whereas an MLP reads the action. Supplying the policy's action removes most of the remaining gap on G1, Ant and Humanoid. The only tested online repair approaching the MLP feeds the action into the nonlinear lift, abandoning the structure. For this use, the MLP world model remains the better choice.
Figures & tables
World model
Return
Failed
Δ
Error
MLP
0.830 ± 0.006
0/0/0
—
0.101
Koopman
0.823 ± 0.006
0/2/0
−0.004 (21)
0.152
Affine MLP
0.821 ± 0.005
0/1/0
−0.004 (21)
0.165
Koopman, rank 0
0.783 ± 0.014
1/2/3
−0.024 (31)
0.311
Koopman, 2-layer encoder
0.806 ± 0.012
1/1/2
−0.006 (22)
0.231
Re-lift every step
0.809 ± 0.007
1/2/2
−0.007 (27)
0.203
TABLE I: Online G1 tracking at iteration 12,000. Return: mean ± s.d. over 3 seeds; failed: failed clips per seed; Δ : seed-mean per-clip return minus the MLP world model’s on the 31 clips that no run reaching iteration 12,000 failed (in parentheses: clips lower); error: online eight-step error with re-lifting at every step, averaged over the last 2,000 iterations. Middle: the bounded Koopman world model under three rollout schemes in imagination ( Section IV-B ); lifting once is shown at iteration 4,000. Bottom: lifts that also read μ , with or without the gradient through it ( Section IV-E ); the detached one is shown at its last checkpoint before it was stopped.
Fig. 1: Online G1 training, 3 seeds (mean and range): (a) return; (b) online 8-step error, moving average over 500 iterations.
Fig. 2: Offline one-step test error relative to the MLP world model (marker: mean of the 3 per-seed ratios; whiskers: range). G1: a fresh collection of the tracking data; others: Gymnasium MuJoCo datasets. Given μ : all world models, the MLP included, also read μ ; n/m: not measured.
Task
Data
World model
None
Inputs
μ^
μ
G1
1 ×
Affine MLP
1.62
1.34
1.50
1.40
G1
1 ×
Koopman
1.44
1.29
1.30
1.20
G1
4 ×
Affine MLP
1.37
1.14
—
1.20
G1
4 ×
Koopman
1.19
1.01
—
1.01
Ant
1 ×
Affine MLP
1.81
1.72
1.43
1.34
Ant
1 ×
Koopman
1.67
1.56
1.35
1.26
TABLE II: One-step error on every test transition, as the 3-seed mean error of each world model over that of the MLP given the same extra input. Extra input: none; the policy’s inputs; μ^ , the data’s actions regressed on them; or μ ( Section IV-D ). Data: 4 × uses the full training split of one collection per task and 1 × a quarter of it; both use the same test set. —: not measured.
Fig. 3: (a) G1 data recollected by the same behavior policy under nominal physics (Nom.), without torque limits (No lim.), with three times softer ground contact (Soft) and with both (one fit each). (b) Excess error (Eaff−EMLP−Ena)/EMLP of the affine MLP (3 seeds, mean and range). Ant’s Ena for p<1 is measured at states from the Minari dataset; at p=1 the filled marker uses states from the recollected data and the open one states from the Minari dataset. ∗ Walker2d’s Ena is unreliable with random actions.
Tracking large human-motion datasets has become the dominant route to general, expressive humanoid whole-body control, but scaling across many motions is expensive. Current pipelines use model-free reinforcement learning, either merging per-motion specialists through lossy distillation, or brute-forcing a single policy with cluster-scale compute. SuperMimic amortizes learning across all motions with one differentiable world model of their shared dynamics, fit from logged transitions of a standard rigid-body simulator. The world model and the tracking policy train together in one loop, and backpropagation-through-time over short imagined rollouts gives the policy analytic gradients. Fast imagined rollouts and analytic gradients make policy learning efficient. In 3 hours on a single GPU, SuperMimic surpasses the return of a state-of-the-art motion tracker baseline trained for 7 days on 128 GPUs. It reduces GPU-hours by more than 1000× and fails on only 25 of 129,514 benchmark motions, versus 179 failures for the baseline. After 9.8 hours on two GPUs, SuperMimic achieves zero failures. From 1 to 264 hours of motion, the world model's prediction error stays stable, and GPU-hours to a target return grow slowly. Our policy transfers to a physical Unitree G1 across diverse and acrobatic motions. SuperMimic removes compute as the bottleneck to scaling humanoid whole-body control, making it practical to measure motion-tracking scaling laws.
World models of embodied agents predict future observations conditioned on an action taken by the agent. For complex embodiments, action spaces are high-dimensional and difficult to specify: for example, precisely controlling a human agent requires specifying the motion of each joint. This makes the world model hard to control and expensive to plan with as search-based methods like CEM scale poorly with action dimensionality. To address this issue, we train a lightweight policy that maps high-level actions to sequences of low-level joint actions. Composing this policy with the frozen world model produces a lifted world model that predicts a sequence of future observations from a single high-level action. We instantiate this framework for a human-like embodiment, defining the high-level action space as a small set of 2D waypoints annotated on the current observation frame, each specifying a near-term goal position for a leaf joint (pelvis, head, hands). Waypoints are low-dimensional, visually interpretable, and easy to specify manually or to search over. We show that the lifted world model substantially outperforms searching directly in low-level joint space (3.8× lower mean joint error to the goal pose), while remaining more compute-efficient and generalizing to environments unseen by the policy.
Alex N. Wang, Trevor Darrell, Pavel Izmailov +2
Computer Science, New York University · BAIR, UC Berkeley
General-purpose motion trackers enable humanoid robots to follow diverse whole-body motions while maintaining balance, but are trained only on flat ground, failing to exploit bipedal mobility over complex terrain. Cross-terrain controllers, meanwhile, are task-specific or accept only low-dimensional locomotion commands. We introduce InterTrack, the first behavior world model (BWM) for robust whole-body tracking with environment interaction. Its Transformer jointly predicts the next action, state, and behavior distribution, learning environment-conditioned dynamics. To scale interaction training data, an automatic annotation pipeline reconstructs 3D support geometry from retargeted motions. At deployment, the policy handles commands implausible in the current environment in a "best-effort" manner. Quantitatively, InterTrack achieves an 81.3% success rate on terrain interaction (4.3 times the best evaluated baseline) and a 99.3% fall-recovery rate, while also improving free-space tracking and outperforming three leading tracking baselines across all of these regimes. To our knowledge, we provide the first demonstration of real-time cross-terrain whole-body teleoperation on a humanoid robot, alongside object interaction, stable responses to missing supports, and robust recovery from falls.
Ziyang Cheng, Tianshu Tang, Jinxin Lan +19
Tsinghua University · GigaAI · University of Shanghai for Science and Technology +3