OneSteer: Unlocking Steerable General Humanoid Motion Tracking via One-Step Diffusion Whole-Body Control
Organizations: Southeast University · Southern University of Science and Technology · The Hong Kong Polytechnic University · Shenzhen University ·
Abstract
Deployment objectives often specify desired outcomes without providing revised trajectories, while parts of the original reference must still be followed. We present OneSteer, a one-step diffusion whole-body controller that combines general motion tracking with selective test-time steering. To accommodate new objectives, differentiable task costs guide optimization of the generator's input noise with frozen weights, while body-group reference masks relax selected tracking requirements. Predicted state-action plans provide a shared interface for task optimization and feedback-conditioned execution. For nominal tracking, the model trained from scratch achieves the lowest local tracking error among the evaluated baselines on successful held-out AMASS and BONES clips. For steering, one frozen distilled controller adapts across diverse deployment objectives in simulation, raising the clean passage rate under a 1.25 m bar from 2% to 90%. On a physical Unitree G1, OneSteer completes all 27 nominal tracking trials across nine diverse whole-body motion sequences without falling. With a steering cost derived from a signed distance field built using only onboard LiDAR, it also passes under a 1.1 m low bar in four of five attempts and steps over a 12 cm hurdle without falling in all three attempts. Beyond control, prediction variation in the model trained from scratch ranks likely tracking failures without physics rollouts. This scoring method is more than 10x faster than simulation-based scoring with five rollouts per clip and guides targeted fine-tuning. Project page: https://onesteer.github.io
Figures & tables
| Body | Real | Trained on | Conditioning | Tracking | Steer | |
|---|---|---|---|---|---|---|
| BeyondMimic [ 30 ] | G1 | yes | VAE rollouts | goals, keyframes | no | yes |
| Diffuse-CLoC [ 20 ] | char. | no | tracker rollouts | keyframes | no | yes |
| PredActor [ 65 ] | G1 | yes | tracker rollouts | text | no | yes |
| UniPhys [ 62 ] | char. | no | tracker rollouts | text, trajectory, goal | no | yes |
| SCDP [ 6 ] | G1 | yes | expert rollouts | velocity or ref. frame | walking | no |
| OneSteer (ours) | G1 | yes | tracker / RL | reference window | general | yes |
| Method | Succ. (%) | (mm) | med. (mm) | Steer. |
|---|---|---|---|---|
| SONIC [ 35 ] (release) | 98.7 / 95.5 | 31.9 / 26.8 | 99.2 / 56.2 | no |
| SONIC [ 35 ] (v1.1) | 98.0 / 95.1 | 30.5 / 27.4 | 126.2 / 67.8 | no |
| HoloMotion [ 7 ] | 95.3 / 94.3 | 43.5 / 38.7 | 119.1 / 96.1 | no |
| HEFT [ 31 ] (pmg) | 92.7 / 95.3 | 29.4 / 23.3 | 127.0 / 67.1 | no |
| Humanoid-GPT [ 46 ] | 95.3 / 91.0 | 44.5 / 31.8 | 254.3 / 106.1 | no |
| MimicLite Huge [ 48 ] (teacher) | 94.0 / 95.3 | 29.9 / 27.0 | 138.6 / 68.0 | no |
| Hurdles | Low bar: clean pass | Reach | Latency | ||||
| Method | feasible | clean | 1.30 m | 1.25 m | 1.20 m | hit | (ms) |
| Teacher, original ref. | 0.48 | 0.01 | 0.36 | 0.02 | 0.00 | 0.01 | 2.5 |
| Teacher, offline edit (J) | 0.58 | 0.01 | 0.72 | 0.06 | 0.00 | 0.38 | 2.5 |
| Teacher, offline edit (J+R) | 0.47 | 0.00 | 0.55 | 0.03 | 0.00 | 0.31 | 2.5 |
| Teacher, closed-loop edit (J) | 0.57 | 0.02 | 0.87 | 0.11 | 0.02 | 0.60 | 330.5 |
| Teacher, closed-loop edit (J+R) | 0.45 | 0.00 | 0.67 | 0.04 | 0.02 | 0.60 | 277 |
Appendix figures & tables33 assets
Supplementary material from the paper’s appendix.
Appendix
| Reference interface | |
|---|---|
| Window | 10 future frames, 0.1 s apart |
| Frame features | 38 (root 9, legs 12, upper body 17) |
| Reference tokens | 2 FSQ tokens of 32 dims |
| Mask prob. (group / window) | 0.3 / 0.1 |
| Generator and action head | |
| Transformer | 4 layers, width 512, 8 heads |
| Method | AMASS (134) | BONES (457) |
|---|---|---|
| SONIC (release) | 29.4 [27.3, 31.8] | 25.2 [24.2, 26.3] |
| SONIC (v1.1) | 27.5 [25.9, 29.6] | 23.8 [22.8, 25.0] |
| HoloMotion | 39.5 [37.0, 42.4] | 35.4 [34.1, 37.1] |
| HEFT (pmg) | 27.5 [25.4, 30.2] | 21.9 [21.2, 22.7] |
| Humanoid-GPT | 39.2 [34.8, 44.3] | 31.3 [29.3, 33.6] |
| MimicLite Huge (teacher) | 28.1 [25.9, 30.8] | 24.2 [23.3, 25.2] |
| Planned / limit | Commanded | Saturated | Corr. planned | ||||
|---|---|---|---|---|---|---|---|
| Weak joint | Success | 16 ticks | now | / limit | ticks (%) | vs. commanded | |
| Left knee | unsteered | 0.71 | 0.85 | 0.66 | 1.00 | 13.9 | 0.88 |
| steered | 0.79 | 0.62 | 0.37 | 0.69 | 9.9 | 0.31 | |
| Left hip pitch | unsteered | 0.77 | 1.09 | 0.91 | 1.47 | 24.1 | 0.92 |
| steered | 0.86 | 0.64 | 0.48 | 1.07 | 19.7 | 0.55 | |
| Plan horizon | ||||
|---|---|---|---|---|
| Task | 1 tick | 5 ticks (0.1 s) | 10 ticks (0.2 s) | |
| Low bar 1.25 m | unsteered | / 1.5 / 5.6 | / 2.7 / 7.8 | / 3.6 / 17.7 |
| steered | 1.3 / 2.0 / 0.2 | 3.9 / 4.0 / 0.2 | 9.6 / 9.7 / 0.2 | |
| Hurdles 10 cm | unsteered | / 2.9 / 11.8 | / 3.0 / 14.3 | / 3.5 / 14.3 |
| steered | 1.2 / 2.3 / 13.1 | 2.8 / 3.5 / 14.8 | 5.5 / 5.8 / 16.2 | |
| Task | Metric | Teacher | OneSteer | OneSteer steered |
|---|---|---|---|---|
| Left knee at 15% torque | success rate | 0.69 | 0.71 | 0.79 |
| Left hip pitch at 15% | success rate | 0.78 | 0.79 | 0.84 |
| Tray balancing (4 cm rim) | payload lost rate | 0.18 | 0.16 | 0.08 |
| Velocity command tracking | velocity error (m/s) | 0.157 | 0.159 | 0.112 |
| Spatial reaching (12 steps) | target hit rate | 0.01 | 0.02 | 0.75 |
| Low bar clean | |||||
|---|---|---|---|---|---|
| Training | GPU-h | Hurdles | High-five | 1.25 m | 1.20 m |
| Distilled from teacher | 32 | 0.70 0.02 | 0.57 0.04 | 0.86 0.02 | 0.53 0.04 |
| RL from scratch | 20 | 0.37 0.06 | 0.46 0.03 | 0.93 0.02 | 0.72 0.02 |
| RL plan + distilled head | – | 0.43 0.04 | 0.56 0.02 | 0.90 0.02 | 0.68 0.04 |
| Push scale | |||||
|---|---|---|---|---|---|
| Metric | Method | 0 | 2 | 4 | 6 |
| Succ. (%) | Teacher | 95.5 | 95.6 | 95.1 | 94.8 |
| OneSteer | 95.5 | 95.9 | 95.5 | 94.6 | |
| (mm) | Teacher | 28.8 | 29.8 | 31.7 | 33.6 |
| OneSteer | 29.0 | 29.7 | 31.4 | 33.5 | |
| RTX 4090 | Orin NX | ||
|---|---|---|---|
| Setting | Gradient steps | median / p95 | median / p95 |
| Tracking, 1 call | 0 | 3.8 / – | 2.4 / 2.4 |
| Tracking, 2 calls (as on the robot) | 0 | 5.3 / 5.8 | 4.2 / 4.2 |
| Steered, 2nd call only (as in the robot runs) | 2 | 10.6–13.7 / 15.3 | 14.4 / 14.6 |
| Steered, both calls | 2 + 2 | 17.2–20.5 / 20.7 | 18.2 / 18.4 |
| Larger budget, small steps | 12 + 12 | 82.4–91.4 / 91.7 | 105.2 / 105.4 |
| Cost evaluated on | Low bar 1.25 m | Hurdles |
|---|---|---|
| – (unsteered) | 0.02 | 0.40 |
| Analytic SDF ( Tab. 3 ) | 0.90 0.01 | 0.68 0.04 |
| Voxel map, exact pose | 0.77 0.06 | 0.65 0.02 |
| drift 3.7 cm / 0.7 ∘ | 0.55 0.08 | 0.58 0.06 |
| drift 7.3 cm / 1.4 ∘ | 0.35 0.07 | 0.52 0.03 |
| drift 11 cm / 2.1 ∘ | 0.21 0.08 | 0.48 0.05 |
| Setting | fall | head (cm) | wrist (cm) | wrist rel. head (cm) |
|---|---|---|---|---|
| Full reference | 0.027 | 10.2 | 10.8 | 4.3 |
| Root from headset, no guidance | 0.093 | 13.8 | 24.7 | 19.6 |
| Waist height given, no guidance | 0.033 | 14.0 | 24.3 | 19.4 |
| Waist height given, guidance 0.5 | 0.033 | 12.1 | 16.4 | 12.9 |
| + world-model projection | 0.033 | 12.9 | 14.5 | 11.6 |
| Hurdles | Low bar: clean pass | Reach (high-five) | |||||||
|---|---|---|---|---|---|---|---|---|---|
| Method | feasible | severe | clean | succ. | 1.30 m | 1.25 m | 1.20 m | hit | dist. (cm) |
| Teacher, original reference | 0.48 | 0.33 | 0.01 | 0.63 | 0.36 | 0.02 | 0.00 | 0.01 | 20 |
| Teacher, offline-edited reference | 0.58 | 0.29 | 0.01 | 0.78 | 0.72 | 0.06 | 0.00 | 0.38 | 12 |
| Teacher, closed-loop edited reference | 0.57 | 0.25 | 0.02 | 0.78 | 0.87 | 0.11 | 0.02 | 0.60 | 8 |
| Teacher, offline edit with root | 0.47 | 0.36 | 0.00 | 0.67 | 0.55 | 0.03 | 0.00 | 0.31 | 13 |
| Teacher, closed-loop edit with root | 0.45 | 0.33 | 0.00 | 0.69 | 0.67 | 0.04 | 0.02 | 0.60 | 9 |
| Low bar clean | |||||
|---|---|---|---|---|---|
| Hurdles | 1.30 m | 1.25 m | 1.20 m | Reach | |
| Teacher, original ref. | 0.48 0.02 | 0.36 0.00 2 | 0.02 0.00 2 | 0.00 0.00 2 | 0.01 0.01 2 |
| Teacher, offline edit (J) | 0.58 0.02 | 0.72 0.01 2 | 0.06 0.00 2 | 0.00 0.00 2 | 0.38 0.01 |
| Teacher, offline edit (J+R) | 0.47 0.05 | 0.55 0.05 | 0.03 0.00 | 0.00 0.00 | 0.31 0.02 |
| Teacher, closed-loop edit (J) | 0.57 0.04 | 0.87 0.02 | 0.11 0.00 | 0.02 0.00 | 0.60 0.01 |
| Teacher, closed-loop edit (J+R) | 0.45 0.09 | 0.67 0.05 | 0.04 0.01 | 0.02 0.00 | 0.60 0.01 |
| Task | Editor (teacher) | Difference | 95% interval | Sign test |
|---|---|---|---|---|
| Hurdles, feasible | offline edit (J) | 0.03 | ||
| closed-loop edit (J) | 0.03 | |||
| Low bar 1.30 m, clean | offline edit (J) | |||
| closed-loop edit (J) | 0.15 | |||
| Low bar 1.25 m, clean | closed-loop edit (J) | |||
| Low bar 1.20 m, clean | closed-loop edit (J) |
| Reference hidden from | |||
| Task where the reference conflicts with the cost | none | generator | generator + executor |
| Velocity command, unguided: velocity error (m/s) | 0.218 | 0.205 | 0.159 |
| Velocity command, unguided: heading error ( ∘ ) | 14.7 | 14.1 | 6.2 |
| Velocity command, root-position cost: velocity error (m/s) | 0.173 | 0.168 | 0.139 |
| Velocity command, root-position cost: heading error ( ∘ ) | 8.2 | 7.3 | 2.4 |
| Left knee at 15%, guided: success rate | 0.79 | 0.79 | 0.82 |
| Arm reference hidden from | Task / | Legs ( ∘ ) | Waist ( ∘ ) | Left arm ( ∘ ) | Right arm ( ∘ ) | Root height (cm) | Heading ( ∘ ) | Drift (cm) | |
|---|---|---|---|---|---|---|---|---|---|
| Reaching (hit rate ) | none, no steering | 0.00 | 4.40 | 2.21 | 2.92 | 3.27 | 0.74 | 2.39 | 20.7 |
| none | 0.34 | 4.49 | 2.24 | 3.46 | 4.13 | 0.76 | 2.57 | 22.4 | |
| generator | 0.29 | 4.49 | 2.25 | 3.48 | 4.26 | 0.76 | 2.58 | 22.3 | |
| executor (default) | 0.53 | 4.53 | 2.19 | 5.66 | 7.27 | 0.77 | 2.53 | 22.2 | |
| generator + executor, no steering | 0.03 | 4.38 | 2.12 | 7.82 | 9.78 | 0.74 | 2.28 | 19.0 | |
| generator + executor | 0.65 | 4.54 | 2.23 | 10.81 | 14.79 | 0.77 | 2.59 | 24.8 |
| Bar | Method | Arms ( ∘ ) | Heading ( ∘ ) | Height (cm) | Legs † ( ∘ ) | (mm) | Success |
|---|---|---|---|---|---|---|---|
| 1.30 m | teacher, unsteered | 3.5 | 2.3 | 0.8 | 3.7 | 24.5 | 1.00 |
| OneSteer, unsteered | 3.1 | 2.6 | 0.9 | 4.8 | 29.3 | 1.00 | |
| OneSteer, steered | 3.1 | 2.8 | 0.9 | 4.9 | 29.9 | 1.00 | |
| 1.25 m | teacher, unsteered | 4.0 | 3.0 | 1.0 | 4.6 | 31.8 | 1.00 |
| OneSteer, unsteered | 3.4 | 3.5 | 1.0 | 5.7 | 37.5 | 1.00 | |
| OneSteer, steered | 3.2 | 2.8 | 1.1 | 5.5 | 32.8 | 1.00 |
| Success | Flat | Down | Up | Slope | Boxes | Rough | All |
|---|---|---|---|---|---|---|---|
| No terrain input | 0.87 | 0.53 | 0.16 | 0.55 | 0.19 | 0.32 | 0.43 |
| Terrain tokens masked | 0.90 | 0.81 | 0.25 | 0.67 | 0.42 | 0.37 | 0.55 |
| Terrain tokens (ours) | 0.97 | 0.99 | 0.91 | 1.00 | 0.86 | 0.76 | 0.90 |
| Flat tracking | Hurdles, success | |||
|---|---|---|---|---|
| Model | success | (mm) | unsteered | steered |
| No terrain input | 0.963 | 26.0 | 0.32 | 0.55 |
| Terrain-conditioned | 0.963 | 27.6 | 0.65 | 0.76 |
| tokens off | 0.965 | 27.8 | 0.77 | 0.76 |
| Variant (hurdles) | feasible | severe | succ. |
|---|---|---|---|
| Both calls, 2 steps per call, | 0.68 | 0.18 | 0.91 |
| 1 step per call | 0.51 | 0.29 | 0.75 |
| 4 steps per call | 0.70 | 0.19 | 0.95 |
| 6 steps per call + line search | 0.74 | 0.16 | 0.95 |
| Strength / | 0.56 / 0.66 | 0.23 / 0.19 | 0.87 / 0.91 |
| Best-of-8, no gradient | 0.38 | 0.42 | 0.58 |
| Low bar clean | Tray | |||||
|---|---|---|---|---|---|---|
| Steered call | Hurdles | 1.30 m | 1.25 m | 1.20 m | Reach | lost |
| None | 0.40 | 0.23 | 0.02 | 0.00 | 0.02 | 0.16 |
| First of two | 0.48 0.01 | 0.32 0.02 | 0.02 0.00 | 0.00 0.00 | 0.00 0.00 | 0.13 0.00 |
| Second of two (real-time) | 0.68 0.04 | 0.95 0.02 | 0.90 0.01 | 0.56 0.04 | 0.57 0.03 | 0.15 0.01 |
| Both of two | 0.66 0.03 | 0.95 0.01 | 0.84 0.02 | 0.52 0.04 | 0.54 0.06 | 0.08 0.01 |
| One call | 0.58 0.02 | – | 0.76 0.04 | – | 0.43 0.02 | – |
| Low bar clean | |||||
|---|---|---|---|---|---|
| Where the cost acts | Hurdles | 1.30 m | 1.25 m | 1.20 m | Reach |
| Unsteered | 0.40 | 0.23 | 0.02 | 0.00 | 0.02 |
| Generator input, gradient (ours) | 0.68 0.04 | 0.95 0.02 | 0.90 0.01 | 0.56 0.04 | 0.57 0.03 |
| Generator input, random direction | 0.41 0.04 | 0.28 0.02 | 0.02 0.00 | 0.00 0.00 | 0.04 0.00 |
| Generator input, sampling (MPPI, 8) | 0.47 0.02 | 0.51 0.01 | 0.02 0.00 | 0.00 0.00 | 0.03 0.01 |
| Generator input, gradient, | 0.68 0.02 | 0.95 0.02 | 0.40 0.02 | 0.10 0.03 | 0.39 0.01 |
| Hurdles | Low bar clean | Upper-body | ||||||
|---|---|---|---|---|---|---|---|---|
| Generator, steering | Updates | feasible | clean | 1.30 m | 1.25 m | 1.20 m | Reach | jerk |
| Flow, unsteered (2 Euler steps) | 0 | 0.46 | 0.02 | – | 0.02 | – | 0.01 | 400 |
| Flow, 2 Euler steps | 2 | 0.71 0.01 | 0.14 0.01 | 0.96 0.01 | 0.82 0.02 | 0.51 0.02 | 0.60 0.04 | 915 |
| Flow, 4 Euler steps | 4 | 0.67 0.05 | 0.29 0.02 | 0.94 0.00 | 0.70 0.07 | 0.44 0.06 | 0.73 0.03 | 963 |
| OneSteer, unsteered | 0 | 0.40 | 0.00 | 0.23 | 0.02 | 0.00 | 0.02 | 381 |
| OneSteer, 2nd call, 2 steps | 2 | 0.68 0.04 | 0.11 | 0.95 0.02 | 0.90 0.01 | 0.56 0.04 | 0.57 0.03 | 571 |
| Hurdles | 1.25 m bar | Reach | Tracking (mm) | ||
| 0.6 | 0.74 0.03 | 0.85 0.02 | 0.53 0.01 | 28.9 | |
| 0.7 | 0.73 0.03 | 0.91 0.02 | 0.57 0.01 | 28.9 | |
| 0.8 (used) | 0.68 0.04 | 0.90 0.01 | 0.57 0.03 | 29.0 | |
| 0.9 | 0.60 0.04 | 0.86 0.02 | 0.55 0.02 | 29.0 | |
| Iteration | 600 | 0.63 | 0.76 | 0.58 | 28.7 |
| 1,200 | 0.74 | 0.74 | 0.52 | 29.0 |
| Low bar clean | ||||||
|---|---|---|---|---|---|---|
| Windows | Method | Hurdles | 1.30 m | 1.25 m | 1.20 m | Reach |
| Original | teacher, original ref. | 0.48 | 0.36 | 0.02 | 0.00 | 0.01 |
| teacher, offline edit | 0.58 | 0.72 | 0.06 | 0.00 | 0.38 | |
| teacher, closed-loop edit | 0.57 | 0.87 | 0.11 | 0.02 | 0.60 | |
| OneSteer, unsteered | 0.40 | 0.23 | 0.02 | 0.00 | 0.02 | |
| OneSteer, 2nd call | 0.68 0.04 | 0.95 0.02 | 0.90 0.01 | 0.56 0.04 | 0.57 0.03 | |
| Tracking (512) | Steering, all calls (79) | ms per step | |||||
|---|---|---|---|---|---|---|---|
| Succ. | (mm) | Hurdles | 1.25 m | Reach | Track | Steer | |
| 1 | 95.3 | 29.2 | 0.58 0.02 | 0.76 0.04 | 0.43 0.02 | 3.9 | 13.0 |
| 2 | 95.5 | 29.0 | 0.68 0.01 | 0.84 0.02 | 0.54 0.06 | 5.2 | 20.5 |
| 4 | 95.5 | 28.9 | 0.73 0.01 | 0.55 0.04 | 0.78 0.01 | 8.2 | 35.3 |
| 8 | 95.3 | 29.0 | 0.70 0.01 | 0.45 0.03 | 0.72 0.01 | 13.9 | 64.9 |
| Score | Needs | AUC | Prec. top 2% | Prec. / rec. top 5% |
|---|---|---|---|---|
| Noise spread (ours) | generator | 0.83 | 0.91 | 0.68 / 0.50 |
| Plan deviation | generator | 0.82 | 0.68 | 0.44 / 0.33 |
| Kinematic rarity (kNN) | library | 0.81 | 0.23 | 0.32 / 0.23 |
| Critic value | RL critic | 0.86 | 0.85 | 0.57 / 0.42 |
| Imagined rollout | policy, world model | 0.91 | 0.93 | 0.71 / 0.52 |
| Random | – | 0.50 | 0.07 | 0.07 / 0.05 |
| Hard clips selected by | Fixed | Broken | Fail rate | Unseen fam. | Seen fam. | Hard |
|---|---|---|---|---|---|---|
| None (base re-evaluated) | 12.2 4.8 | 0.8 0.3 | 6.4 0.0 | 24.5 2.6 | 9.7 2.3 | 33.6 |
| Random (same budget) | 26.9 2.0 | 1.6 0.2 | 6.2 0.1 | 34.8 3.8 | 17.2 6.5 | 34.2 0.7 |
| Random, same candidate pool | 35.1 5.9 | 1.3 0.3 | 5.4 0.4 | 35.8 8.2 | 25.8 4.3 | 33.9 1.2 |
| Imagined rollout | 46.3 2.8 | 1.3 0.1 | 4.7 0.2 | 43.0 1.0 | 39.3 0.9 | 35.1 1.0 |
| Critic value | 41.2 2.3 | 1.3 0.1 | 5.0 0.2 | 37.9 2.8 | 31.7 3.4 | 35.0 0.8 |
| Kinematic rarity | 30.5 3.1 | 1.3 0.1 | 5.7 0.3 | 35.2 3.7 | 19.9 0.9 | 34.0 1.1 |
Humanoid Motion Tracking
Humanoid motion tracking trains robots to reproduce human-like whole-body movements while remaining physically stable and feasible.
See all 86 papers