Continual Humanoid Motion Learning
Organizations: New York University Abu Dhabi, Abu Dhabi, United Arab Emirates. · Fudan University, Shanghai, China.
Abstract
Humanoid whole-body controllers can now track a diverse set of dynamic motions, but they are typically trained offline and then frozen, so teaching such a controller a new skill tends to erode the skills it already mastered. We study continual learning for humanoid whole-body motion, where a single controller must acquire skills from a sequential task stream without revisiting past data. We introduce Similarity-guided LoRA-PNN, a progressive neural network (PNN) policy that prevents catastrophic forgetting by construction while reusing knowledge across skills through lightweight low-rank adaptation. A two-level motion-similarity measure, built from dynamic time warping aggregated by optimal transport, decides which prior skill to build on and how much new capacity to allocate, yielding strong forward transfer and large efficiency gains. Across six sequentially learned skill categories, our similarity-guided LoRA policy attains the best forward transfer (0.125 vs. 0.079) and the highest average accuracy among all methods, while saving up to 94.5% of trainable parameters and 40.8% of training time. The resulting controller reaches 96.13% sim-to-sim transfer and is deployed on a physical Unitree G1. Our code is available at https://anonymous.4open.science/r/continual-humanoid-learning-35D3.
Figures & tables
| Strategy | Rank Rule |
| Fixed | |
| Binned | |
| Linear |
| Observation / signal | Teacher | Student | Deployable | Notes |
| Proprioception (common, deployable) | ||||
| dof_pos | ✓ | ✓ | ✓ | Joint positions |
| dof_vel | ✓ | ✓ | ✓ | Joint velocities |
| actions | ✓ | ✓ | ✓ | Previous action |
| next_step_ref_motion | ✓ | ✓ | ✓ | Reference-motion conditioning (available in deployment when tracking a given motion) |
| history | ✓ | ✓ | Past frames of | |
| Networks | actor MLP , critic (LayerNorm), Conv1D encoder , SiLU |
| PPO | lr , clip , , GAE , entropy , steps, mini-batches, epochs |
| Budget | K iterations, envs, evaluation episodes per task |
| LoRA | , , dropout |
| Rank rules | , , , ( binned) |
| Sim. windows | s |
| Metric | Description | Definition |
| AA | Average accuracy over the tasks learned so far (overall continual performance after stage ) | |
| AIA | Average incremental accuracy: the historical average of AA up to stage (quality of the whole learning trajectory) | |
| FM | Forgetting measure: how much performance on previous tasks degrades after learning new tasks, relative to the historical best | |
| BWT | Backward transfer: net change on past tasks after learning subsequent tasks; negative values indicate forgetting | |
| IM | Intransigence: gap to the joint-training reference, i.e . , how far the continual learner is from the multi-task upper bound | |
| FWT | Forward transfer: benefit (or harm) on a new task before learning it, relative to a single-task reference trained from scratch |
| Metric | Unit | Meaning | Formula |
| Succ. | % | Success rate over evaluation episodes (episode finishes without failure or early termination) | \frac{1}{N}\sum_{n=1}^{N}\mathbb{I}[\text{episode n succeeds}] |
| mm | Mean per-joint position error, root-relative (global translation removed by aligning the root) | ||
| mm | Global MPJPE: world-frame joint position error, no root alignment | ||
| mm | Mean per-body keypoint position error over a predefined set (key bodies / end-effectors) | ||
| mm/s | Mean per-joint velocity error (finite difference) | ||
| mm/s 2 | Mean per-joint acceleration error (second finite difference) |
| Method | AA | AIA | FM | BWT | IM | FWT |
| Multi-motion humanoid baselines | ||||||
| KungfuBot2 | 0.920 0.058 | 0.946 0.053 | 0.088 0.006 | -0.089 0.006 | -0.028 0.003 | 0.079 0.005 |
| ASAP | 0.809 0.044 | 0.917 0.052 | 0.222 0.122 | -0.222 0.122 | -0.025 0.003 | 0.059 0.003 |
| HumanoidVerse | 0.827 0.023 | 0.898 0.026 | 0.204 0.007 | -0.203 0.006 | -0.028 0.000 | 0.060 0.002 |
| Standard CL methods | ||||||
| EWC | 0.488 0.026 | 0.645 0.032 | 0.220 0.121 | -0.220 0.120 | – | – |
| Method | Succ. | |||||||
| KungfuBot2 | 100% | 87.38 | 147.84 | 29.81 | 105.64 | 90.45 | 4.27 | 3.54 |
| ASAP | 100% | 77.0 | 129.0 | – | – | – | 2.69 | 5.65 |
| HumanoidVerse | 97.8% | 122.12 | 146.16 | 28.87 | 98.44 | 90.61 | 4.17 | 3.55 |
| Ours | 100% | 80.7 | 135.36 | 30.67 | 91.79 | 84.92 | 4.72 | 4.23 |
| Variant | AA | AIA | FM | Trainable Params | Saved Params (%) | Time (h/5k) | Saved Time (%) |
| w/o PNN (single backbone) | 0.843 0.024 | 0.892 0.028 | 0.137 0.004 | 2.4M | – | – | – |
| PNN, full FT (w/o LoRA) | 0.938 0.043 | 0.949 0.044 | 0.000 | 2.4M | – | 2.94 | – |
| PNN + LoRA, w/o sim. init | 0.896 0.041 | 0.923 0.044 | 0.000 | 132K | 94.50 | 1.74 | 40.82 |
| PNN + LoRA, fixed rank | 0.895 0.038 | 0.929 0.041 | 0.000 | 132K | 94.50 | 1.74 | 40.82 |
| PNN + LoRA, binned rank | 0.928 0.025 | 0.946 0.036 | 0.000 | 182K | 92.42 | 2.28 | 22.55 |
| PNN + LoRA, linear rank | 0.945 0.030 | 0.964 0.032 | 0.000 | 218K | 90.92 | 2.75 | 6.53 |
| Success rate | MPJPE | |||
| Rank rule | AA | AIA | AA | AIA |
| Fixed | 0.895 0.038 | 0.929 0.041 | -116.542 3.793 | -128.423 4.841 |
| Binned | 0.928 0.025 | 0.946 0.036 | -98.062 2.783 | -94.542 3.767 |
| Linear | 0.945 0.030 | 0.964 0.032 | -88.524 4.173 | -94.303 2.510 |
| Transfer Success Rate (%) | (%) | (%) |
| Distillation strategy | Sim-to-sim SR (%) |
| Task-wise teachers unified student | |
| Continual teacher continual student ( Ours ) |
Appendix figures & tables4 assets
Supplementary material from the paper’s appendix.
Appendix
| Parameter | Value | Description |
| Learning rate | For both actor and critic | |
| Clip parameter | PPO clipping range | |
| Discount factor | Reward discount | |
| GAE | Generalized advantage estimation | |
| Value-loss coefficient | Weight of the value-function loss | |
| Entropy coefficient | Entropy regularization weight |
| Component | Configuration |
| Motion encoder | Conv1D, hidden dimension , output dimension , SiLU |
| History encoder | Conv1D, hidden dimension , output dimension , SiLU |
| Privileged encoder | MLP , output dimension , SiLU |
| Task column (actor) | MLP , SiLU |
| Critic | MLP , SiLU, LayerNorm |
| Lateral connection | Linear adapter, zero-initialized |
| Parameter | Range |
| Link mass | Scaled by |
| Link inertia | Scaled by |
| Base COM offset | : 0.05\text{,}\mathrm{m} ; $z$ : $\pm$0.01\text{\,}\mathrm{m} |
| PD gains ( , ) | Scaled by |
| Ground friction | Uniform in |
| Torque RFI | Additive noise in |
| Motion-tracking rewards | |||
| Term | Weight | Description | |
| teleop_root_vel | 1.0 | 1.0 | Root velocity tracking |
| teleop_anchor_body_position | 1.5 | 0.1 | Anchor-body position tracking |
| teleop_anchor_body_rotation | 1.5 | 0.15 | Anchor-body rotation tracking |
| local_key_body_position | 3.0 | 0.1 | Local key-body position tracking |
| local_key_body_rotation | 2.0 | 0.15 | Local key-body rotation tracking |