MAPO: Mixture of Experts with Alternating Policy Optimization for Universal Humanoid Motion Tracking
Organizations: School of Electrical Engineering, KAIST · Robotics Program, KAIST · Graduate School of AI, KAIST · Config
Abstract
Universal humanoid motion tracking is essential for enabling humanoid robots to perform diverse tasks in human environments. To capture the wide diversity of human motions, mixture-of-experts (MoE) policies have been widely adopted, leveraging dynamic routing to enhance policy expressiveness. However, we identify that divergence between old and updated policies can be misestimated during policy-gradient updates of MoE policies, due to their dynamic routing. Because most policy-gradient algorithms regulate policy updates based on the divergence, this misestimation can lead to either overly conservative or excessively large updates, which can yield a suboptimal policy. In this paper, we propose Mixture of Experts with Alternating Policy Optimization (MAPO), which alternately optimizes the router and the experts to avoid the divergence misestimation. MAPO consistently tracks 182 motions that PPO with MoE fails to track, whereas PPO with MoE tracks only one motion that MAPO fails to track. MAPO also attains the highest tracking success rate among the compared methods and the lowest tracking error across most error metrics, demonstrating its superior performance. Videos can be found on the project website: https://mapo-corl2026.github.io/MAPO/
Figures & tables
| (a) Training set (AMASS [ 9 ] + LAFAN1 [ 10 ] ) | |||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| Method | Success | ||||||||||
| Monolithic PPO | 81.78 % | 165.7 | 96.50 | 5.787 | 16.35 | 0.060 | 0.532 | 0.169 | 0.136 | ||
| PPO with MoE | 83.38 % | 153.1 | 87.98 | 5.499 | 14.30 | 0.058 | 0.501 | 0.120 | 0.105 | ||
| MAPO with fixed router | 83.78 % | 152.7 | 86.65 | 5.250 | 14.42 | 0.052 | 0.487 | 0.144 | 0.122 | ||
| MAPO (Ours) | 86.60 % | 147.4 | 78.10 | 4.576 | 11.78 | 0.048 | 0.406 | 0.138 | 0.117 | ||
| Ablation on | PPO with MoE | 83.96 % | 156.5 | 85.01 | 5.301 | 13.75 | 0.054 | 0.490 | 0.131 | 0.119 | |
Appendix figures & tables9 assets
Supplementary material from the paper’s appendix.
Appendix
| Reward term | Equation | Weight |
|---|---|---|
| Root orientation | 1.0 | |
| Relative body-link position | 20.0 | |
| Relative body-link orientation | 1.0 | |
| Body-link linear velocity | 1.0 | |
| Body-link angular velocity | 1.0 | |
| Action rate | -0.2 |
| Hyperparameter | Value |
|---|---|
| Num parallel environments | 4096 |
| Num steps per environment | 24 |
| Num mini-batches | 4 |
| Learning epochs | 5 |
| Discount | 0.99 |
| GAE | 0.95 |
| Domain Randomization Parameter | Value |
|---|---|
| Joint P gain scale | |
| Joint D gain scale | |
| Root mass | + default kg |
| Root center of mass offset (x, y, z) | m |
| Control delay | ms |
| Push robot magnitude | m/s |
Humanoid Motion Tracking
Humanoid motion tracking trains robots to reproduce human-like whole-body movements while remaining physically stable and feasible.
See all 86 papers