MetaWorld-X: Hierarchical World Modeling via VLM-Orchestrated Experts for Humanoid Loco-Manipulation
Authors: Yutong Shen, Hangxu Liu, Penghui Liu, Jiashuo Luo, Yongkang Zhang, Rex Morvley, Chen Jiang, Jianwei Zhang, Lei Zhang
Organizations: School of Information Science and Technology, Beijing University of Technology, Beijing, China · School of Information Science and Engineering, Fudan University, Shanghai, China · University of Alberta, Canada · University of Hamburg, Hamburg, Germany
Learning natural, stable, and compositionally generalizable whole-body control policies for humanoid robots performing simultaneous locomotion and manipulation (loco-manipulation) remains a fundamental challenge in robotics. Existing reinforcement learning approaches typically rely on a single monolithic policy to acquire multiple skills, which often leads to cross-skill gradient interference and motion pattern conflicts in high-degree-of-freedom systems. As a result, generated behaviors frequently exhibit unnatural movements, limited stability, and poor generalization to complex task compositions. To address these limitations, we propose MetaWorld-X, a hierarchical world model framework for humanoid control. Guided by a divide-and-conquer principle, our method decomposes complex control problems into a set of specialized expert policies (Specialized Expert Policies, SEP). Each expert is trained under human motion priors through imitation-constrained reinforcement learning, introducing biomechanically consistent inductive biases that ensure natural and physically plausible motion generation. Building upon this foundation, we further develop an Intelligent Routing Mechanism (IRM) supervised by a Vision-Language Model (VLM), enabling semantic-driven expert composition. The VLM-guided router dynamically integrates expert policies according to high-level task semantics, facilitating compositional generalization and adaptive execution in multi-stage loco-manipulation tasks.
Figures & tables
Fig. 2 : MetaWorld-X achieves natural humanoid control through the dynamic orchestration of expert policies guided by a Vision-Language Model (VLM). The framework consists of two core modules: the Skill Expert Pool (SEP) trains specialized policy networks for fundamental motor skills using human motion data, while the Intelligent Router Module (IRM) employs a VLM as a supervisory teacher. The VLM guides the router’s training via few-shot inference, enabling a seamless transition from supervised learning to autonomous operation.
Fig. 3 : Architecture of the SEP module. We project human motion priors to the robot’s configuration space via operator M , then utilize alignment operator A to compute tracking signals for expert policies πθi . By optimizing JSEP with a dynamic weighting mechanism that prioritizes poorly-tracked joints, the module yields the converged expert policies.
Fig. 4 : Example of VLM prompt for Semantic Router.
Algorithm
Stand
Walk
Run
Sit
Carry
R. ± stderr.
S.(M)
R. ± stderr.
S.(M)
R. ± stderr.
S.(M)
R. ± stderr.
S.(M)
R. ± stderr.
S.(M)
PPO [ 28 ]
28.8 ± 0.4
5.3
28.8 ± 0.4
5.3
28.8 ± 0.4
5.3
28.8 ± 0.4
5.3
25.9 ± 0.4
5.4
SAC [ 29 ]
93.6 ± 17.7
4.9
32.7 ± 11.4
5.4
18.3 ± 1.5
5.4
275.9 ± 15.7
6.0
43.4 ± 9.7
5.3
TD-MPC2 [ 7 ]
749.8 ± 63.1
1.8
644.2 ± 162.3
1.8
66.1 ± 4.7
2.0
733.9 ± 120.6
1.1
438.0 ± 72.9
1.9
DreamerV3 [ 8 ]
699.3 ± 62.7
5.5
428.2 ± 14.5
6.0
298.5 ± 84.5
6.0
709.6 ± 98.4
5.2
427.8 ± 51.2
6.0
Ours (IRM)
815.9 ± 0.3 ↑
0.6 ↓
1118.7 ± 7.1 ↑
0.5 ↓
2056.9 ± 13.6 ↑
1.0 ↓
862.2 ± 2.1 ↑
0.6 ↓
963.5 ± 5.1 ↑
0.5 ↓
TABLE I: Expert policy evaluation: Our IRM module vs. baselines. We report maximum returns ( R.±stderr. ) and convergence steps S in Millions (M). ↑ indicates higher is better, ↓ indicates lower is better.
Fig. 5 : We evaluate the performance of four foundational motor skills: Walking, Running, Standing, and Sitting , after 50w training steps. The dashed lines in the learning curves qualitatively represent the reward thresholds associated with the successful execution of each skill.
Fig. 6 : In comparison with TD-MPC2, Green shaded areas highlight the superior performance of Ours , whereas red regions indicate limitations observed in baseline algorithms.
Method
Stand
Walk
Run
Sit
Carry
PPO [ 28 ]
0/10
0/10
0/10
0/10
0/10
SAC [ 29 ]
1/10
1/10
0/10
2/10
1/10
DreamerV3 [ 8 ]
2/10
2/10
1/10
3/10
2/10
TD-MPC2 [ 7 ]
3/10
3/10
2/10
4/10
3/10
Ours
9/10
9/10
9/10
8/10
9/10
TABLE II: Performance comparison on locomotion tasks (Success Rate / 10). Ours outperforms all baseline RL and model-based algorithms.
Fig. 7 : We evaluated the success rates of our method and Others on basic expert skills over 500k training iterations. For each task, ten independent trials were conducted. A trial was considered successful if the agent did not fall within a 30‑second episode and complete the task. The results demonstrate that after 500k steps, our method achieves a significantly higher success rate than Others. This confirms the robustness and reliability of our expert‑policy learning in these locomotion tasks.
Fig. 8 : The effectiveness of each ablated method on the door-opening task is illustrated in this figure.
Method
Suceess
steps(w)
return
TD-MPC2 [ 7 ]
✓
32.38
198.42
w/o Router
✓
20.36
296.57
w/o VLM
×
NULL
NULL
w/o IL
×
∞
193.61
Full Model
✓
12.64
303.95
TABLE III : Among the ablated variants, the model without Imitation Learning (IL) exhibits poor performance, primarily due to its incompatibility with our subsequent training and adaptation pipeline. Meanwhile, the without Vision-Language Model (VLM) configuration is not trained, as we aim to directly employ a rule-based router to achieve zero-shot compositional generalization without semantic guidance.
Fig. 9 : This figure illustrates a performance comparison between Ours and several baselines across various manipulation tasks. The plot includes a ”Reward Switcher” curve, which indicates the specific reference expert policy being utilized for reinforcement learning at each stage of the training process.
Task
Router (M)
PPO [ 28 ]
SAC [ 29 ]
TD-MPC2 [ 7 ]
DreamerV3 [ 8 ]
Ours
Door
0.17(S) → 0.74(W) → 2.0(R)
58.0±13.5
168.7±3.9
285.0±12.0
257.7±37.9
470.0±2.2
Basketball
0.87(S) → 2.0(R)
58.0±1.3
45.4±5.5
148.4±3.3
77.6±1.2
250.0±11.9
Push
1.09(S) → 2.0(R)
36.8±11.0
−201.3±13.6
−113.8±6.8
−59.9±7.3
70.0±2.1
Truck
0.57(S) → 1.61(W) → 2.0(R)
58.0±13.5
1142.6±15.8
1213.2±1.1
1383.4±2.5
1500.0±15.6
Package
0.51(S) → 2.0(R)
−8112.3±191.1
−6662.1±137.2
−6788.5±552.7
−5817.2±903.8
−5200.0±47.2
TABLE IV : Performance comparison on manipulation tasks. Routing schedules are indicated in Million steps (M) under each task, where S , W , and R denote Stand , Walk , and Reach reference stages, respectively.
Long-horizon humanoid loco-manipulation requires composing versatile whole-body skills and reliable high-level decision making. Existing methods often coordinate pretrained skills with scripted planners, finite-state machines or task-specific model-free policies, restricting their ability to handle complex task sequences. To address this limitation, we propose LUCID, a hierarchical model-based reinforcement learning framework that plans over reusable skills through imagined rollouts of a learned dynamics model. LUCID first trains a structured latent-conditioned low-level policy via adversarial imitation and then freezes it while jointly learning a high-level policy and macro-dynamics world model. The world model predicts the temporally extended state transitions induced by latent decisions, enabling high-level policy optimization through imagined rollouts. We evaluate our framework across various simulated multi-object rearrangement scenarios. Experimental results show that LUCID improves the full-task success and partial-completion rates compared to prior baseline methods, demonstrating its effectiveness in complex sequential loco-manipulation tasks.
Cheng Guo, Mingzhe Ni, Angelo Cangelosi +1
Department of Computer Science, University of Manchester, Manchester, UK · Human-Robot Interfaces and Interaction Laboratory, Italian Institute of Technology, Genoa, Italy
Humans achieve complex manipulation through coordinated whole-body control, whereas most Vision-Language-Action (VLA) models treat robot body parts largely independently, making high-DoF humanoid control challenging and often unstable. We present HEX, a state-centric framework for coordinated manipulation on full-sized bipedal humanoid robots. HEX introduces a humanoid-aligned universal state representation for scalable learning across heterogeneous embodiments, and incorporates a Mixture-of-Experts Unified Proprioceptive Predictor to model whole-body coordination and temporal motion dynamics from large-scale multi-embodiment trajectory data. To efficiently capture temporal visual context, HEX uses lightweight history tokens to summarize past observations, avoiding repeated encoding of historical images during inference. It further employs a residual-gated fusion mechanism with a flow-matching action head to adaptively integrate visual-language cues with proprioceptive dynamics for action generation. Experiments on real-world humanoid manipulation tasks show that HEX achieves state-of-the-art performance in task success rate and generalization, particularly in fast-reaction and long-horizon scenarios.
Shuanghao Bai, Meng Li, Xinyuan Lv +15
Beijing Innovation Center of Humanoid Robotics · Xi’an Jiaotong University · Nankai University +1
While recent advances have demonstrated strong performance in individual humanoid skills such as upright locomotion, fall recovery and whole-body coordination, learning a single policy that masters all these skills remains challenging due to the diverse dynamics and conflicting control objectives involved. To address this, we introduce X-Loco, a framework for training a vision-based generalist humanoid locomotion policy. X-Loco trains multiple oracle specialist policies and adopts a synergetic policy distillation with a case-adaptive specialist selection mechanism, which dynamically leverages multiple specialist policies to guide a vision-based student policy. This design enables the student to acquire a broad spectrum of locomotion skills, ranging from fall recovery to terrain traversal and whole-body coordination skills. To the best of our knowledge, X-Loco is the first framework to demonstrate vision-based humanoid locomotion that jointly integrates upright locomotion, whole-body coordination and fall recovery, while operating solely under velocity commands without relying on reference motions. Experimental results show that X-Loco achieves superior performance, demonstrated by tasks such as fall recovery and terrain traversal. Ablation studies further highlight that our framework effectively leverages specialist expertise and enhances learning efficiency.
Dewei Wang, Xinmiao Wang, Chenyun Zhang +4
University of Science and Technology of China · Institute of Artificial Intelligence (TeleAI), China Telecom · Harbin Engineering University