LAW: Unifying Language, Action, and World Modeling for Agentic Physical Intelligence
Organizations: State Key Laboratory of Multimedia Information Processing, School of Computer Science, Peking University · Beijing Innovation Center of Humanoid Robotics
Abstract
World-action models jointly predict actions and future world states, but long-horizon manipulation further requires language reasoning to plan subtask objectives, assess execution progress, and guide action selection. We present LAW, a closed-loop VLM--WAM framework that unifies language reasoning, action generation, and world modeling through semantic subtask progress. Within the LAW agent, a hybrid-expert vision-language model plans the remaining subtasks, estimates the values of and fuses actions proposed by a world-action model, and asynchronously assesses execution status to support continued execution and subtask transitions. Within the world-action model, Mixture-of-Transformer Experts integrates modality-specific and shared computation for joint video--action modeling. Dual-stream hierarchical visual memory maintains temporal context for both models, while subtask-local value learning aligns action evaluation with semantic objectives and preserves supervision from successful prefixes of failed trajectories. LAW achieves 94.21% average success across 50 RoboTwin tasks. Replacing the WAM backbone with OpenWAM- further raises the average success rate to 95.56%, establishing a new open-source state of the art on RoboTwin. Despite having only 2B parameters, our VLM also outperforms variants using Embodied-R1.5 and GPT-5.5 High. On three long-horizon real-robot tasks, LAW achieves 65% average success, compared with 28.33% for and 25% for Fast-WAM.
Figures & tables
Appendix figures & tables9 assets
Supplementary material from the paper’s appendix.
Appendix
| Task | LAW + OpenWAM- | LAW + Fast-WAM | Fast-WAM | LingBot-VA | ||||||
| Clean (%) | Rand. (%) | Clean (%) | Rand. (%) | Clean (%) | Rand. (%) | Clean (%) | Rand. (%) | Clean (%) | Rand. (%) | |
| Adjust Bottle | 100 | 100 | 100 | 100 | 100 | 100 | 90 | 94 | 100 | 99 |
| Beat Block Hammer | 97 | 100 | 96 | 99 | 99 | 97 | 96 | 98 | 96 | 93 |
| Blocks Ranking RGB | 100 | 99 | 95 | 93 | 100 | 100 | 99 | 98 | 92 | 85 |
| Blocks Ranking Size | 91 | 91 | 79 | 75 | 94 | 98 | 94 | 96 | 49 | 26 |
| Click Alarmclock | 99 | 99 | 100 | 99 | 100 | 100 | 99 | 100 | 98 | 89 |
| Task | Ordered semantic subtasks |
|---|---|
| Adjust Bottle | (1) Pick up the target bottle and hold it upright. |
| Beat Block Hammer | (1) Hit the block with the hammer. |
| Blocks Ranking RGB | (1) Place the red block at the left position; (2) Place the green block at the middle position; (3) Place the blue block at the right position. |
| Blocks Ranking Size | (1) Place the small block at the right position; (2) Place the medium block at the middle position; (3) Place the large block at the left position. |
| Click Alarmclock | (1) Press the alarm clock button. |
| Click Bell | (1) Press the top of the bell. |
| Task | Ordered semantic subtasks |
|---|---|
| Arrange Largest Number | (1) Arrange the digits from left to right to form the largest number. |
| Build Tower | (1) Place the two base blocks of the first tower layer; (2) Place the first-layer beam across the base blocks; (3) Place the two base blocks of the second tower layer; (4) Place the second-layer beam across the second-layer base; (5) Place the cuboid block on the tower; (6) Place the pyramid on top to complete the tower. |
| Classify Objects | (1) Sort objects into the white basket; (2) Sort objects into the blue basket; (3) Sort objects into the red basket. |
| Cover Blocks | (1) Cover the leftmost block; (2) Cover the middle block; (3) Cover the rightmost block; (4) Uncover the red block; (5) Uncover the green block; (6) Uncover the blue block. |
| Deposit Coin | (1) Pick up the coin; (2) Hand the coin to the other hand; (3) Insert the coin into the coin bank. |
| DLC | (1) Arrange the letters to spell “RoboDojo” in a row. |
| Task | Ordered semantic subtasks |
|---|---|
| Domino | (1–4) Place the first, second, third, and fourth dominoes upright in sequence; (5) initiate the toppling sequence. |
| Puzzle | (1) Insert the first piece into its matching recess; (2) insert the second piece; (3) insert the third piece. |
| Organize | (1) Open the drawer for block 1; (2) place block 1 inside; (3) close the drawer; (4) open the drawer for block 2; (5) place block 2 inside; (6) close the drawer; (7) open the drawer for block 3; (8) place block 3 inside; (9) close the drawer. |
Long-Horizon Robotic Manipulation
Long-horizon robotic manipulation studies how robots plan, remember, and recover across extended sequences of physical tasks.
See all 405 papers