LAW: Unifying Language, Action, and World Modeling for Agentic Physical Intelligence

Jiajun Cao; Kai Tang; Bohua Zhang; Zezhong Qian; Ying Li; Xiaozhu Ju; Xiaobao Wei; Jiaming Liu

cs.ROOct 16, 2026

LAW: Unifying Language, Action, and World Modeling for Agentic Physical Intelligence

Authors: Jiajun Cao, Kai Tang, Bohua Zhang, Zezhong Qian, Ying Li, Xiaozhu Ju, Xiaobao Wei, Jiaming Liu,

Organizations: State Key Laboratory of Multimedia Information Processing, School of Computer Science, Peking University · Beijing Innovation Center of Humanoid Robotics

Abstract

World-action models jointly predict actions and future world states, but long-horizon manipulation further requires language reasoning to plan subtask objectives, assess execution progress, and guide action selection. We present LAW, a closed-loop VLM--WAM framework that unifies language reasoning, action generation, and world modeling through semantic subtask progress. Within the LAW agent, a hybrid-expert vision-language model plans the remaining subtasks, estimates the values of and fuses actions proposed by a world-action model, and asynchronously assesses execution status to support continued execution and subtask transitions. Within the world-action model, Mixture-of-Transformer Experts integrates modality-specific and shared computation for joint video--action modeling. Dual-stream hierarchical visual memory maintains temporal context for both models, while subtask-local value learning aligns action evaluation with semantic objectives and preserves supervision from successful prefixes of failed trajectories. LAW achieves 94.21% average success across 50 RoboTwin tasks. Replacing the WAM backbone with OpenWAM- $alpha$ further raises the average success rate to 95.56%, establishing a new open-source state of the art on RoboTwin. Despite having only 2B parameters, our VLM also outperforms variants using Embodied-R1.5 and GPT-5.5 High. On three long-horizon real-robot tasks, LAW achieves 65% average success, compared with 28.33% for $pi_{0.5}$ and 25% for Fast-WAM.

Figures & tables

Figure 2: LAW agent’s architecture and loop.The hybrid-expert VLM uses planner, state, and value experts on a shared frozen backbone, while the MoTE-WAM is co-trained with video and action objectives and generates candidate action chunks at inference. Semantic subtask progress connects execution monitoring, subtask advancement, candidate ranking and fusion, action execution, and environment feedback in a closed loop.

Figure 3: Visualization of semantic subtask annotations, temporal boundaries, and failure cases in RoboTwin.

Figure 4: Mixed-frequency coordination for progress-aware control in LAW. The planner operates at semantic subtask boundaries, while WAM generation and value-based selection synchronously determine each action chunk. The state expert asynchronously estimates subtask completion during action execution and triggers the next planning decision when the current subtask is completed.

Figure 5: MoTE architecture for joint video–action modeling. The video and action DiT branches exchange information through MoT mixed self-attention. After mixed attention, modality-specific top-1 routers select private experts or a shared expert available to both streams. The routed expert output is scaled by $\alpha$ and added residually before the remaining Transformer transformations.

Figure 6: Hierarchical visual memory and attention pattern in the WAM. Short-term and subtask-boundary observations are retained at full resolution, while older observations are spatially compressed.

Figure 7: Real-world manipulation results. Representative executions illustrate the semantic subtasks of Puzzle (three), Domino (five), and Organize (nine). (a) Complete-task success rate. (b) Task score, computed from the fraction of completed subtasks and scaled to 0–100. LAW is compared with $\pi_{0.5}$ and Fast-WAM.

Figure 8: Cumulative ablation of subtask-completion prediction from left to right. Each row-normalized matrix adds one component to the preceding setting; cells report true negatives (TN), false positives (FP), false negatives (FN), and true positives (TP), with sample counts in parentheses.

Figure 9: Action-loss curves for different MoE configurations.

Appendix figures & tables9 assets

Supplementary material from the paper’s appendix.

Appendix

Task	LAW + OpenWAM- $\alpha$		LAW + Fast-WAM		Fast-WAM		LingBot-VA		$\bm{\pi}_{0.5}$
	Clean (%)	Rand. (%)	Clean (%)	Rand. (%)	Clean (%)	Rand. (%)	Clean (%)	Rand. (%)	Clean (%)	Rand. (%)
Adjust Bottle	100	100	100	100	100	100	90	94	100	99
Beat Block Hammer	97	100	96	99	99	97	96	98	96	93
Blocks Ranking RGB	100	99	95	93	100	100	99	98	92	85
Blocks Ranking Size	91	91	79	75	94	98	94	96	49	26
Click Alarmclock	99	99	100	99	100	100	99	100	98	89

Appendix

Table 12: Per-task results on the 50-task RoboTwin benchmark.

Figure 10: AgileX Cobot Magic platform. Two follower arms perform bimanual manipulation, while two leader arms support demonstration collection. The observation configuration comprises one scene camera and two wrist cameras. Hardware reference image from AgileX Robotics .

Task	Ordered semantic subtasks
Adjust Bottle	(1) Pick up the target bottle and hold it upright.
Beat Block Hammer	(1) Hit the block with the hammer.
Blocks Ranking RGB	(1) Place the red block at the left position; (2) Place the green block at the middle position; (3) Place the blue block at the right position.
Blocks Ranking Size	(1) Place the small block at the right position; (2) Place the medium block at the middle position; (3) Place the large block at the left position.
Click Alarmclock	(1) Press the alarm clock button.
Click Bell	(1) Press the top of the bell.

Appendix

Table 13: Semantic subtask sequences for 50 RoboTwin tasks.

Task	Ordered semantic subtasks
Arrange Largest Number	(1) Arrange the digits from left to right to form the largest number.
Build Tower	(1) Place the two base blocks of the first tower layer; (2) Place the first-layer beam across the base blocks; (3) Place the two base blocks of the second tower layer; (4) Place the second-layer beam across the second-layer base; (5) Place the cuboid block on the tower; (6) Place the pyramid on top to complete the tower.
Classify Objects	(1) Sort objects into the white basket; (2) Sort objects into the blue basket; (3) Sort objects into the red basket.
Cover Blocks	(1) Cover the leftmost block; (2) Cover the middle block; (3) Cover the rightmost block; (4) Uncover the red block; (5) Uncover the green block; (6) Uncover the blue block.
Deposit Coin	(1) Pick up the coin; (2) Hand the coin to the other hand; (3) Insert the coin into the coin bank.
DLC	(1) Arrange the letters to spell “RoboDojo” in a row.

Appendix

Table 14: Semantic subtask sequences for 35 RoboDojo tasks.

Task	Ordered semantic subtasks
Domino	(1–4) Place the first, second, third, and fourth dominoes upright in sequence; (5) initiate the toppling sequence.
Puzzle	(1) Insert the first piece into its matching recess; (2) insert the second piece; (3) insert the third piece.
Organize	(1) Open the drawer for block 1; (2) place block 1 inside; (3) close the drawer; (4) open the drawer for block 2; (5) place block 2 inside; (6) close the drawer; (7) open the drawer for block 3; (8) place block 3 inside; (9) close the drawer.

Appendix

Table 15: Semantic subtask sequences for real-world tasks.

Figure 11: Semantic subtask segmentation on RoboTwin. From top to bottom: Handover Block, Put Bottles Dustbin, Stack Bowls Three, and Blocks Ranking RGB. The examples distinguish acquisition, inter-arm transfer, and placement, as well as successive placements involving different objects.

Figure 12: Additional RoboTwin examples. From top to bottom: Blocks Ranking Size, Handover Mic, Place Dual Shoes, and Shake Bottle. The stage boundaries separate size-dependent placements, an inter-arm transfer, left- and right-shoe placements, and bottle acquisition from shaking.

Figure 13: Semantic subtask segmentation on RoboDojo. From top to bottom: Cover Blocks, Swap Blocks, Classify Objects, and Stack Blocks. Covering and uncovering are represented as distinct stages; in Swap Blocks, each confirmation stage corresponds to a button press following a transfer. Sorting is grouped by the destination basket.

Figure 14: Additional RoboDojo examples. From top to bottom: Sweep Blocks, Organize Table, Play Stacking Toy, and Fasten Screws. The decompositions preserve tool acquisition and transfer, object-specific placement objectives, and repeated fastening operations. In Fasten Screws, insertion and five tightening turns form one subtask per screw.

Long-Horizon Robotic Manipulation

Long-horizon robotic manipulation studies how robots plan, remember, and recover across extended sequences of physical tasks.

See all 405 papers

World-Language-Action Model for Unified World Modeling, Language Reasoning, and Action Synthesis
Jun 4, 2026Yi Yang, Zhihong Liu, Siqi Kou +9World Models for Robotics Robot Foundation Models
SJTU · SII · HUST +4
World-Value-Action Model: Implicit Planning for Vision-Language-Action Systems
Apr 16, 2026Runze Li, Hongyin Zhang, Junxi Jin +5Vision-Language-Action Models Model-Based Planning
Westlake University Hangzhou, China · Nanjing University Suzhou Campus, 1520 Taihu Avenue, Suzhou, China
WAVE-0: Bridging VLA Policies and WAMs with Lightweight Latent Future Prediction
Oct 13, 2026Haomin Wang, Anqiong Ji, Ke Wang +4World Models for Robotics World Action Models
AutoLab, School of Artificial Intelligence, Shanghai Jiao Tong University · Baidu AI Cloud · Awaken Robotics +1

Jun 4, 2026J/K move · Enter open · S save

World-Language-Action Model for Unified World Modeling, Language Reasoning, and Action Synthesis

Yi Yang, Zhihong Liu, Siqi Kou, Yiyang Chen+8

SJTU · SII · HUST+4

World Models for Robotics Robot Foundation Models Vision-Language-Action Models World Models for VLA Models

Open paper AskPDF arXiv

We propose world-language-action (WLA) models as a new class of embodied foundation models. WLA takes textual instructions, images, and robot states as inputs to jointly predict textual subtasks, subgoal images, and robot actions, conjoining the world modeling interface to learn from extensive egocentric videos as in the world-action model (WAM) and the language reasoning capacities to solve complex long-horizon tasks as in vision-language-action (VLA) models. At the core of WLA lies an autoregressive (AR) Transformer backbone, instead of a bidirectional diffusion Transformer as in WAMs, to predict the next state, comprising the semantic-level textual intention and complementary fine-grained physical dynamics. The physical dynamics are supervised by the world modeling objective based on a dedicated World Expert, and are leveraged to ease the characterization of the state-action correlation for the Action Expert. WLA leverages meta-queries to make the world prediction implicitly impact the action generation so that the former can be disabled during inference. The world prediction can also be activated to enable test-time scaling for improved robot control. Our WLA-0 prototype, with 2B active parameters, achieves 40 ms per inference on an NVIDIA RTX 5090. Evaluations across simulated and real-world environments demonstrate that WLA-0 achieves state-of-the-art multi-task and long-horizon learning abilities, e.g., 92.94% success rate on RoboTwin2.0 Clean and 56.5% success rate on RMBench. WLA-0 also holds the promise to learn novel tasks directly from cross-embodiment robot videos without action annotations.

Abstract

Long-Horizon Robotic Manipulation

Explore similar work

World-Language-Action Model for Unified World Modeling, Language Reasoning, and Action Synthesis

World-Value-Action Model: Implicit Planning for Vision-Language-Action Systems

WAVE-0: Bridging VLA Policies and WAMs with Lightweight Latent Future Prediction