Organizations: The Chinese University of Hong Kong · State Key Laboratory of Multimedia Information Processing, School of Computer Science, Peking University · Simplexity Robotics
World-Action Models couple action learning with future prediction, enabling robotic policies to capture environmental dynamics for robust control. However, prevailing methods take a reconstruction-oriented approach to model future videos, allocating substantial capacity to appearance details that are incidental to control. In this work, we reformulate world modeling for manipulation as control-centric future prediction in latent space, and present LaST1.0, a unified latent World-Action Model. Rather than modeling an observation-complete video future, LaST1.0 predicts compact, geometry-aware latent targets derived from future wrist views, focusing the supervision on the evolving robot-object interactions required for fine-grained manipulation. To better support this control-centric paradigm, LaST1.0 revisits the visual input interface and attention mechanism of its unified architecture. It adopts a vision-encoder-free design with a lightweight patch embedder, exposing fine-grained visual cues directly to the backbone, and introduces LaST-Attention for structured spatiotemporal interactions across modalities. Evaluations across two robot embodiments, five challenging manipulation tasks, and three simulation benchmarks---RLBench, LIBERO, and LIBERO-Plus---demonstrate that LaST1.0 achieves state-of-the-art success rates and generalizes to novel scenarios. These results demonstrate the effectiveness of control-centric latent world modeling for robust manipulation.
Figures & tables
Figure 1: Overview of LaST 1.0 . The top panel illustrates the design roadmap of LaST 1.0 , highlighting the interaction-centric, geometry-aware future-latent supervision, alongside early fusion of visual patches and structured 4D spatiotemporal attention. The bottom panel summarizes the model’s performance across simulation, real-world, generalization, and inference latency.
Figure 2: LaST 1.0 Framework. (a) LaST 1.0 employs an encoder-free unified backbone that takes projected multimodal context to jointly predict flow velocities for latent and action tokens. The latent targets are extracted via VGGT- Ω from future wrist views to enable interaction-centric geometry-aware reasoning. (b) Each decoder block in LaST 1.0 features LaST-Attention, which independently rotates queries and keys across token sequence, image height, image width, and time axes to enhance spatiotemporal fusion. A subsequent Task-Routed MoE effectively expands model capacity. (c) Multimodal Tokens are represented in a 4D space with separate sequence, spatial and temporal coordinates, preserving their inherent structure.
Collection
Trajectories
Hours
AGIBot
153,717
2,534.8
GenRobot
36,849
1,026.9
Simplexity Robotics i7
65,194
313.2
Total
255,760
3,874.9
Table 1: Pre-training data composition. Durations are computed at 10 Hz, with synchronized camera views counted as a single timestep.
Method
Close
Close
Toilet
Sweep to
Close
Phone
Take
Take Frame
Place Wine
Water
Average
Box
Laptop Lid
Seat Down
Dustpan
Fridge
on Base
Umbrella
off Hanger
at Rack
Plants
OpenVLA-OFT [ 3 ]
96.0
70.0
80.0
50.0
84.0
38.0
44.0
20.0
38.0
26.0
54.6
π0.5 [ 1 ]
90.0
96.0
94.0
84.0
100.0
18.0
28.0
86.0
76.0
38.0
71.0
HybridVLA [ 2 ]
92.0
96.0
100.0
90.0
96.0
44.0
58.0
68.0
48.0
44.0
73.6
Fast-WAM [ 11 ]
94.0
90.0
96.0
74.0
100.0
30.0
66.0
56.0
8.0
14.0
62.8
Cosmos Policy [ 9 ]
94.0
94.0
100.0
60.0
76.0
78.0
80.0
70.0
54.0
58.0
76.4
Table 2: Results on RLBench. All methods are trained in the multi-task setting, and we report average success rates (%).
Models
Spatial
Object
Goal
Long
Average
OpenVLA-OFT [ 3 ]
97.6
98.4
97.9
94.5
97.1
π0.5 [ 1 ]
98.8
98.2
98.0
92.4
96.9
StarVLA- α [ 21 ]
99.0
99.8
98.5
94.1
97.9
Fast-WAM [ 11 ]
98.2
100.0
97.0
95.2
97.6
Cosmos Policy [ 9 ]
98.1
100.0
98.2
97.6
98.5
LingBot-VA [ 35 ]
98.5
99.6
97.2
98.5
98.5
Table 3: Results on LIBERO. We perform three test runs and report the average success rate (%).
Models
Camera
Robot
Language
Light
Background
Noise
Layout
Average
OpenVLA-OFT [ 3 ]
56.4
31.9
79.5
88.7
93.3
75.8
74.2
69.6
π0.5 [ 1 ]
78.4
73.6
80.8
96.2
94.1
89.0
84.5
84.4
StarVLA- α [ 21 ]
48.7
63.4
86.8
95.8
94.6
75.0
80.2
76.1
Fast-WAM [ 11 ]
16.4
44.5
68.9
78.2
53.7
37.7
60.7
50.0
Cosmos Policy [ 9 ]
75.8
63.3
81.7
96.5
88.9
92.7
82.2
82.2
LingBot-VA [ 35 ]
40.9
83.0
86.4
82.3
53.1
64.4
76.2
69.5
Table 4: Results on LIBERO-Plus. We perform three test runs and report the average success rate (%).
Figure 3: Ablation studies of LaST 1.0 on RLBench. (a–c) Ablations on the latent world modeling design, including future latent target, observation views, and the branch-decoupled mask. (d–f) Ablations on the architecture design, comparing the encoder-free interface, LaST-Attention with 4D-RoPE, and the Task-Routed MoE. We report the average success rate (%).
VGGT- Ω -Register
VGGT- Ω -Full
Uni3D-g
Wan2.2 VAE
DINOv3
Error (cm) ↓
0.23
0.29
0.77
1.88
2.69
Table 5: Future latent probing results on RLBench.
Value
Latent Loss Weight λlat
Future Frames F
0
0.1
0.5
1.0
1
2
4
8
Success rates (%)
81.8
92.0
89.2
87.8
88.2
89.8
92.0
90.4
Table 6: Hyperparameter Ablations on RLBench. We ablate the latent loss weight λlat and the sampled frames F .
Figure 4: Action-to-Vision Attention Maps. Qualitative examples for the encoder-free, ViT, and VAE variants. In these examples, the encoder-free variant shows localized attention around task-relevant objects and interaction regions.
Method
Franka + Gripper
Tianji + Gripper
Tianji + Hand
Overall
Put Object in Bag
Put Object in Drawer
Parcel-Box Packing
Assemble Truck Toy
Stack Cups
Stage Average
Task Average
Inference Time (ms)
S1
S2
S1
S2
S3
S1
S2
S3
S1
S2
S3
S4
S1
S2
π0.5 [ 1 ]
75.0
50.0
90.0
70.0
65.0
90.0
70.0
20.0
85.0
10.0
5.0
5.0
60.0
40.0
52.5
36.0
60.4
Cosmos Policy [ 9 ]
80.0
55.0
85.0
45.0
35.0
70.0
60.0
10.0
70.0
5.0
0.0
0.0
60.0
50.0
44.6
30.0
229.0
LaST 0 [ 4 ]
80.0
60.0
90.0
60.0
60.0
75.0
70.0
15.0
75.0
20.0
15.0
5.0
65.0
35.0
51.8
35.0
138.3
LaST 1.0 (Ours)
95.0
70.0
90.0
80.0
75.0
90.0
75.0
45.0
85.0
45.0
35.0
30.0
75.0
50.0
67.1
54.0
54.7
Table 7: Results on Real-World Tasks. We decompose each task into subtasks and record the success rates (%). Inference time (ms) is measured per model inference on an NVIDIA RTX 4090 GPU under identical settings.
Figure 5: Visualization of LaST 1.0 performing diverse tasks across different robotic embodiments in real-world executions.
Model
Original
Object
Background
Position
π0.5 [ 1 ]
50.0
35.0 ( ↓ 30%)
50.0 ( ↓ 0%)
20.0 ( ↓ 60%)
Cosmos Policy [ 9 ]
55.0
20.0 ( ↓ 64%)
35.0 ( ↓ 36%)
10.0 ( ↓ 82%)
LaST 0 [ 4 ]
60.0
45.0 ( ↓ 25%)
45.0 ( ↓ 25%)
35.0 ( ↓ 42%)
LaST 1.0 (Ours)
70.0
55.0 ( ↓ 21%)
70.0 ( ↓ 0%)
50.0 ( ↓ 29%)
Table 8: Generalization Analysis. Task-level success rates (%) for completing both stages of “Put Object in Bag”, evaluated over 20 rollouts per model per setting. Parenthesized values indicate relative decreases from the original-setting success rate.
World Action Models (WAMs) have recently emerged as a promising paradigm for robotic manipulation, endowing policies with strong capabilities for modeling temporal and physical dynamics. Existing WAMs, encompassing both video-first action prediction and joint video-action modeling, directly transfer global physical representations from world models to action generation. However, global scene representations may underemphasize local robot-object interactions, creating a mismatch between representations optimized for world evolution and those required for manipulation. To bridge this gap, we propose LaST0.5, a novel WAM that introduces an interaction-centric physical representation between world modeling and action generation. Specifically, LaST0.5 learns a compact interaction-centric latent space under the supervision of visual interaction cues between the robot and the environment. This reasoning space is instantiated as a latent expert that bridges the world expert and the action expert within a Mixture-of-Transformers architecture. Furthermore, LaST0.5 performs adaptive world-to-action transfer by jointly predicting future representations and progress values, using progress consistency to adjust action execution and trigger future resampling. Finally, we pretrain LaST0.5 on a large-scale robotic data corpus comprising over 14k hours of data, establishing interaction-centric physical reasoning as a scalable and effective paradigm. LaST0.5 achieves 99.5% on LIBERO, surpasses the previous state of the art on ten evaluated RLBench tasks, and demonstrates robust long-horizon execution and precise dexterous manipulation in real-world experiments.
Xuheng Zhang, Yiming Zhang, Jialin Gao +14
State Key Laboratory of Multimedia Information Processing, School of Computer Science, Peking University · The Chinese University of Hong Kong · Simplexity Robotics
World-action modeling has emerged as a promising paradigm for robotic control, as it empowers models to go beyond reacting to observations and anticipate how a scene will evolve. However, existing WAMs often incur substantial computational overhead. Pixel-space methods often allocate substantial capacity to visual details that may not be directly relevant to control, while some latent-space methods require multi-stage training to construct the reasoning space. The resulting training cost can make such methods difficult to train under modest computational budgets. In this work, we propose LiLa-WAM, a lightweight world-action model that reasons about the future in a compact latent space and can be trained end-to-end on a single 24GB GPU. Its core design is a compact latent reasoning space jointly shaped by future-state prediction and action generation, which keeps the model lightweight while remaining well aligned with control. For task specification, we further propose the Visual Transition Token(VTT), a language-free task representation that encodes each task as a direction in visual feature space. Experiments on RoboTwin~2.0, LIBERO, and real-robot tasks demonstrate LiLa-WAM's effectiveness, achieving 90.48% success across 50 RoboTwin tasks with single-GPU training.
Fan Yang, Yuting Su, Xiaobo Wang +7
1Tianjin University · 2Shenzhen University of Advanced Technology · 5Sangfor Technologies Inc. +2
World-Action Models (WAMs) build robot control on video-generation backbones, which jointly predict dense future visual trajectories and robot actions. We argue that video generation is an unnecessary intermediate objective for world-action modeling. For robotic manipulation, the goal of a world model is not to reproduce how the world looks at every intermediate moment, but to predict the state that the world will reach after an action is executed. The intermediate frames only describe the visual transition between physical states, which consumes substantial model capacity and computation, but do not directly specify the physical outcome that the robot action is intended to produce. In this paper, we propose DELE-w0.5, which infers robot actions from predicted future states without relying on video generation. Concretely, DELE-w0.5 infers the action sequence from its corresponding compact future latent state. The future latent state captures the action-relevant physical outcome of robot interaction and serves as an explicit bridge between world modeling and action generation. The core design principle of DELE-w0.5 is to model how the physical world changes under robot actions, rather than how its visual appearance evolves frame by frame. This formulation removes the high-dimensional visual redundancy introduced by dense video representations, and it therefore enables cheaper training and low-latency inference. Across 640 real-robot trials on four long-horizon manipulation tasks, our DELE-w0.5 achieves the best performance among all compared policies, attaining 62.5% overall full-task success and 81.3% macro ordered-stage progress. It outperforms the strongest baseline by 32.5 percentage points in full-task success and 20.1 percentage points in macro progress.