Organizations: The Chinese University of Hong Kong · State Key Laboratory of Multimedia Information Processing, School of Computer Science, Peking University · Simplexity Robotics
World-Action Models couple action learning with future prediction, enabling robotic policies to capture environmental dynamics for robust control. However, prevailing methods take a reconstruction-oriented approach to model future videos, allocating substantial capacity to appearance details that are incidental to control. In this work, we reformulate world modeling for manipulation as control-centric future prediction in latent space, and present LaST1.0, a unified latent World-Action Model. Rather than modeling an observation-complete video future, LaST1.0 predicts compact, geometry-aware latent targets derived from future wrist views, focusing the supervision on the evolving robot-object interactions required for fine-grained manipulation. To better support this control-centric paradigm, LaST1.0 revisits the visual input interface and attention mechanism of its unified architecture. It adopts a vision-encoder-free design with a lightweight patch embedder, exposing fine-grained visual cues directly to the backbone, and introduces LaST-Attention for structured spatiotemporal interactions across modalities. Evaluations across two robot embodiments, five challenging manipulation tasks, and three simulation benchmarks---RLBench, LIBERO, and LIBERO-Plus---demonstrate that LaST1.0 achieves state-of-the-art success rates and generalizes to novel scenarios. These results demonstrate the effectiveness of control-centric latent world modeling for robust manipulation.
Figures & tables
Figure 1: Overview of LaST 1.0 . The top panel illustrates the design roadmap of LaST 1.0 , highlighting the interaction-centric, geometry-aware future-latent supervision, alongside early fusion of visual patches and structured 4D spatiotemporal attention. The bottom panel summarizes the model’s performance across simulation, real-world, generalization, and inference latency.
Figure 2: LaST 1.0 Framework. (a) LaST 1.0 employs an encoder-free unified backbone that takes projected multimodal context to jointly predict flow velocities for latent and action tokens. The latent targets are extracted via VGGT- Ω from future wrist views to enable interaction-centric geometry-aware reasoning. (b) Each decoder block in LaST 1.0 features LaST-Attention, which independently rotates queries and keys across token sequence, image height, image width, and time axes to enhance spatiotemporal fusion. A subsequent Task-Routed MoE effectively expands model capacity. (c) Multimodal Tokens are represented in a 4D space with separate sequence, spatial and temporal coordinates, preserving their inherent structure.
Collection
Trajectories
Hours
AGIBot
153,717
2,534.8
GenRobot
36,849
1,026.9
Simplexity Robotics i7
65,194
313.2
Total
255,760
3,874.9
Table 1: Pre-training data composition. Durations are computed at 10 Hz, with synchronized camera views counted as a single timestep.
Method
Close
Close
Toilet
Sweep to
Close
Phone
Take
Take Frame
Place Wine
Water
Average
Box
Laptop Lid
Seat Down
Dustpan
Fridge
on Base
Umbrella
off Hanger
at Rack
Plants
OpenVLA-OFT [ 3 ]
96.0
70.0
80.0
50.0
84.0
38.0
44.0
20.0
38.0
26.0
54.6
π0.5 [ 1 ]
90.0
96.0
94.0
84.0
100.0
18.0
28.0
86.0
76.0
38.0
71.0
HybridVLA [ 2 ]
92.0
96.0
100.0
90.0
96.0
44.0
58.0
68.0
48.0
44.0
73.6
Fast-WAM [ 11 ]
94.0
90.0
96.0
74.0
100.0
30.0
66.0
56.0
8.0
14.0
62.8
Cosmos Policy [ 9 ]
94.0
94.0
100.0
60.0
76.0
78.0
80.0
70.0
54.0
58.0
76.4
Table 2: Results on RLBench. All methods are trained in the multi-task setting, and we report average success rates (%).
Models
Spatial
Object
Goal
Long
Average
OpenVLA-OFT [ 3 ]
97.6
98.4
97.9
94.5
97.1
π0.5 [ 1 ]
98.8
98.2
98.0
92.4
96.9
StarVLA- α [ 21 ]
99.0
99.8
98.5
94.1
97.9
Fast-WAM [ 11 ]
98.2
100.0
97.0
95.2
97.6
Cosmos Policy [ 9 ]
98.1
100.0
98.2
97.6
98.5
LingBot-VA [ 35 ]
98.5
99.6
97.2
98.5
98.5
Table 3: Results on LIBERO. We perform three test runs and report the average success rate (%).
Models
Camera
Robot
Language
Light
Background
Noise
Layout
Average
OpenVLA-OFT [ 3 ]
56.4
31.9
79.5
88.7
93.3
75.8
74.2
69.6
π0.5 [ 1 ]
78.4
73.6
80.8
96.2
94.1
89.0
84.5
84.4
StarVLA- α [ 21 ]
48.7
63.4
86.8
95.8
94.6
75.0
80.2
76.1
Fast-WAM [ 11 ]
16.4
44.5
68.9
78.2
53.7
37.7
60.7
50.0
Cosmos Policy [ 9 ]
75.8
63.3
81.7
96.5
88.9
92.7
82.2
82.2
LingBot-VA [ 35 ]
40.9
83.0
86.4
82.3
53.1
64.4
76.2
69.5
Table 4: Results on LIBERO-Plus. We perform three test runs and report the average success rate (%).
Figure 3: Ablation studies of LaST 1.0 on RLBench. (a–c) Ablations on the latent world modeling design, including future latent target, observation views, and the branch-decoupled mask. (d–f) Ablations on the architecture design, comparing the encoder-free interface, LaST-Attention with 4D-RoPE, and the Task-Routed MoE. We report the average success rate (%).
VGGT- Ω -Register
VGGT- Ω -Full
Uni3D-g
Wan2.2 VAE
DINOv3
Error (cm) ↓
0.23
0.29
0.77
1.88
2.69
Table 5: Future latent probing results on RLBench.
Value
Latent Loss Weight λlat
Future Frames F
0
0.1
0.5
1.0
1
2
4
8
Success rates (%)
81.8
92.0
89.2
87.8
88.2
89.8
92.0
90.4
Table 6: Hyperparameter Ablations on RLBench. We ablate the latent loss weight λlat and the sampled frames F .
Figure 4: Action-to-Vision Attention Maps. Qualitative examples for the encoder-free, ViT, and VAE variants. In these examples, the encoder-free variant shows localized attention around task-relevant objects and interaction regions.
Method
Franka + Gripper
Tianji + Gripper
Tianji + Hand
Overall
Put Object in Bag
Put Object in Drawer
Parcel-Box Packing
Assemble Truck Toy
Stack Cups
Stage Average
Task Average
Inference Time (ms)
S1
S2
S1
S2
S3
S1
S2
S3
S1
S2
S3
S4
S1
S2
π0.5 [ 1 ]
75.0
50.0
90.0
70.0
65.0
90.0
70.0
20.0
85.0
10.0
5.0
5.0
60.0
40.0
52.5
36.0
60.4
Cosmos Policy [ 9 ]
80.0
55.0
85.0
45.0
35.0
70.0
60.0
10.0
70.0
5.0
0.0
0.0
60.0
50.0
44.6
30.0
229.0
LaST 0 [ 4 ]
80.0
60.0
90.0
60.0
60.0
75.0
70.0
15.0
75.0
20.0
15.0
5.0
65.0
35.0
51.8
35.0
138.3
LaST 1.0 (Ours)
95.0
70.0
90.0
80.0
75.0
90.0
75.0
45.0
85.0
45.0
35.0
30.0
75.0
50.0
67.1
54.0
54.7
Table 7: Results on Real-World Tasks. We decompose each task into subtasks and record the success rates (%). Inference time (ms) is measured per model inference on an NVIDIA RTX 4090 GPU under identical settings.
Figure 5: Visualization of LaST 1.0 performing diverse tasks across different robotic embodiments in real-world executions.
Model
Original
Object
Background
Position
π0.5 [ 1 ]
50.0
35.0 ( ↓ 30%)
50.0 ( ↓ 0%)
20.0 ( ↓ 60%)
Cosmos Policy [ 9 ]
55.0
20.0 ( ↓ 64%)
35.0 ( ↓ 36%)
10.0 ( ↓ 82%)
LaST 0 [ 4 ]
60.0
45.0 ( ↓ 25%)
45.0 ( ↓ 25%)
35.0 ( ↓ 42%)
LaST 1.0 (Ours)
70.0
55.0 ( ↓ 21%)
70.0 ( ↓ 0%)
50.0 ( ↓ 29%)
Table 8: Generalization Analysis. Task-level success rates (%) for completing both stages of “Put Object in Bag”, evaluated over 20 rollouts per model per setting. Parenthesized values indicate relative decreases from the original-setting success rate.
State Key Laboratory of Multimedia Information Processing, School of Computer Science, Peking University · The Chinese University of Hong Kong · Simplexity Robotics