Many vision-language-action policies rely on action-labeled robot demonstrations and lack an explicit mechanism for predicting future visual states. These limitations hinder learning from action-free videos and can encourage overfitting to demonstration conditions, weakening generalization and robustness. We present TS-WAM, a lightweight Teacher-Student World-Action Model that addresses these challenges through self-supervised temporal learning and explicit future prediction. Its vlm-jepa framework learns from heterogeneous data, including robot demonstrations and action-free human videos, using an EMA target branch without requiring a separately trained external teacher. By coupling vlm-jepa with delta-lwm, TS-WAM learns to predict latent actions from the current observation and instruction and uses them to predict future visual features that guide action generation at inference. A unified end-to-end training architecture jointly optimizes temporal representation learning, visual dynamics, and robot control without a separate latent-action or world-model pretraining stage. The EMA target branch is removed at inference, keeping the deployed policy lightweight. Experiments on LIBERO, LIBERO-Plus, and SimplerEnv, together with real-world manipulation experiments on Franka Research 3 and RevoTron, demonstrate that TS-WAM achieves strong manipulation performance, generalization, and robustness.
Figures & tables
Fig. 1: Comparison of conventional IDM-based world-action models and our JEPA-based TS-WAM. Pre-training followed by fine-tuning often requires substantial computational resources and time, whereas TS-WAM achieves strong performance with limited resources through end-to-end training without a separate pre-training stage.
Fig. 2: Overview of TS-WAM. Vlm-jepa predicts temporal latent actions, delta-lwm predicts visual feature changes, and the flow-matching action expert generates robot actions conditioned on predicted future features and VLM context. The EMA teacher provides supervision during training.
Fig. 3: Vlm-jepa architecture. The online X -VLM and predictor infer latent actions from the current observation and instruction; the EMA Y -VLM uses the future frame to provide a stop-gradient alignment target.
Table 4
Method
Params
Camera
Robot
Language
Light
Background
Noise
Layout
Overall
WorldVLA [ 27 ]
7B
0.1
27.9
41.6
43.7
17.1
10.9
38.0
25.0
Mantis [ 28 ]
5.8B
15.7
41.8
45.9
45.1
28.9
39.2
62.5
39.8
UniVLA [ 17 ]
7B
4.3
50.3
71.8
59.1
80.0
25.3
34.3
41.5
π0 [ 24 ]
3.3B
13.8
6.0
58.8
85.0
81.4
79.0
68.9
53.6
VLA-JEPA [ 29 ]
2B
40.3
55.7
72.9
88.2
70.5
38.2
74.6
62.9
Spatial Forcing [ 35 ]
7B
20.1
13.4
40.9
29.1
33.4
25.7
39.3
28.8
TABLE III: Zero-shot transfer results on LIBERO-Plus. Success rates (%) are reported across seven perturbation categories and overall.
Fig. 4: Latent-action representation diversity. Each point is a temporal window, projected onto two principal components; colors identify tasks. The upper and lower rows show student and teacher latent actions, respectively.
Fig. 5: Effect of latent-action conditioning on LWM predictions. Rows A–D show four task examples. The first two columns contain the observed temporal pair; the remaining columns visualize the ground-truth feature change and predictions using the normal student LA (TS-WAM), a fixed mean LA, and a zero LA. All feature maps share the same GT-fitted PCA basis and RGB scale.
Method
Franka Research 3
RevoTron
Overall
Stack Bowls
Drawer- Long
Item Sorting
Drawer- Long
π0.5
IND
75
40
60
45
55.00
OOD-LC
60
20
50
35
41.25
OOD-OL
45
20
40
25
32.50
OOD-BT
40
25
30
30
31.25
TABLE IV: Real-world performance comparison under IND and OOD conditions. Success rates (%) are reported over 20 trials per task and condition; Overall is the mean across four tasks. OOD-LC, OOD-OL, and OOD-BT denote OOD-Light Conditions, OOD-Objects Layout, and OOD-Background Textures, respectively.
Fig. 6: LIBERO ablation results. “w/o human video” denotes training without ego data; “w/o Pred delta F” denotes an action expert that receives only semantic features from the VLM, without future features from delta-lwm; “w/o vlm-jepa” denotes training with a single VLM without the JEPA architecture.
Fig. 8: TS-WAM executing Drawer-Long on RevoTron under IND and three OOD conditions. Rows from top to bottom show the original IND scene, OOD-Light conditions (an additional light switched on), OOD-Objects Layout (bread placed on a plate instead of directly on the table), and OOD-Background Textures (the white tabletop covered with a black cloth). Time progresses from left to right. Each sequence shows successful completion of opening the drawer, placing the bread inside, and closing the drawer despite the visual changes.
Robust robot control benefits from explicitly modeling state transitions, but video-generation world action models (WAMs) introduce substantial deployment cost. Existing latent WAMs avoid explicit future generation, but often compress predictive representations or separate predictive modeling from the representations used for action generation. We introduce JEPA-WAM, a latent WAM built in a pretrained V-JEPA space, which couples latent transition prediction with continuous action generation through a shared predictor. JEPA-WAM predicts a spatially structured joint current-future target that captures task-shared visual temporal structure between current and future observations, while preserving dense patch-level correspondence. Through the shared predictor, transition supervision directly shapes the backbone, from which dedicated representations are extracted for action prediction. The same design can also be instantiated in pretrained VLA policies while preserving their original perception and action pathways. On LIBERO-Plus, JEPA-WAM achieves 79.2%, the best result without large-scale robot-policy pretraining, while its pretrained π0.5 instantiation reaches 86.3%, achieving the best overall performance. Experiments on RoboTwin 2.0 and real-world bimanual manipulation further demonstrate strong generalization under visual and spatial shifts.
Yihan Lin, Jiawei He, Shifeng Bao +6
School of Information, Renmin University of China, Beijing, China · XYZ Embodied AI, Beijing, China · Key Laboratory of Data Engineering and Knowledge Engineering, Beijing, China +3
Vision-Language-Action models (VLAs) leverage large-scale vision-language pretraining for semantic robot control, but often lack explicit foresight into how robot actions change the scene. World-Action Models (WAMs) address this limitation by conditioning policies on predicted futures, yet existing approaches typically rely on computationally expensive video generation with substantial pixel-level redundancy. We present LaWAM, a Latent World Action Model that exposes predictive dynamics to robot policies through compact latent visual subgoals instead of reconstructed future video. At the core of LaWAM is a latent-action-conditioned Latent World Model (LaWM). We obtain LaWM by training a latent action model in the latent space of a pretrained vision foundation model and repurposing its forward decoder to predict future observation features for scene evolution. LaWAM then conditions action generation on these predicted latent visual subgoals to enable dynamics-aware robot control. LaWAM achieves state-of-the-art or competitive success rates (SRs) across LIBERO (98.6% SR), RoboTwin (91.22% SR), and real-world manipulation tasks while retaining low-latency inference. LaWAM runs in 187 ms per action-chunk prediction and achieves up to 24x lower wall-clock latency than pixel-space WAMs.
Jialei Chen, Kai Wang, Kang Chen +9
Jilin University · Zhongguancun Academy · Nankai University +4
Vision-language-action (VLA) policies excel at generalist robotic manipulation but generate actions without explicitly modeling their environmental consequences. World action models (WAMs) incorporate future prediction, yet most rely on costly video generation, while latent alternatives either use predicted futures only as auxiliary targets or infer latent actions from suboptimal visual transitions through costly large-scale video pretraining. We introduce WAVE-0 (World-model-Augmented VLA Execution), a framework that bridges VLAs and WAMs through a lightweight latent world model. By connecting the semantic understanding of pretrained VLAs with explicit future prediction, WAVE-0 allows anticipated action consequences to directly guide action generation. Built on a pretrained VLA, its 257M world model translates VLM-generated latent action plans into predicted future representations that condition the action expert, unifying latent planning, future prediction, and action generation. The world model is trained directly on ground-truth robot actions with a counterfactual ranking objective, requiring neither pixel reconstruction nor large-scale video pretraining. WAVE-0 achieves success rates of 91.47% on RoboTwin 2.0 and 98.7% on LIBERO, along with 88.0%, 61.4%, and 75.0% on LIBERO-Plus, LIBERO-Pro, and LIBERO-Para, respectively. On real-world tasks, it improves the success rate of π0.5 from 38.3% to 73.3%. These results demonstrate strong manipulation performance and generalization across diverse distribution shifts.
Haomin Wang, Anqiong Ji, Ke Wang +4
AutoLab, School of Artificial Intelligence, Shanghai Jiao Tong University · Baidu AI Cloud · Awaken Robotics +1