World-action models (WAMs) incorporate the prediction of future observations into robot policy learning. However, most WAMs build on billion-parameter vision-language or video-generation backbones, which incur high training cost and inference latency. In this letter, we present GAWM (Geometry-Aware World Model), a compact multi-task WAM that predicts future observations in the latent space of a frozen DINOv3 encoder. GAWM compresses the features of each camera view into a small set of tokens. A lightweight predictor, conditioned on the task rather than on actions, forecasts these tokens at two future horizons. The forecast is supervised with the dense patch features of future frames, which requires it to encode the spatial layout of the future scene. An action decoder directly regresses a chunk of future actions from the current and predicted tokens. With fewer than 100M parameters and no robot or video pretraining, GAWM outperforms billion-parameter WAMs on RoboTwin 2.0 and performs comparably to billion-parameter VLAs on LIBERO. On a dual-arm robot, it achieves lower action-prediction error on held-out demonstrations than a fine-tuned billion-parameter VLA, with substantially lower inference latency. Ablation studies show that the latent world model substantially improves performance when training data or training time is limited. Code is available at https://github.com/gaoxiang12/ga-wm.
Figures & tables
Fig. 1: Success rate versus model size on LIBERO (left; GAWM-S with additional in-domain data) and on the clean (middle) and randomized (right) settings of RoboTwin 2.0. Top: success rate versus size. Bottom: methods ranked by success rate (SR) with size (as in Table I , or as reported; N/R: not reported). Baselines (references in Table I and [ 15 , 16 , 17 , 18 , 19 , 20 , 21 , 22 , 23 , 24 , 25 , 26 , 27 , 28 ] ): blue, as reported by TurboVLA [ 29 ] ; gray, other sources. On RoboTwin 2.0, GAWM-S is the smallest and best-performing of the plotted methods.
Fig. 2: Overview of GAWM. A frozen DINOv3 encoder and a query adapter encode each camera view into 64 visual tokens, stacked into zt . The latent predictor, conditioned on the task code gk , forecasts these tokens at two future horizons, and the action decoder directly regresses a chunk of future actions from the current tokens, the forecast, the robot state and the task code. Dashed panel: training only; a patch decoder maps the forecast to the encoder’s patch features of the recorded future frames (thumbnails as in Fig. 4 ).
LIBERO
RoboTwin 2.0
Method
Params
Robot PT
Task input
Spatial
Object
Goal
Long
Avg.
Clean
Rand.
Vision-language-action models
OpenVLA [ 2 ]
7.5B ∗
✓
Lang.
84.7
88.4
79.2
53.7
76.5
–
–
π0 [ 3 ]
3.2B ∗
✓
Lang.
96.8
98.8
95.8
85.2
94.2
–
–
π0.5 [ 4 ]
3.4B
✓
Lang.
98.8
98.2
98.0
92.4
96.9
82.74 a
76.76 a
OpenVLA-OFT [ 43 ]
7.6B
✓
Lang.
97.7
98.0
96.1
95.3
96.8
–
–
TABLE I: Success Rates (%) on LIBERO and RoboTwin 2.0
LIBERO, 41 data
LIBERO
RoboTwin
Spatial
Object
Goal
Long
Avg.
Extended
191k steps
GAWM-S
90.2
85.4
87.8
58.4
80.5
95.6
90.7
w/o world model
77.2
89.0
80.2
37.4
71.0
93.1
18.8
Δ
+ 13.0
− 3.6
+ 7.6
+ 21.0
+ 9.5
+ 2.5
+ 71.9
95% CI
[5.2, 14.2]
[ − 0.8, 6.1]
[65.2, 78.0]
TABLE II: Ablation of the World Model
Fig. 3: Inference latency per action-chunk query. Official implementations on the same GPU model (NVIDIA RTX PRO 5000) with batch size 1 and default precision and denoising steps. Latency covers the modules counted in Table I , from preprocessed inputs to the action chunk (median of 200 queries). † : LIBERO configuration; all others use RoboTwin 2.0. Parentheses: parameters as in Table I . Baselines are randomly initialized, which does not change the computation.
Fig. 4: Decoded forecasts. GAWM-S on RoboTwin 2.0, head camera. Left: frames t , t+16 and t+32 . Right: first three principal components of the encoder patch features of the true future frames (True) and of the decoded forecast (Pred.), projected on a PCA basis shared within each example. Labels: mean per-patch cosine similarity to the true features for the forecast and for frame t . Examples: first clean demonstrations, at the frame with the largest 32-frame feature change within 20–80% of the episode.
Fig. 5: Attention of GAWM-S on RoboTwin 2.0. Top: head-camera frame t of the examples of Fig. 4 and the adapter cross-attention over its patches (mean over queries, heads and layers; brighter is higher). Bottom: share of the action decoder’s cross-attention on each memory group (current tokens, near and far forecasts, state tokens) per action step (mean over heads, layers and 1,000 anchors; bands: interquartile range).
Fig. 6: Real-robot task. Head-camera frames of a demonstration of the green-bottle task. The four tasks differ only in the target bottle.
Vision-Language-Action models (VLAs) leverage large-scale vision-language pretraining for semantic robot control, but often lack explicit foresight into how robot actions change the scene. World-Action Models (WAMs) address this limitation by conditioning policies on predicted futures, yet existing approaches typically rely on computationally expensive video generation with substantial pixel-level redundancy. We present LaWAM, a Latent World Action Model that exposes predictive dynamics to robot policies through compact latent visual subgoals instead of reconstructed future video. At the core of LaWAM is a latent-action-conditioned Latent World Model (LaWM). We obtain LaWM by training a latent action model in the latent space of a pretrained vision foundation model and repurposing its forward decoder to predict future observation features for scene evolution. LaWAM then conditions action generation on these predicted latent visual subgoals to enable dynamics-aware robot control. LaWAM achieves state-of-the-art or competitive success rates (SRs) across LIBERO (98.6% SR), RoboTwin (91.22% SR), and real-world manipulation tasks while retaining low-latency inference. LaWAM runs in 187 ms per action-chunk prediction and achieves up to 24x lower wall-clock latency than pixel-space WAMs.
Jialei Chen, Kai Wang, Kang Chen +9
Jilin University · Zhongguancun Academy · Nankai University +4
World-action modeling has emerged as a promising paradigm for robotic control, as it empowers models to go beyond reacting to observations and anticipate how a scene will evolve. However, existing WAMs often incur substantial computational overhead. Pixel-space methods often allocate substantial capacity to visual details that may not be directly relevant to control, while some latent-space methods require multi-stage training to construct the reasoning space. The resulting training cost can make such methods difficult to train under modest computational budgets. In this work, we propose LiLa-WAM, a lightweight world-action model that reasons about the future in a compact latent space and can be trained end-to-end on a single 24GB GPU. Its core design is a compact latent reasoning space jointly shaped by future-state prediction and action generation, which keeps the model lightweight while remaining well aligned with control. For task specification, we further propose the Visual Transition Token(VTT), a language-free task representation that encodes each task as a direction in visual feature space. Experiments on RoboTwin~2.0, LIBERO, and real-robot tasks demonstrate LiLa-WAM's effectiveness, achieving 90.48% success across 50 RoboTwin tasks with single-GPU training.
Fan Yang, Yuting Su, Xiaobo Wang +7
1Tianjin University · 2Shenzhen University of Advanced Technology · 5Sangfor Technologies Inc. +2
World modeling has emerged as an effective co-training objective for robot policies, giving rise to World Action Models (WAMs) that jointly predict actions and future states. However, most WAMs predict future states in the native representation space of pretrained visual backbones, resulting in high-dimensional targets with substantial training cost. We introduce MiniWAM, which instead predicts compact future representations learned from privileged current-future transitions. To construct these targets, we propose Predictive Representations via Inverse Spatiotemporal Modeling (PRISM), which combines inverse-dynamics supervision with feature reconstruction to emphasize control-relevant transition information while preserving useful future-state information. With the learned PRISM encoder frozen, MiniWAM is trained to jointly predict the resulting targets and robot actions from current observations. With 65× fewer native future feature tokens, MiniWAM consistently outperforms native future-feature prediction with both DINOv3 and WAN2.1 VAE features, while achieving up to an 8× speedup in world-action training. At 0.25B parameters, MiniWAM is already competitive with substantially larger WAMs on LIBERO, LIBERO-Plus, and RoboTwin 2.0 simulation benchmarks. Representation analyses further show that PRISM contributes behavioral structure beyond feature reconstruction alone. These results demonstrate that effective world-action modeling does not require predicting native visual futures, and that compact predictive representations provide a strong and substantially more efficient target for policy learning. The project page is available at: https://j1dan.github.io/MiniWAM.
Jie Chen, Ruofei Bai, Yuxin Cai +5
Department of Mechanical Engineering, National University of Singapore, Singapore · A*STAR Institute of Advanced Intelligence and Computing (A*STAR IAIC), Singapore · Nanyang Technological University, Singapore