GAWM: A Compact Latent World-Action Model for Multi-Task Robot Manipulation

Yang Liu; Xiaoli Liu; Liuzhao Li; Yuduo Song; Muyuan Liu; Xiang Gao

cs.ROOct 27, 2026

GAWM: A Compact Latent World-Action Model for Multi-Task Robot Manipulation

Authors: Yang Liu, Xiaoli Liu, Liuzhao Li, Yuduo Song, Muyuan Liu, Xiang Gao

Organizations: Genisom AI, Beijing, China

Abstract

World-action models (WAMs) incorporate the prediction of future observations into robot policy learning. However, most WAMs build on billion-parameter vision-language or video-generation backbones, which incur high training cost and inference latency. In this letter, we present GAWM (Geometry-Aware World Model), a compact multi-task WAM that predicts future observations in the latent space of a frozen DINOv3 encoder. GAWM compresses the features of each camera view into a small set of tokens. A lightweight predictor, conditioned on the task rather than on actions, forecasts these tokens at two future horizons. The forecast is supervised with the dense patch features of future frames, which requires it to encode the spatial layout of the future scene. An action decoder directly regresses a chunk of future actions from the current and predicted tokens. With fewer than 100M parameters and no robot or video pretraining, GAWM outperforms billion-parameter WAMs on RoboTwin 2.0 and performs comparably to billion-parameter VLAs on LIBERO. On a dual-arm robot, it achieves lower action-prediction error on held-out demonstrations than a fine-tuned billion-parameter VLA, with substantially lower inference latency. Ablation studies show that the latent world model substantially improves performance when training data or training time is limited. Code is available at https://github.com/gaoxiang12/ga-wm.

Figures & tables

Fig. 1: Success rate versus model size on LIBERO (left; GAWM-S with additional in-domain data) and on the clean (middle) and randomized (right) settings of RoboTwin 2.0. Top: success rate versus size. Bottom: methods ranked by success rate (SR) with size (as in Table I , or as reported; N/R: not reported). Baselines (references in Table I and [ 15 , 16 , 17 , 18 , 19 , 20 , 21 , 22 , 23 , 24 , 25 , 26 , 27 , 28 ] ): blue, as reported by TurboVLA [ 29 ] ; gray, other sources. On RoboTwin 2.0, GAWM-S is the smallest and best-performing of the plotted methods.

Fig. 2: Overview of GAWM. A frozen DINOv3 encoder and a query adapter encode each camera view into 64 visual tokens, stacked into $z_{t}$ . The latent predictor, conditioned on the task code $g_{k}$ , forecasts these tokens at two future horizons, and the action decoder directly regresses a chunk of future actions from the current tokens, the forecast, the robot state and the task code. Dashed panel: training only; a patch decoder maps the forecast to the encoder’s patch features of the recorded future frames (thumbnails as in Fig. 4 ).

Vision-language-action models
				LIBERO					RoboTwin 2.0
Method	Params	Robot PT	Task input	Spatial	Object	Goal	Long	Avg.	Clean	Rand.
OpenVLA [ 2 ]	7.5B ∗	$\checkmark$	Lang.	84.7	88.4	79.2	53.7	76.5	–	–
$\pi_{0}$ [ 3 ]	3.2B ∗	$\checkmark$	Lang.	96.8	98.8	95.8	85.2	94.2	–	–
$\pi_{0.5}$ [ 4 ]	3.4B	$\checkmark$	Lang.	98.8	98.2	98.0	92.4	96.9	82.74 a	76.76 a
OpenVLA-OFT [ 43 ]	7.6B	$\checkmark$	Lang.	97.7	98.0	96.1	95.3	96.8	–	–

TABLE I: Success Rates (%) on LIBERO and RoboTwin 2.0

	LIBERO, $\frac{1}{4}$ data					LIBERO	RoboTwin
	Spatial	Object	Goal	Long	Avg.	Extended	191k steps
GAWM-S	90.2	85.4	87.8	58.4	80.5	95.6	90.7
w/o world model	77.2	89.0	80.2	37.4	71.0	93.1	18.8
$\Delta$	$+$ 13.0	$-$ 3.6	$+$ 7.6	$+$ 21.0	$+$ 9.5	$+$ 2.5	$+$ 71.9
95% CI					[5.2, 14.2]	[ $-$ 0.8, 6.1]	[65.2, 78.0]

TABLE II: Ablation of the World Model

Fig. 3: Inference latency per action-chunk query. Official implementations on the same GPU model (NVIDIA RTX PRO 5000) with batch size 1 and default precision and denoising steps. Latency covers the modules counted in Table I , from preprocessed inputs to the action chunk (median of 200 queries).

\dagger

: LIBERO configuration; all others use RoboTwin 2.0. Parentheses: parameters as in Table I . Baselines are randomly initialized, which does not change the computation.

Fig. 4: Decoded forecasts. GAWM-S on RoboTwin 2.0, head camera. Left: frames $t$ , $t{+}16$ and $t{+}32$ . Right: first three principal components of the encoder patch features of the true future frames (True) and of the decoded forecast (Pred.), projected on a PCA basis shared within each example. Labels: mean per-patch cosine similarity to the true features for the forecast and for frame $t$ . Examples: first clean demonstrations, at the frame with the largest 32-frame feature change within 20–80% of the episode.

Fig. 5: Attention of GAWM-S on RoboTwin 2.0. Top: head-camera frame $t$ of the examples of Fig. 4 and the adapter cross-attention over its patches (mean over queries, heads and layers; brighter is higher). Bottom: share of the action decoder’s cross-attention on each memory group (current tokens, near and far forecasts, state tokens) per action step (mean over heads, layers and 1,000 anchors; bands: interquartile range).

Fig. 6: Real-robot task. Head-camera frames of a demonstration of the green-bottle task. The four tasks differ only in the target bottle.

	GAWM-S	$\pi_{0.5}$ (40k)	$\pi_{0.5}$ (60k)
Joint error, first step	0.072	0.098	0.092
Joint error, full chunk	0.088 a	0.106 b	0.101 b
Gripper error, first step	0.020	0.028	0.023
Closed-loop success, yellow bottle	46/50 c	–	39/50
Latency, median / 95th pct. (ms)	9.3 / 9.6	63.8 / 63.9

TABLE III: Evaluation on the Real Robot

World Action Models

World Action Models connect predictions of how scenes change under action to executable policies for robots, vehicles, and other embodied agents.

See all 221 papers

LaWAM: Latent World Action Models for Efficient Dynamics-Aware Robot Policies
Jun 14, 2026Jialei Chen, Kai Wang, Kang Chen +9Latent Action Learning Action-Conditioned World Models
Jilin University · Zhongguancun Academy · Nankai University +4
LiLa-WAM: Lightweight Latent Reasoning World-Action Model for Robotic Manipulation
Aug 4, 2026Fan Yang, Yuting Su, Xiaobo Wang +7World Models for Robotics Latent Visual Reasoning
1Tianjin University · 2Shenzhen University of Advanced Technology · 5Sangfor Technologies Inc. +2
MiniWAM: Learning Compact Future Targets for Efficient World-Action Modeling
Oct 8, 2026Jie Chen, Ruofei Bai, Yuxin Cai +5World Models for Robotics World Action Models
Department of Mechanical Engineering, National University of Singapore, Singapore · A*STAR Institute of Advanced Intelligence and Computing (A*STAR IAIC), Singapore · Nanyang Technological University, Singapore

Jun 14, 2026J/K move · Enter open · S save

LaWAM: Latent World Action Models for Efficient Dynamics-Aware Robot Policies

Jialei Chen, Kai Wang, Kang Chen, Shuaihang Chen+8

Jilin University · Zhongguancun Academy · Nankai University+4

Latent Action Learning Action-Conditioned World Models Vision-Language-Action Models World Action Models Latent World Models Vision-Based Robot Control

Open paper AskPDF arXiv

Vision-Language-Action models (VLAs) leverage large-scale vision-language pretraining for semantic robot control, but often lack explicit foresight into how robot actions change the scene. World-Action Models (WAMs) address this limitation by conditioning policies on predicted futures, yet existing approaches typically rely on computationally expensive video generation with substantial pixel-level redundancy. We present LaWAM, a Latent World Action Model that exposes predictive dynamics to robot policies through compact latent visual subgoals instead of reconstructed future video. At the core of LaWAM is a latent-action-conditioned Latent World Model (LaWM). We obtain LaWM by training a latent action model in the latent space of a pretrained vision foundation model and repurposing its forward decoder to predict future observation features for scene evolution. LaWAM then conditions action generation on these predicted latent visual subgoals to enable dynamics-aware robot control. LaWAM achieves state-of-the-art or competitive success rates (SRs) across LIBERO (98.6% SR), RoboTwin (91.22% SR), and real-world manipulation tasks while retaining low-latency inference. LaWAM runs in 187 ms per action-chunk prediction and achieves up to 24x lower wall-clock latency than pixel-space WAMs.

Abstract

World Action Models

Explore similar work

LaWAM: Latent World Action Models for Efficient Dynamics-Aware Robot Policies

LiLa-WAM: Lightweight Latent Reasoning World-Action Model for Robotic Manipulation

MiniWAM: Learning Compact Future Targets for Efficient World-Action Modeling