Vision-Language-Action or World-Action? Just Action Models

Yonghao Yu; Lang Huang; Runyi Li; Toshihiko Yamasaki

cs.CVOct 27, 2026

Vision-Language-Action or World-Action? Just Action Models

Authors: Yonghao Yu, Lang Huang, Runyi Li, Toshihiko Yamasaki

Organizations: The University of Tokyo · National Institute of Informatics

Abstract

The vision-language-action (VLA) and world-action model (WAM) communities debate whether control policies should inherit pretrained knowledge from vision-language models or from visual generators. The debate, however, bundles three separable choices: backbone initialization, the objective applied to future observations, and whether the machinery for that future loss is kept at inference. A gap between a VLA and a WAM therefore cannot simply be attributed to any one of them. We show that decoupling these choices yields a common framework for both families, which we term Just Action Model (JAM): a pretrained visual backbone with an action expert, trained with an action loss and a foresight loss. The foresight loss trains current-observation features to predict frozen encodings of the recorded future through a prediction head discarded after training, so no visual generation is involved in either training or inference, and the backbone can be chosen freely. We first compare objectives within fixed policy architectures, then evaluate the foresight recipe across backbone families. The resulting JAM policies outperform their matched FastWAM and ImageWAM counterparts on LIBERO and LIBERO-Plus, and achieve similar or slightly higher success rates on RoboTwin. Compared with FastWAM, JAM improves the success rates from 97.0%/55.3%/91.2% to 98.2%/64.5%/91.4%, respectively. With cached foresight targets, JAM reduces estimated online training FLOPs per sample per update by 73% relative to FastWAM. Our results show that future supervision improves control without visual generation or additional inference machinery, and strong foresight performance is attainable with either generator-initialized or vision-language backbones. Project Page: https://y0uroy.github.io/JAM

Figures & tables

Figure 1: The three choices that separate VLAs, WAMs, and JAM: backbone initialization, the objective on recorded future observations, and the modules kept at inference. (a) The VLA reference uses no future objective, only the action loss $\mathcal{L}_{\mathrm{act}}$ . (b) WAMs add visual generation loss $\mathcal{L}_{\mathrm{G}}$ . The dashed loop shows generated future observations entering the action expert in some WAMs. (c) JAM uses foresight loss $\mathcal{L}_{\mathrm{F}}$ across pretraining families, discards its supervision branch, and retains the corresponding action path.

Figure 2: JAM training (left) and deployment (right), with the notation of Section 3.2 . The input and text encoders stand for architecture-specific interfaces. Recorded futures $f_{1:T}$ enter only the future anchor branch inside the dashed box; deployment discards this branch and the prediction head, leaving the backbone and action expert.

Panel of Figure 3 — Figure 3: Matched LIBERO-Long rollouts for ImageWAM and JAM, both on Ovis-U1.

Policy	$\mathcal{L}_{\mathrm{G}}$	$\mathcal{L}_{\mathrm{F}}$	Future anchor	Readout offsets	LIBERO	LIBERO-Plus
FastWAM	$\times$	$\times$	–	–	95.25	45.55
FastWAM	✓	$\times$	–	–	97.00	55.31
JAM (Wan)	$\times$	✓	Pretrained Wan	$[0]$	95.80	44.62
	$\times$	✓		$[1,2]$	97.45	55.58
	$\times$	✓		$[0,1,2]$	97.40	54.18
	$\times$	✓	Random Wan	$[1,2]$	97.65	58.63

Table 1: Generation versus foresight supervision with the policy architecture fixed.

\mathcal{L}_{\mathrm{G}}

is the generative loss and

\mathcal{L}_{\mathrm{F}}

the foresight loss. The target rows also vary the future anchor. Readout offsets are the set

\mathcal{K}

of target positions, with

0

the current frame. The total foresight weight

\lambda_{\mathrm{F}}

is fixed at

0.6

for Wan and

0.3

for Ovis-U1 and is divided equally among the readout offsets.

Panel of Figure 4 — Figure 4: Decoded future change for put both moka pots on the stove , for ImageWAM and the two JAM (Ovis-U1) policies of Table 1 with readout offsets $[0]$ and $[1]$ . Two MLP readouts reconstruct the current frame and the frame $16$ steps ahead from the same frozen features, and their difference is compared with the recorded change. Red marks pixels that get brighter and blue marks pixels that get darker, on a common scale. In this example, the features of JAM trained with offset $[1]$ make the future-frame change more readily decodable than the features of ImageWAM or JAM trained with offset $[0]$ .

Table 6

Panel of Figure 5 — Figure 5: Action-to-visual attention on one LIBERO-Long observation for put the white mug on the left plate and put the yellow and white mug on the right plate , in the agent and wrist views. (a) ImageWAM (Ovis-U1) and JAM share the deployed policy architecture but use different training objectives. The difference map compares normalized display intensities, with red indicating higher values for JAM and blue for ImageWAM. (b) JAM on a self-supervised and a vision-language backbone. Bars show attention shares over text (orange), image (blue), and action keys (green), averaged over 100 reference frames.

Table 8

VLM-based policies
Method	P.T.	Camera	Robot	Language	Light	Background	Noise	Layout	AVG
UniVLA ( Bu et al., 2025 )	✓	1.8	46.2	69.6	69.0	81.0	21.2	31.9	42.9
OpenVLA-OFT ( Kim et al., 2025a )	✓	56.4	31.9	79.5	88.7	93.3	75.8	74.2	69.6
$\pi_{0}$ ( Black et al., 2025b )	✓	13.8	6.0	58.8	85.0	81.4	79.0	68.9	53.6
$\pi_{0}$ -Fast ( Pertsch et al., 2025 )	✓	65.1	21.6	61.0	73.2	73.2	74.4	68.8	61.6
WorldVLA ( Cen et al., 2025 )	$\times$	0.1	27.9	41.6	43.7	17.1	10.9	38.0	25.0

Table 6: LIBERO-Plus success (%) by perturbation category, with the task-weighted average. Published rows retain their original evaluation protocols; † marks our evaluation on a fixed 1,506-task subset.

Appendix figures & tables17 assets

Supplementary material from the paper’s appendix.

Appendix

Wan VAE	$0,4,\ldots,32$	$18/30$	$[1,2]$	Wan + UMT5
Future anchor	Training targets			Action inference
Future anchor	Frames	Layer	Readout	Context
Wan	$0,4,\ldots,32$	$18/30$	$[1,2]$	Wan + UMT5
Yak18	$0,16$	$11/18$	$[1]$	Yak18 + Ovis
FLUX.2	$0,16$	$15/25$	$[1]$	FLUX.2 + Qwen3-4B
Qwen3.5-2B	$0,16$	$13/24$	$[1]$	Causal image–text prefix

Appendix

Table 7: Training targets and action contexts across backbones. Frame offsets are measured in environment steps relative to the current observation; Layer reports the supervised policy block / total depth.

Policy	Input $D$	Target $D_{g}$	Trunk $d$	Layers	Heads	FFN	Readouts
JAM (Wan)	3072	3072	1536	2	24	6144	2
JAM (Ovis-U1)	1536	1536	768	2	12	3072	1
JAM (FLUX.2)	3072	3072	1536	2	24	6144	1
JAM (Qwen3.5-2B)	2048	2048	1024	2	16	4096	1
JAM (DINOv2-B)	768	768	384	2	6	1536	1
JAM (DINOv2-L)	1024	1024	512	2	8	2048	1

Appendix

Table 8: Default prediction-head dimensions. Layers, Heads, and FFN describe the shared transformer trunk; Readouts denotes the number of supervised offsets.

		Training Params. (M)		TFLOPs per sample per update							Measured GPU time
Policy	Targets	Head	Anchor	Visual	Action	Head	Target	VAE	Total	Reduction	s / step	Reduction
FastWAM	–	–	–	$8.93$	$0.05$	–	–	$6.54$	$15.52$	–	$0.789$	–
JAM (Wan)	online	$71$	$0$	$3.17$	$0.05$	$0.03$	$0$	$6.54$	$9.80$	$36.9\%$	$0.532$	$32.6\%$
JAM (Wan)	cached	$71$	$0$	$3.17$	$0.05$	$0.03$	$0$	$0.93$	$4.19$	$73.0\%$	$0.258$	$67.3\%$
ImageWAM (Ovis-U1)	–	–	–	$4.91$	$0.07$	–	–	$0.83$	$5.81$	–	$0.473$	–
JAM (Ovis-U1)	online	$16.5$	$1{,}127$	$3.13$	$0.07$	$0.05$	$0.80$	$0.83$	$4.88$	$15.9\%$	$0.403$	$14.8\%$

Appendix

Table 9: Training-only parameters, estimated online FLOPs per sample per update, and measured GPU time over the profiled operations.

Policy	Future anchor	Spatial	Object	Goal	Long	AVG
FastWAM	–	97.4	99.6	96.6	94.4	97.00
FastWAM action-only	–	95.0	99.6	94.4	92.0	95.25
JAM (Wan)	Pretrained Wan	98.2	100.0	97.6	94.0	97.45
JAM (Wan)	Online Wan EMA	95.0	99.6	96.4	94.6	96.40

Appendix

Table 10: Frozen pretrained and EMA Wan future anchors on LIBERO. Success rates are in percent.

Policy	Depth	Spatial	Object	Goal	Long	AVG
FastWAM	–	97.4	99.6	96.6	94.4	97.00
FastWAM action-only	–	95.0	99.6	94.4	92.0	95.25
JAM (Wan)	$14$	98.0	99.8	96.4	94.2	97.10
	$16$	99.2	99.8	96.2	94.0	97.30
	$\mathbf{18}$	98.2	100.0	97.6	94.0	97.45
	$20$	96.8	99.2	95.4	90.4	95.45

Appendix

Table 11: LIBERO success in percent by Wan block. Depth indexes the 30 visual blocks.

Policy	$\lambda_{\mathrm{F}}$	Spatial	Object	Goal	Long	AVG
FastWAM	–	97.4	99.6	96.6	94.4	97.00
FastWAM action-only	–	95.0	99.6	94.4	92.0	95.25
JAM (Wan)	$0.4$	96.8	99.4	96.6	90.0	95.70
	$0.5$	95.6	99.8	95.2	97.0	96.90
	$\mathbf{0.6}$	98.2	100.0	97.6	94.0	97.45
	$0.7$	97.4	99.8	96.0	93.8	96.75

Appendix

Table 12: LIBERO success rates in percent across foresight loss weights

\lambda_{\mathrm{F}}

for Wan.

Policy	Head	Width	FFN	Params (M)	Spatial	Object	Goal	Long	AVG
FastWAM	–	–	–	–	97.4	99.6	96.6	94.4	97.00
FastWAM action-only	–	–	–	–	95.0	99.6	94.4	92.0	95.25
JAM (Wan)	Transformer	3072	4096	145	97.4	100.0	96.2	91.6	96.30
		1536	6144	71	98.2	100.0	97.6	94.0	97.45
		768	6144	31	96.4	100.0	97.0	89.6	95.75
	MLP	3072	–	38	95.8	100.0	94.2	91.4	95.35

Appendix

Table 13: Prediction-head capacity on LIBERO. Heads use 3,072-D inputs and outputs, two-layer trunks, and two readouts. Transformers have 24 attention heads. Their input projections are learned except at width 3,072, which uses the identity.

Method	Camera	Robot	Language	Light	Background	Noise	Layout	AVG
Qwen3.5-2B action-only	80.00	68.02	72.54	92.55	54.84	93.02	75.45	77.03
JAM (Qwen3.5-2B)	80.83	61.42	71.72	93.79	81.72	94.19	84.55	81.14
DINOv2-L action-only	37.92	32.99	86.48	86.96	85.48	78.29	72.73	68.26
JAM (DINOv2-L)	55.42	39.09	80.74	85.71	77.42	81.40	73.18	70.39

Appendix

Table 14: Action-only controls on LIBERO-Plus. Success rates are in percent; AVG is micro-averaged. Bold marks the higher score within each backbone pair.

Policy	Actions	Params	Camera	Robot	Language	Light	Background	Noise	Layout	AVG
JAM (Qwen3.5-2B)	Expert	3.60B	80.8	61.4	71.7	93.8	81.7	94.2	84.5	81.14
JAM (Qwen3.5-2B)	Shared	2.22B	80.4	64.0	68.9	92.5	76.9	96.1	78.2	79.61
JAM (Qwen3.5-4B)	Shared	4.55B	80.4	67.5	74.6	91.9	76.3	95.3	77.7	80.68

Appendix

Table 15: Separate action expert versus shared backbone weights for Qwen3.5 on LIBERO-Plus. Success rates are in percent; AVG is task-weighted and Params counts deployed parameters.

Policy	Updates	LIBERO	LIBERO-Plus
FastWAM	$10$ k	95.90	55.25
FastWAM	$20$ k	97.00	55.31
JAM (Wan)	$10$ k	97.45	48.47
JAM (Wan)	$20$ k	97.45	55.58

Appendix

Table 16: Success rates in percent for FastWAM and JAM with Wan at two training budgets. Each run has its own full learning-rate schedule, so the

10

k rows are independently trained endpoints.

Backbone	Anchor	Spatial	Object	Goal	Long	AVG
Pretrained	Pretrained	98.2	100.0	97.6	94.0	97.45
Pretrained	Random	98.0	99.8	97.0	95.8	97.65
Random	Random	85.6	98.0	88.2	65.8	84.40
Random	Pretrained	76.0	95.0	70.4	43.2	71.15

Appendix

Table 17: LIBERO success rates in percent for pretrained and random initialization of the Wan backbone and future anchor.

VLM-based policies
Method	P.T.	Spatial	Object	Goal	Long	AVG
OpenVLA ( Kim et al., 2025b )	✓	84.7	88.4	79.2	53.7	76.5
GR00T N1 ( NVIDIA et al., 2025 )	✓	94.4	97.6	93.0	90.6	93.9
$\pi_{0}$ ( Black et al., 2025b )	✓	96.8	98.8	95.8	85.2	94.1
$\pi_{0.5}$ ( Black et al., 2025a )	✓	98.8	98.2	98.0	92.4	96.9
JAM (Qwen3.5-2B, Qwen3.5 targets) †	$\times$	97.8	99.6	97.2	91.6	96.55

Appendix

Table 18: LIBERO success rates in percent by suite. AVG is the four-suite mean. Published rows follow FastWAM ( Yuan et al., 2026 ) , except GR00T N1, reported by LingBot-VA ( Li et al., 2026b ) ; they retain their source protocols.

Panel of Figure 6 — Figure 6: Paired LIBERO-Plus outcomes on 1,506 shared tasks. Left bars count tasks solved only by the baseline; right bars count tasks solved only by JAM. Panel a uses Ovis-U1, and panel b uses Wan with VAE-direct $[1,2]$ targets.

VLM-based policies
Method	P.T.	Clean	Rand.	AVG
$\pi_{0}$ ( Black et al., 2025b )	✓	65.92	58.40	62.16
$\pi_{0.5}$ ( Black et al., 2025a )	✓	82.74	76.76	79.75
JAM (Qwen3.5-2B) †	$\times$	89.70	89.86	89.78
Video generator-based policies
Motus ( Bi et al., 2026 )	✓	88.66	87.02	87.84

Appendix

Table 19: RoboTwin 2.0 success rates in percent. AVG is the mean of the Clean and Randomized settings. Published Clean and Randomized values are from FastWAM ( Yuan et al., 2026 ) , Table 1; rows marked

\dagger

are evaluated by us.

Panel of Figure 9 — Figure 9: Mean future-frame PSNR.

Policy family	Benchmark	$\Delta$	95% task-bootstrap CI	Baseline only / JAM only	McNemar $p$
FastWAM	LIBERO	$+1.15$	$[+0.20,+2.10]$	$30/53$	$0.015$
ImageWAM	LIBERO	$+3.85$	$[+1.00,+7.40]$	$42/119$	$1.01\!\times\!10^{-9}$
FastWAM	LIBERO-Plus	$+9.23$	$[+6.84,+11.69]$	$111/250$	$1.91\!\times\!10^{-13}$
ImageWAM	LIBERO-Plus	$+14.67$	$[+12.15,+17.13]$	$89/310$	$9.69\!\times\!10^{-30}$

Appendix

Table 20: Fixed-checkpoint comparisons with generative baselines, using VAE-direct

[1,2]

targets for Wan and pretrained Yak

[1]

for Ovis-U1.

\Delta

is the gain in percentage points; counts are successes unique to each policy.

Policy	Run	Spatial	Object	Goal	Long	AVG	LIBERO-Plus
JAM (Wan)	1	98.80	100.00	98.00	95.80	98.15	64.54
	2	98.60	99.80	98.20	95.40	98.00	64.28
	3	99.00	100.00	97.80	96.20	98.25	65.01
	Mean $\pm$ SD	$98.80\!\pm\!0.20$	$99.93\!\pm\!0.12$	$98.00\!\pm\!0.20$	$95.80\!\pm\!0.40$	$98.13\!\pm\!0.13$	$64.61\!\pm\!0.37$
JAM (Ovis-U1)	1	98.60	99.60	98.40	90.40	96.75	79.95
	2	98.80	99.60	98.40	90.80	96.90	80.35

Appendix

Table 21: Success rates in percent across three independent training runs. Summaries give mean

\pm

sample standard deviation, computed before rounding.

VLA Model Training

See all 50 papers

JEPA-WAM: Learning Vision-Language-Action Policies with Joint-Embedding World Modeling
Aug 10, 2026Yihan Lin, Jiawei He, Shifeng Bao +6Joint-Embedding Predictive Architecture Robotic Manipulation
School of Information, Renmin University of China, Beijing, China · XYZ Embodied AI, Beijing, China · Key Laboratory of Data Engineering and Knowledge Engineering, Beijing, China +3
V-JEPA Policy: Building Effective World-Action Models on Predictive Visual Latents
Sep 29, 2026Yang Zhang, Jiangyuan Zhao, Chenyou Fan +4Visual Representation Learning Visuomotor Policy Learning
Tsinghua University · Shanghai Jiao Tong University · Fudan University +4
LaWAM: Latent World Action Models for Efficient Dynamics-Aware Robot Policies
Jun 14, 2026Jialei Chen, Kai Wang, Kang Chen +9Latent Action Learning Action-Conditioned World Models
Jilin University · Zhongguancun Academy · Nankai University +4

Aug 10, 2026J/K move · Enter open · S save

JEPA-WAM: Learning Vision-Language-Action Policies with Joint-Embedding World Modeling

Yihan Lin, Jiawei He, Shifeng Bao, Chen Zhao+5

School of Information, Renmin University of China, Beijing, China · XYZ Embodied AI, Beijing, China · Key Laboratory of Data Engineering and Knowledge Engineering, Beijing, China+3

Joint-Embedding Predictive Architecture Robotic Manipulation Vision-Language-Action Models Latent World Models Embodied World Models

Open paper AskPDF arXiv

Robust robot control benefits from explicitly modeling state transitions, but video-generation world action models (WAMs) introduce substantial deployment cost. Existing latent WAMs avoid explicit future generation, but often compress predictive representations or separate predictive modeling from the representations used for action generation. We introduce JEPA-WAM, a latent WAM built in a pretrained V-JEPA space, which couples latent transition prediction with continuous action generation through a shared predictor. JEPA-WAM predicts a spatially structured joint current-future target that captures task-shared visual temporal structure between current and future observations, while preserving dense patch-level correspondence. Through the shared predictor, transition supervision directly shapes the backbone, from which dedicated representations are extracted for action prediction. The same design can also be instantiated in pretrained VLA policies while preserving their original perception and action pathways. On LIBERO-Plus, JEPA-WAM achieves 79.2%, the best result without large-scale robot-policy pretraining, while its pretrained

π_{0.5}

instantiation reaches 86.3%, achieving the best overall performance. Experiments on RoboTwin 2.0 and real-world bimanual manipulation further demonstrate strong generalization under visual and spatial shifts.

Abstract

VLA Model Training

Explore similar work

JEPA-WAM: Learning Vision-Language-Action Policies with Joint-Embedding World Modeling

V-JEPA Policy: Building Effective World-Action Models on Predictive Visual Latents

LaWAM: Latent World Action Models for Efficient Dynamics-Aware Robot Policies