How Representational Priors Shape Generalization in World Action Models

Junha Chun; Youngjoon Jeong; Taesup Kim

cs.ROOct 21, 2026

How Representational Priors Shape Generalization in World Action Models

Authors: Junha Chun, Youngjoon Jeong, Taesup Kim

Organizations: Department of Electrical and Computer Engineering, Seoul National University · Graduate School of Data Science, Seoul National University

Abstract

World action models (WAMs) are shown to substantially diverge under generalization performance based on their generative backbones. We investigate this gap through the representational geometry that WAMs inherit from their backbones and how policy training reorganizes it. Our central hypothesis is that a useful representational prior would structure geometry that separates nuisance information from the in-distribution manifold in advance of training as a policy. We quantify this with nuisance orthogonality; orthogonality of the feature under visual variation outside the empirical in-distribution manifold. Across various image and video backbones, we show that a higher orthogonality from the representational prior is associated with stronger robustness to visual variations after WAM training, enabling policy generalization measurable prospectively for world models. In addition, analyzing the resulting robust WAMs reveals nuisance suppression; relative suppression of feature displacement caused by such variations and robust action generation of the policy. Together, these findings provide a geometric perspective on how the representational prior of the world model shapes the generalization performance of WAMs.

Figures & tables

Figure 1: Nuisance orthogonality and suppression lead to generalization of WAM. We show that orthogonality of feature displacement with respect to in-distribution data manifold is related to generalization of WAM. This arises from the reorganization of representations during policy adaptation, which we explain through nuisance suppression. Since adaptation reshapes the representation space, the figure provides a conceptual illustration.

Figure 2: Zero-shot editing on LIBERO. Image editing models are already capable of zero-shot editing LIBERO data with visual variations.

Model	Camera		Background		Light		Sensor Noise		Average
Model	Orth.	SR (%)	Orth.	SR (%)	Orth.	SR (%)	Orth.	SR (%)	Orth.	SR (%)
Baselines
Wan2.2 $\dagger$	0.673	16.4	0.420	53.7	0.631	78.2	0.653	37.7	0.594	46.5
FLUX.2	0.803	77.6	0.774	84.9	0.845	98.5	0.773	94.9	0.799	89.0
Scratch FLUX.2	0.754	65.3	0.584	56.8	0.778	71.4	0.778	62.0	0.724	63.9
OmniGen2 $\dagger$	0.802	80.0	0.700	69.4	0.712	82.6	0.768	77.1	0.746	77.3

Table 1: Nuisance Orthgonality and LIBERO-Plus Success Rate. Nuisance orthogonality of the generative world models and LIBERO-Plus evaluation suite success rate across visual variations after WAM training.

\dagger

represents results from the original papers.

Figure 3: Visualization of Nuisance Orthgonality and LIBERO-Plus Success Rate. Overview and visualization of nuisance orthogonality and LIBERO-Plus generalization performance in Table 1 . The left figure shows tendency across variation types and the right shows the mean.

Figure 4: Rank ablation. Nuisnace orthogonality is consistent to retained rank changes.

Panel of Figure 5 — Figure 5: Nuisance responses before and after WAM adaptation. (a) and (b) show the absolute mean responses $\mathbb{E}_{x}[s(x,T)]$ and $\mathbb{E}_{x}[s_{\perp}(x,T)]$ for each WM and WAM. (c) and (d) show the corresponding WAM/WM ratios $\rho_{T}$ and $\rho_{\perp,T}$ . Suppression takes place when the values are under the dashed line marking $1.0$ .

Panel of Figure 6 — Figure 6: Representations of Camera variation. We present example for camera sweep experiments in (a) and 3D PCA Visualizations in (b). There are fitted and scaled independently in each panel for visibility.

Appendix figures & tables2 assets

Supplementary material from the paper’s appendix.

Appendix

Model	Type	Backbone & Size		Action Expert Size
FastWAM	Video	Wan2.2-TI2V ( Wan et al., 2025 )	5.00B	1.02B
( Yuan et al., 2026 )
ImageWAM (FLUX.2)	Image	FLUX.2 ( Labs, 2025 )	3.88B	0.64B
( Zhang et al., 2026b )
ImageWAM (Scratch FLUX.2)	Image	Scratch FLUX.2	3.88B	0.64B
ImageWAM (OmniGen2)	Image	OmniGen2 ( Wu et al., 2025 )	3.97B	0.76B

Appendix

Table 2: Model specifications.

Backbone	FastWAM design		ImageWAM design
Backbone	FlashMotion	Kiwi-Edit	SANA-Video 2.0	MAGI-1
Action Expert Configurations
Layers	30	30	30	30
Hidden Dimension	1024	1024	1024	1024
FFN Dimension	4096	4096	4096	4096
Activation	GELU	GELU	SwiGLU	SwiGLU

Appendix

Table 3: Additional Backbone Model Specifications and Training configurations.

World Action Models

World Action Models connect predictions of how scenes change under action to executable policies for robots, vehicles, and other embodied agents.

See all 223 papers

What Visual Representations Do World Action Models Need?
Oct 9, 2026Kai Liu, Jiangran Lyu, Tingyu Deng +9World Action Models World Models for Robotics
Institute of Automation, Chinese Academy of Sciences · Galbot · Peking University +2
What Makes World Action Models Generalize? An Empirical Study of Test-Time Future Modeling
Sep 28, 2026Renping Zhou, Zanlin Ni, Zihao Fan +8Action-Conditioned World Models Efficient VLA Model Inference
Leap Lab, Tsinghua University · University of Science and Technology of China · Beijing Institute of Technology
OpenWAM: An Open, Modular Exploration Towards Systematic World-Action Model Pretraining
Sep 7, 2026Yuran Wang, Siqiao Huang, Mingleyang Li +21World Model Learning World Models for Robotics
National University of Singapore · Tsinghua University · Peking University +4

Oct 9, 2026J/K move · Enter open · S save

What Visual Representations Do World Action Models Need?

Kai Liu, Jiangran Lyu, Tingyu Deng, Yixin Zheng+8

Institute of Automation, Chinese Academy of Sciences · Galbot · Peking University+2

World Action Models World Models for Robotics Robotic Manipulation Visual Representation Learning Multimodal Fusion

Open paper AskPDF arXiv

World Action Models (WAMs) inherit powerful priors from pretrained video generation models, but their variational autoencoder (VAE) representations are optimized for visual reconstruction rather than manipulation. External visual representations can improve manipulation, but existing methods couple representation choices with different integration mechanisms, obscuring their individual effects. We systematically compare visual representations, five integration mechanisms, and feature combinations under controlled settings. Our study yields three main findings. First, strong general-purpose visual representations provide broader robustness gains and can match or outperform representations pretrained for specific visual capabilities. Second, channel-wise joint prediction achieves the highest average success rate; further analyses suggest that future-feature prediction and the action expert's effective use of added visual information contribute to its gains. Third, additional sources should be selected for the task-relevant information they contribute beyond existing representations, as the same source can help one combination but harm another. However, even the best evaluated composition provides a smaller additional gain than the initial improvement from adding a strong general-purpose representation to the VAE-only baseline. We instantiate the resulting Representation Recipe as R

^2

-WAM. It achieves state-of-the-art performance on RoboCasa GR1 Tabletop with an 84.7% success rate and reaches 92.8% on RoboTwin 2.0, improving over the VAE-only baselines by 5.9 and 2.2 percentage points, respectively. Taken together, these findings suggest that future work should prioritize stronger general-purpose visual representations and more effective mechanisms for integrating them into WAMs, with feature composition serving as a supplementary route to improvement.

Abstract

World Action Models

Explore similar work

What Visual Representations Do World Action Models Need?

What Makes World Action Models Generalize? An Empirical Study of Test-Time Future Modeling

OpenWAM: An Open, Modular Exploration Towards Systematic World-Action Model Pretraining