cs.LGAug 23, 2026

On the Capability Separation Between World-Model Policy Learning and Imitated World-Action Models

Authors: Yang Yu

Organizations: Nanjing University

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

World-action models predict a future outcome and then infer an associated action. Although this factorization can improve representation learning and data efficiency, it is unclear whether it provides stronger control capability than direct behavior cloning when both are trained from the same observational demonstrations. We compare a direct behavior-cloning policy, an imitation-trained world-action policy, and a policy optimized with an action-conditioned world model. At the controller-class level, every world-action policy can be flattened into a direct stochastic policy with the same closed-loop trajectory distribution. At the population level, under realizability, exact optimization, common deployment information, and distribution-preserving deployment, direct behavior cloning and world-action imitation both recover the observational behavior policy. Thus, future prediction changes the learning factorization but not the unrestricted external policy class or ideal imitation target. Action-conditioned world-model learning differs by predicting outcomes under specified actions and comparing them through a control objective. We characterize the irreducible action-specific prediction error of future models that do not condition on the candidate action, identify conditions under which a world-action joint can recover an interventional forward model, and show that observational demonstrations do not identify action effects in general. Finally, we construct an environment family in which every observational learner has positive worst-case regret, whereas one informative intervention permits zero regret. The key distinction is therefore between predicting futures associated with observed behavior and predicting consequences of specified actions for policy optimization.

Figures & tables

PolicyLearned objectPopulation targetDecision principle
πA\pi_{\mathrm{A}}πA(a∣h)\pi_{\mathrm{A}}(a\mid h)pμ(a∣h)p_{\mu}(a\mid h)Reproduce demonstrated actions
πWA\pi_{\mathrm{WA}}qF(y∣h)qI(a∣h,y)q_{F}(y\mid h)q_{I}(a\mid h,y)pμ(y,a∣h)p_{\mu}(y,a\mid h)Generate a future under the behavior distribution and decode its associated action
πWM\pi_{\mathrm{WM}}T^(dy∣h,a)\widehat{\mathsf{T}}(dy\mid h,a) or K^(dh′∣h,a)\widehat{\mathsf{K}}(dh^{\prime}\mid h,a)An interventional outcome or transition kernel, when identifiedCompare candidate actions through predicted consequences and a utility
Table 1: The three policy-learning paradigms.
More in this topic

World Action Models

World Action Models connect predictions of how scenes change under action to executable policies for robots, vehicles, and other embodied agents.

See all 221 papers

Explore similar work

CardsList
Oct 15, 2026

Can World and Action Modeling Benefit Each Other under Unified Causal Training?

World and action modeling are closely related, yet their reciprocal benefits under unified causal training remain unclear. We introduce OneWM, a simple framework for unified causal training that treats actions as both conditions for world prediction and targets for policy learning within a shared video-action architecture. Using this framework, we investigate whether learning forward dynamics conditioned on specified actions improves action learning, and whether learning to generate actions in turn improves action-conditioned world modeling. We use LIBERO, LIBERO-Plus, and RoboCasa365 as embodied testbeds to evaluate both policy learning and world modeling, complemented by real-world policy experiments. Learning the visual consequences of specified actions improves policy performance and generalization across changing scenes. In turn, action learning improves multi-view world-prediction fidelity under the same architecture. Together, these results answer both questions positively, supporting reciprocal benefits between world modeling and action generation under unified causal training.

Yue Su, Bokai Lin, Xinkai Wang +9

IAAI Team · SII · THU +1

Action-Conditioned World ModelsWorld Model LearningVisuomotor Policy LearningRobotic Manipulation
AskPDF
Mar 30, 2026

Enhancing Policy Learning with World-Action Model

This paper presents the World-Action Model (WAM), an action-regularized world model that jointly reasons over future visual observations and the actions that drive state transitions. Unlike conventional world models trained solely via image prediction, WAM incorporates an inverse dynamics objective into DreamerV2 that predicts actions from latent state transitions, encouraging the learned representations to capture action-relevant structure critical for downstream control. We evaluate WAM on enhancing policy learning across eight manipulation tasks from the CALVIN benchmark. We first pretrain a diffusion policy via behavioral cloning on world model latents, then refine it with model-based PPO inside the frozen world model. Without modifying the policy architecture or training procedure, WAM improves average behavioral cloning success from 59.4% to 71.2% over DreamerV2 and DiWA baselines. After PPO fine-tuning, WAM achieves 92.8% average success versus 79.8% for the baseline, with two tasks reaching 100%, using 8.7x fewer training steps.

Yuci Han, Alper Yilmaz

Photogrammetry and Computer Vision Lab, The Ohio State University, Columbus, OH 43210, USA.

Robot Policy LearningWorld Action ModelsAction-Conditioned World ModelsModel-Based RLRobot Imitation Learning
AskPDF
Jul 1, 2026

From World Models to World Action Models: A Concise Tutorial for Robotics

World models are increasingly used in embodied intelligence and generative simulation, yet their scope remains ambiguous across communities. This tutorial presents a design-space view of world models as action-conditioned predictive models that estimate the future evolution of task-relevant observations or states. We categorize existing methods into observation-space and state-space world models, comparing their trade-offs in visual fidelity, spatial structure, physical interpretability, and control usability. We further introduce world action models, which connect predicted futures with executable robot actions, and summarize four representative paradigms: imagine-then-execute, video-feature-conditioned action prediction, joint video-action modeling, and auxiliary video prediction for policy learning. The goal of this tutorial is to clarify the conceptual scope of world (action) models and provide a structured taxonomy for embodied prediction and control.

Xiaoxiong Zhang, Xiong Zeng, Wei Zhang

School of Automation and Intelligent Manufacturing, Southern University of Science and Technology, Shenzhen, China · LimX Dynamics, Shenzhen, China

World Model LearningWorld Models for RoboticsAction-Conditioned World ModelsWorld Action ModelsEmbodied World Models
AskPDF