Action-Conditioned World Models

Follow this topic Ask about this topic

Action-conditioned world models learn how observations or compact states evolve when an agent acts, helping agents anticipate consequences before acting in the real world. Approaches range from diffusion-based video simulators to latent JEPA dynamics models, often paired with model-predictive control, policy learning, or action generation. Research spans robot manipulation and navigation, autonomous driving, games, and other interactive settings, with methods using object-centric states, geometry, memory, and learned action representations. A central challenge is making imagined futures faithfully reflect commanded actions over long horizons: visual realism alone does not guarantee reliable planning or policy evaluation.

Latest papers

345 papers111 papers in the last four weeks, up 147% on the four weeks before. 0.6% of all new papers.

From World Models to World Action Models: A Concise Tutorial for Robotics
Jul 1, 2026Xiaoxiong Zhang, Xiong Zeng, Wei ZhangWorld Model Learning World Models for Robotics
School of Automation and Intelligent Manufacturing, Southern University of Science and Technology, Shenzhen, China · LimX Dynamics, Shenzhen, China
ABot-M0.5: Unified Mobility-and-Manipulation World Action Model
Jul 1, 2026Ronghan Chen, Yandan Yang, Zuojin Tang +18Latent Action Learning Long-Horizon Robotic Manipulation
AMAP CV Lab Alibaba Group
3D Point World Models: Point Completion Enables More Accurate Dynamics Learning
Jun 30, 2026Skand Peri, Hung Nguyen, Chanho Kim +2World Model Learning World Models for Robotics
Oregon State University
Delta-JEPA: Learning Action-Sensitive World Models via Latent Difference Decoding
Jun 30, 2026Zhenghao Zhang, Yuanxiang Wang, Zhenyu Guan +11World Models World Model-Based Planning
School of Computer Science and Technology, University of Chinese Academy of Sciences, Beijing · Institute of Information Engineering, Chinese Academy of Sciences, Beijing · School of Computer Science and Technology, Harbin Institute of Technology, Weihai +2
Efficient Sim-to-Real Transfer of World-Action Models from Synthetic Priors
Jun 30, 2026Zixing Wang, Kausik Sivakumar, Jinghuan Shang +5Sim-to-Real Transfer Action-Conditioned World Models
1Purdue University · 2Robotics and AI Institute
DreamForge-World 0.1 Preview: A Low-Compute Real-Time Controllable World Model
Jun 29, 2026Daniyel Ayupov, Artur Markov-TsoyReal-Time Interactive Video Generation World Models
DreamForge AI Lab, Kazakhstan
Learning Transferable Dynamics Priors from Action to World Modeling
Jun 28, 2026Ze Huang, Jiahui Zhang, Hairuo Liu +3Action-Conditioned Video Generation Robot Policy Learning
School of Data Science, Fudan University, Shanghai, China · Shanghai Innovation Institute, Shanghai, China · Shanghai Jiao Tong University, Shanghai, China +1
J-LAW: Joint Localization and Action-Conditioned World Modeling via Coupled Latent Factor Graphs
Jun 27, 2026Guanqun Cao, Liang ChenSimultaneous Localization and Mapping Factor Graph Optimization
Geely Technology Europe, Pumpgatan 1, 417 55 Gothenburg Sweden · State Key Laboratory of Information Engineering in Surveying, Mapping and Remote Sensing (LIESMARS), Wuhan University, China
DIM-WAM: World-Action Modeling with Diverse Historical Event Memory
Jun 26, 2026Kai Wang, Zhaopeng Gu, Yixiang Chen +7Long-Horizon Robotic Manipulation World Models for Robotics
Institute of Automation, Chinese Academy of Sciences (CASIA), Beijing, China · Shenzhen Yinwang Intelligent Technology Co., Ltd., Shenzhen, China · FiveAges, Beijing, China
ReWorld: Learning Better Representations for World Action Models
Jun 25, 2026Tianze Xia, Lijun Zhou, Kaixin Xiong +9Representation Learning World Model Learning
Huazhong University of Science and Technology · Xiaomi EV
World Action Models Enable Continual Imitation Learning with Recurrent Generative Replays
Jun 25, 2026Manish Kumar Govind, Dominick Reilly, Smit Patel +2Generative Replay Continual Robot Learning
Department of Computer Science University of North Carolina at Charlotte, United States
Hallucination in World Models is Predictable and Preventable
Jun 25, 2026Nicklas Hansen, Xiaolong WangWorld Model Learning World Models
1UC San Diego
Not All Actions Are Equal: Rethinking Conditioning for Dexterous World Model
Jun 25, 2026Zizhao Yuan, Zhengtu Liang, Taowen Wang +7Action-Conditioned Video Generation Action Representation Learning
The Hong Kong University of Science and Technology (Guangzhou) · Shenzhen University · Beijing University of Technology +1
Fast LeWorldModel
Jun 24, 2026Yuntian Gao, Xiangyu XuLatent Dynamics Modeling World Model-Based Planning
Xi’an Jiaotong University
Beyond the Next Step: Variable-Length Latent World Models for Long-Horizon Planning
Jun 19, 2026Tianqi Du, Qi Zhang, Yifei Wang +1Latent Dynamics Modeling World Models
State Key Lab of General Artificial Intelligence, School of Intelligence Science and Technology, Peking University · Amazon AGI SF Lab · Institute for Artificial Intelligence, Peking University
MemoryWAM: Efficient World Action Modeling with Persistent Memory
Jun 18, 2026Sizhe Yang, Juncheng Mu, Tianming Wei +8Long-Horizon Robotic Manipulation World Models for Robotics
The Chinese University of Hong Kong · Tsinghua University · Zhejiang University
World Action Models: A Survey
Jun 18, 2026Qiuhong Shen, Shihua Zhang, Yue Liao +5Action-Conditioned Video Generation Action-Conditioned World Models
National University of Singapore
ImageWAM: Do World Action Models Really Need Video Generation, or Just Image Editing?
Jun 17, 2026Yuyang Zhang, Wenyao Zhang, Zekun Qi +7Action-Conditioned World Models World Action Models
Shanghai Jiao Tong University · Tsinghua University · Tencent Robotics X +1
Mem-World: Memory-Augmented Action-Conditioned World Models for Persistent Robot Manipulation
Jun 17, 2026Zirui Zheng, Jiaqian Yu, Xiongfeng Peng +7Long-Horizon Robotic Manipulation World Models for Robotics
1Dalian University of Technology · 2Samsung R&D Institute China-Beijing (SRCB), China
WAM-RL: World-Action Model Reinforcement Learning with Reconstruction Rewards and Online Video SFT
Jun 16, 2026Zezhong Qian, Xiaowei Chi, Yu Qi +3World Model Learning Action-Conditioned World Models
State Key Laboratory of Multimedia Information Processing, School of Computer Science, Peking University · Northeastern University · Tsinghua University
ActWorld: From Explorable to Interactive World Model via Action-Aware Memory
Jun 16, 2026Zhexiao Xiong, Yizhi Song, Hao Kang +11HOI Generation World Model Learning
1Washington University in St. Louis · 2Intelligent Creation, ByteDance · *Work done during internship at ByteDance
Qwen-RobotWorld Technical Report: Unifying Embodied World Modeling through Language-Conditioned Video Generation
Jun 15, 2026Jie Zhang, Xiaoyue Chen, Anzhe Chen +36Action-Conditioned Video Generation Video Diffusion Models
Qwen Team
Kairos: A Regret-Aware Native World-Action Model Stack for Physical AI
Jun 15, 2026Kairos Team, Fei Wang, Shan You +21Efficient Inference Action-Conditioned World Models
Kairos Team
BadWorld: Adversarial Attacks on World Models
Jun 15, 2026Linghui Shen, Mingyue Cui, Xingyi YangAdversarial Attacks on World Models Adversarial Attacks
The Hong Kong Polytechnic University
LaWAM: Latent World Action Models for Efficient Dynamics-Aware Robot Policies
Jun 14, 2026Jialei Chen, Kai Wang, Kang Chen +9Latent Action Learning Action-Conditioned World Models
Jilin University · Zhongguancun Academy · Nankai University +4
Causal Object-Centric Models for Planning with Monte Carlo Tree Search
Jun 12, 2026Rodion Vakhitov, Leonid Ugadiarov, Alexey Skrynnik +1World Models World Model-Based Planning
MIRAI Moscow, Russia · CogAILab & MIRAI Moscow, Russia
FlowMo-WM: A World Model with Object Momentum and Hidden Ambient Drift
Jun 11, 2026Yitao Jiang, Luyang Zhao, Muhao Chen +1World Model Learning World Models for Robotics
Department of Computer Science Dartmouth College, Hanover, NH · Department of Electrical and Computer Engineering Clemson University, Clemson, SC · Department of Mechanical and Aerospace Engineering University of Houston, Houston, TX
Diffusion Transformer World-Action Model for AV Scene Prediction
Jun 11, 2026Ruslan Sharifullin, Benjamin Jiang, Kai Xi ChewAction-Conditioned Video Generation Diffusion Transformer
Stanford University
Making Foresight Actionable: Repurposing Representation Alignment in World Action Models
Jun 10, 2026Lu Qiu, Yizhuo Li, Yi Chen +3Action-Conditioned Video Generation Representation Learning
The University of Hong Kong · XPENG Robotics
HiMem-WAM: Hierarchical Memory-Gated World Action Models for Robotic Manipulation
Jun 9, 2026Xiaoquan Sun, Ruijian Zhang, Chen Cao +12Latent Action Learning Long-Horizon Robotic Manipulation
3Huazhong University of Science and Technology · 1The University of Hong Kong · 2INFIFORCE +3

Jul 1, 2026J/K move · Enter open · S save

From World Models to World Action Models: A Concise Tutorial for Robotics

Xiaoxiong Zhang, Xiong Zeng, Wei Zhang

School of Automation and Intelligent Manufacturing, Southern University of Science and Technology, Shenzhen, China · LimX Dynamics, Shenzhen, China

World Model Learning World Models for Robotics Action-Conditioned World Models World Action Models Embodied World Models

Open paper AskPDF arXiv

World models are increasingly used in embodied intelligence and generative simulation, yet their scope remains ambiguous across communities. This tutorial presents a design-space view of world models as action-conditioned predictive models that estimate the future evolution of task-relevant observations or states. We categorize existing methods into observation-space and state-space world models, comparing their trade-offs in visual fidelity, spatial structure, physical interpretability, and control usability. We further introduce world action models, which connect predicted futures with executable robot actions, and summarize four representative paradigms: imagine-then-execute, video-feature-conditioned action prediction, joint video-action modeling, and auxiliary video prediction for policy learning. The goal of this tutorial is to clarify the conceptual scope of world (action) models and provide a structured taxonomy for embodied prediction and control.