Action-Conditioned World Models

Overview

Action-conditioned world models learn how observations or compact states evolve when an agent acts, helping agents anticipate consequences before acting in the real world. Approaches range from diffusion-based video simulators to latent JEPA dynamics models, often paired with model-predictive control, policy learning, or action generation. Research spans robot manipulation and navigation, autonomous driving, games, and other interactive settings, with methods using object-centric states, geometry, memory, and learned action representations. A central challenge is making imagined futures faithfully reflect commanded actions over long horizons: visual realism alone does not guarantee reliable planning or policy evaluation.

Latest papers

CardsList

378 papers45 papers in the last four weeks, down 59% on the four weeks before. 0.3% of all new papers.

  1. DiWALI: Diffusion Policy Learning in World Models via Latent Action Inference from Human and Robot Play

    Nov 18, 2026Akshay L Chandra, Iman Nematollahi, Lingyue Zhao +4Diffusion PolicyRobot Learning from Video

  2. Counterfactual World Models: JEPAs That Reason About "What If"

    Nov 18, 2026Randall Balestriero, Yann LeCun, Kerda VarakuCounterfactual PredictionCausal World Models

  3. Target-Consistent Action Guidance via Semi-Unbalanced Optimal Transport for Drifting World Models

    Nov 18, 2026Md Abtahi Majeed Chowdhury, Md Rifat Ur Rahman, Md Sakib Hossain Shovon +2Action-Conditioned World ModelsWorld Models for Robotics

  4. Plausible Videos Can Lead to Poor Decisions: Diagnosing and CURE-ing World Models for Planning

    Nov 17, 2026Raphael C. Kim, Michele Santacatterina, Jingsen Zhu +1World Model-Based PlanningAction-Conditioned World Models

  5. WM-Quant: Rollout-Aware Post-Training Quantization for JEPA-Style World Models

    Nov 16, 2026William Hammer, Andrew Heuer, Nikhil Krishna +2Post-Training QuantizationWorld Models

  6. ThinkWAM: Rethinking Latent Futures for Action Refinement

    Nov 16, 2026Haoyun Tang, Baqiao Deng, Enjin Zhou +7Action-Conditioned World ModelsLatent World Models

  7. WA-JEPA: Goal-Aware Predictive Feedback Action Refinement for a World-Action Model

    Nov 14, 2026Tao An, Weihang Liang, Tianqi Wang +7Action-Conditioned World ModelsEfficient Inference for World Action Models

  8. Rethinking Actions: Continuity-Aware Priors for World-Model Planning

    Nov 12, 2026Shaobo Yang, Zichen Ai, Ziyang Ma +2World Model-Based PlanningAction-Conditioned World Models

  9. ActWorld-JEPA: Action-Conditioned JEPA World Models Meet Latent Planning for End-to-End Autonomous Driving

    Nov 12, 2026Ziying Song, Shengkai Zhang, Caiyan Jia +10World Models for Autonomous DrivingAutonomous Driving Planning

  10. Does Your World Model Know Actions Have Consequences?

    Nov 11, 2026Ayush Gaggar, Jared Berry, Todd D. MurpheyWorld Model EvaluationAction-Conditioned World Models

  11. Fast Inference-Time Policy Refinement via Action-Affine World Models

    Nov 11, 2026Run Wang, Yilin Mo, Liang WuInference-Time OptimizationAction-Conditioned World Models

  12. QCC-WM: Linking Macro-State Dynamics and Visual Generation via Quotient Causal Construction

    Nov 9, 2026Zhizhong Fu, Wei Zhou, Zhaoyang Jiang +7Action-Conditioned World ModelsVideo World Models

  13. Learning to Act with World Models by Valuing What Comes Next

    Nov 9, 2026Chenghao Gao, Zhi Zhang, Dian Jin +2Model-Based RLSparse-Reward RL

  14. How Many Candidates Does a One-Step World Model Need? A Random-Codebook Account of Nearest-of-m Training on Frozen DINO Tokens

    Nov 9, 2026Yong Hwi Song, Keon Myung LeeWorld Model LearningOne-Step Generative Modeling

  15. Personalized health coaching via coachable health age and counterfactual wearable trajectories

    Nov 5, 2026Simon A. Lee, Zeinab Esmaeilpour, Menglian Zhou +17HealthcareAction-Conditioned World Models

  16. PlayGen2: Learning Real-Time Playable Worlds with Explicit 3D State

    Nov 5, 2026Jihai Zhang, Yuhan Zhou, Junyou Li +3Game World ModelsVideo Games

  17. MKD2World: A Typed Spatial MarkDown World Model For Web Agents

    Nov 4, 2026Zhihao Nan, Sifan Zhou, Yiming Cheng +5World Model LearningWorld Models

  18. PrimeSim: A Generalist World Model Simulator from Large-Scale Cross-Embodiment Demonstrations

    Nov 3, 2026Travis Davies, Wentao Ma, Lihao Guo +10World Models for RoboticsAction-Conditioned World Models

  19. Seeing Through the Illusion: Test-Time Scaling for World Action Models

    Nov 2, 2026Cheikh Ahmed, Mahdi Mostajabdaveh, Milad Yazdani +1World Models for RoboticsEfficient Inference for World Action Models

  20. INTERACT: Learning Interaction-Aware World Dynamics from Driving Videos for Autonomous Driving

    Nov 2, 2026Zhan Chen, Songyan Zhang, Wenhui Huang +5World Models for Autonomous DrivingAutonomous Driving Planning

  21. Stand-In Effects and Pooling Amplification: Lessons from Biological World Models for Testing Action Effects

    Nov 1, 2026Bowen Xu, Boyu Chen, Jiaying LiWorld Model EvaluationStatistical Hypothesis Testing

  22. Agentic World Modeling for Agent Test-time Scaling without Environment Replica

    Oct 29, 2026Wendi Li, Huseyin A. Inan, Wei-Ning Chen +4Action-Conditioned World ModelsTest-Time Scaling