World Model Learning

Follow this topic Ask about this topic

World model learning studies how agents can represent a changing environment and predict what may happen next, especially after an action. Common approaches include latent-state models such as RSSMs and JEPAs, video diffusion models, and structured representations based on objects, graphs, geometry, physics, or executable rules. These models are used for imagined rollouts and planning in settings such as robot manipulation, autonomous driving, games, and web or GUI agents, as well as for scientific forecasting and simulation. A central challenge is that strong one-step or visual predictions do not guarantee useful decisions: long-horizon consistency, action accuracy, uncertainty, and transfer to unfamiliar settings remain difficult to balance with computational cost.

Latest papers

419 papers88 papers in the last four weeks, up 83% on the four weeks before. 0.5% of all new papers.

HorizonDrive: Self-Corrective Autoregressive World Model for Long-horizon Driving Simulation
May 12, 2026Conglang Zhang, Yifan Zhan, Qingjie Wang +10World Model Learning Long-Horizon Video Generation
Wuhan University · The University of Tokyo · Horizon Robotics +3
PROWL: Prioritized Regret-Driven Optimization for World Model Learning
May 11, 2026Ahmet H. Güzel, Jenny Seidenschwarz, Benjamin Graham +3Adversarial Training World Model Learning
University College London AI Centre · Odyssey · University of Basel
DeepSight: Long-Horizon World Modeling via Latent States Prediction for End-to-End Autonomous Driving
May 11, 2026Lingjun Zhang, Changjie Wu, Linzhe Shi +6World Model Learning End-to-End Autonomous Driving
Tsinghua University · Amap, Alibaba Group · Nanyang Technological University
How Mobile World Model Guides GUI Agents?
May 11, 2026Weikai Xu, Kun Huang, Yunren Feng +10World Model Learning GUI Agents
1Nanyang Technological University · 2MiLM Plus, Xiaomi Inc. · 3Independent Researchers +3
Do multimodal models imagine electric sheep?
May 10, 2026Santhosh Kumar Ramakrishnan, Carl Vondrick, Raja Giryes +2Visual Spatial Reasoning World Model Learning
Apple
DeformMaster: An Interactive Physics-Neural World Model for Deformable Objects from Videos
May 10, 2026Can Li, Zhoujian Li, Ren Li +4World Model Learning World Models
Nankai University · Rightly Robotics, A4X · Zhejiang University +2
Sub-JEPA: Subspace Gaussian Regularization for Stable End-to-End World Models
May 10, 2026Kai Zhao, Dongliang Nie, Yuchen Lin +4Representation Learning Representation Collapse
1Shanghai University · 2The University of Manchester · 3Nankai University
Probing the Impact of Scale on Data-Efficient, Generalist Transformer World Models for Atari
May 9, 2026Jooyeon KimWorld Model Learning Transformer
Graduate School of Artificial Intelligence UNIST
GEM: Generating LiDAR World Model via Deformable Mamba
May 8, 2026Yang Wu, Zhaojiang Liu, Qiang Meng +5Mamba World Model Learning
SJTU · FDU · NTU +1
Sword: Style-Robust World Models as Simulators via Dynamic Latent Bootstrapping for VLA Policy Post-Training
May 8, 2026Jiaxuan Gao, Yongjian Guo, Zhong Guan +5World Model Learning World Models
Tianjin University · Tsinghua University · Swinburne University of Technology +1
Three-in-One World Model: Energy-Based Consistency, Prediction, and Counterfactual Inference for Marketing Intervention
May 8, 2026Junichiro NiimiCausal Effect Estimation World Model Learning
1Meijo University
Learning Visual Feature-Based World Models via Residual Latent Action
May 8, 2026Xinyu Zhang, Zhengtong Xu, Yutian Tao +3Latent Action Learning World Model Learning
Rutgers University · Purdue University · University of Wisconsin-Madison
AGWM: Affordance-Grounded World Models for Environments with Compositional Prerequisites
May 7, 2026Qinshi Zhang, Weipeng Deng, Zhihan Jiang +4World Model Learning Interactive World Models
University of California, San Diego · University of Hong Kong · Columbia University +2
Reconstruction or Semantics? What Makes a Latent Space Useful for Robotic World Models
May 7, 2026Nilaksh, Saurav Jha, Artem Zholus +1World Model Learning World Models for Robotics
Chandar Research Lab · Mila – Quebec AI Institute · Polytechnique Montréal +1
Earth-o1: A Grid-free Observation-native Atmospheric World Model
May 7, 2026Junchao Gong, Kaiyi Xu, Wangxu Wei +22World Model Learning Weather Forecasting
School of Electronic Information and Electrical Engineering, Shanghai Jiao Tong University, Shanghai, China. · School of Information Science and Technology, University of Science and Technology of China, Anhui, China. · State Key Laboratory of Earth System Numerical Modeling and Application, Institute of Atmospheric Physics, Chinese Academy of Sciences, Beijing, China. +5
HaM-World: Soft-Hamiltonian World Models with Selective Memory for Planning
May 7, 2026Haoyun Tang, Haodong Cui, Keyao Xu +2World Model Learning Latent Dynamics Modeling
1Xi’an Jiaotong University · 2Huazhong University of Science and Technology · 3Nankai University +1
Driver-WM: A Driver-Centric Traffic-Conditioned Latent World Model for In-Cabin Dynamics Rollout
May 6, 2026Haozhuang Chi, Daosheng Qiu, Hao Su +4World Model Learning Causal World Models
Nanyang Technological University, Singapore · Hubei University, Wuhan, China · Osaka University, Osaka, Japan
Learning to Theorize the World from Observation
May 5, 2026Doojin Baek, Gyubin Lee, Junyeob Baek +2Computational Cognitive Modeling World Model Learning
KAIST
HERMES++: Toward a Unified Driving World Model for 3D Scene Understanding and Generation
Apr 30, 2026Xin Zhou, Dingkang Liang, Xiwu Chen +4World Model Learning 3D Scene Generation
Huazhong University of Science and Technology · Mach Drive · the University of Hong Kong
World Model for Robot Learning: A Comprehensive Survey
Apr 30, 2026Bohan Hou, Gen Li, Jindou Jia +15World Model Learning World Models for Robotics
Nanyang Technological University · University of California, Berkeley · Stanford University +6
Graph World Models: Concepts, Taxonomy, and Future Directions
Apr 30, 2026Jiawei Liu, Senqiao Yang, Mingjun Wang +2World Model Learning World Models
The Chinese University of Hong Kong, Hong Kong, China · Tsinghua University, Beijing, China
Privileged Foresight Distillation: Zero-Cost Future Correction for World Action Models
Apr 28, 2026Pengcheng Fang, Hongli Chen, Xiaohao CaiWorld Model Learning Action-Conditioned World Models
The University of Southampton · The University of Queensland
Agentic World Modeling: Foundations, Capabilities, Laws, and Beyond
Apr 24, 2026Meng Chu, Xuan Billy Zhang, Kevin Qinghong Lin +47World Model Learning Action-Conditioned World Models
Mask World Model: Predicting What Matters for Robust Robot Policy Learning
Apr 21, 2026Yunfan Lou, Xiaowei Chi, Xiaojie Zhang +9Robot Policy Learning Video Diffusion Models
National University of Singapore, Singapore · Beijing Academy of Artificial Intelligence, Beijing, China · The Hong Kong University of Science and Technology, Hong Kong, China +3
Curiosity-Critic: Cumulative Prediction Error Improvement as a Tractable Intrinsic Reward for World Model Training
Apr 20, 2026Vin Bhaskara, Haicheng WangIntrinsic Reward Methods Reinforcement Learning
Department of Computer Science, University of Toronto, Toronto, Canada.
Infrastructure-Centric World Models: Bridging Temporal Depth and Spatial Breadth for Roadside Perception
Apr 19, 2026Siyuan Meng, Chengbo AiWorld Model Learning World Models
Department of Civil and Environmental Engineering, University of Massachusetts Amherst
Active World-Model with 4D-informed Retrieval for Exploration and Awareness
Apr 17, 2026Elaheh Vaezpour, Amirhosein Javadi, Tara JavidiActive Perception World Model Learning
KavAI · UCSD
Human Cognition in Machines: A Unified Perspective of World Models
Apr 17, 2026Timothy Rupprecht, Pu Zhao, Amir Taherin +20Computational Cognitive Modeling World Model Learning
Northeastern University, Boston, MA, USA · EmbodyX Inc., San Mateo, CA, USA · Tulane University, New Orleans, LA, USA +2
The Global Neural World Model: Spatially Grounded Discrete Topologies for Action-Conditioned Planning
Apr 17, 2026Noureddine KermicheWorld Model Learning World Model-Based Planning
Western Digital Corporation, Irvine, CA, USA
Learning Ad Hoc Network Dynamics via Graph-Structured World Models
Apr 16, 2026Can Karacelebi, Yusuf Talha Sahin, Elif Surer +1World Model Learning World Models
Department of Computer Engineering, Middle East Technical University, Ankara, Turkiye · Türk Telekom, Ankara, Turkiye · Graduate School of Informatics, Middle East Technical University, Ankara, Turkiye

May 12, 2026J/K move · Enter open · S save

HorizonDrive: Self-Corrective Autoregressive World Model for Long-horizon Driving Simulation

Conglang Zhang, Yifan Zhan, Qingjie Wang, Zhanpeng Ouyang+9

Wuhan University · The University of Tokyo · Horizon Robotics+3

World Model Learning Long-Horizon Video Generation Diffusion Model Distillation World Models for Autonomous Driving Autoregressive Video Generation Autonomous Driving Simulation

Open paper AskPDF arXiv

Closed-loop driving simulation requires real-time interaction beyond short offline clips, pushing current driving world models toward autoregressive (AR) rollout. Existing AR distillation approaches typically rely on frame sinks or student-side degradation training. The former transfers poorly to driving due to fast ego-motion and rapid scene changes, while the latter remains bounded by the teacher's single-pass output length and thus provides only a limited supervision horizon. A natural question is: can the teacher itself be extended via AR rollout to provide unbounded-horizon supervision at bounded memory cost? The key difficulty is that a standard teacher drifts under its own predictions, contaminating the supervision it provides. Our key insight is to make the teacher rollout-capable, ensuring reliable supervision from its own AR rollouts. This is instantiated as HorizonDrive, an anti-drifting training-and-distillation framework for AR driving simulation. First, scheduled rollout recovery (SRR) trains the base model to reconstruct ground-truth future clips from prediction-corrupted histories, yielding a teacher that remains stable across long AR rollouts. Second, the rollout-capable teacher is extended via AR rollout, providing long-horizon distribution-matching supervision under bounded memory, while a short-window student aligns to it with teacher rollout DMD (TRD) for efficient real-time deployment. HorizonDrive natively supports minute-scale AR rollout under bounded memory; on nuScenes, HorizonDrive reduces FID by 52% and FVD by 37%, and lowers ARE and DTW by 21% and 9% relative to the strongest long-horizon streaming baselines, while remaining competitive with single-pass driving video generators.