World Model Learning

Follow this topic Ask about this topic

World model learning studies how agents can represent a changing environment and predict what may happen next, especially after an action. Common approaches include latent-state models such as RSSMs and JEPAs, video diffusion models, and structured representations based on objects, graphs, geometry, physics, or executable rules. These models are used for imagined rollouts and planning in settings such as robot manipulation, autonomous driving, games, and web or GUI agents, as well as for scientific forecasting and simulation. A central challenge is that strong one-step or visual predictions do not guarantee useful decisions: long-horizon consistency, action accuracy, uncertainty, and transfer to unfamiliar settings remain difficult to balance with computational cost.

Latest papers

419 papers88 papers in the last four weeks, up 83% on the four weeks before. 0.5% of all new papers.

Beyond State Consistency: Behavior Consistency in Text-Based World Models
Apr 15, 2026Youling Huang, Guanqiao Chen, Junchi Yao +8World Models World Model Learning
Dalian University of Technology · Microsoft · MBZUAI +1
Zero-shot World Models Are Developmentally Efficient Learners
Apr 11, 2026Khai Loong Aw, Klemen Kotar, Wanhee Lee +6Zero-Shot Learning Computational Cognitive Modeling
Stanford University
Advantage-Guided Diffusion for Model-Based Reinforcement Learning
Apr 10, 2026Daniele Foffano, Arvid Eriksson, David Broman +2Model-Based RL Diffusion Model Guidance
Division of Decision and Control Systems of the EECS School at KTH Royal Institute of Technology, Stockholm, Sweden. · Digital Futures · Division of Software and Computer Systems of the EECS School at KTH Royal Institute of Technology, Stockholm, Sweden.
Morphology-Conditioned World Model for Cross-Embodiment Quadrupedal Locomotion
Apr 9, 2026Mohamad H. Danesh, Chenhao Li, Amin Abyaneh +5World Models for Robotics Cross-Embodiment Robot Learning
McGill University, Canada · Mila - Quebec AI Institute, Canada · ETH Zurich, Switzerland +1
A Frame is Worth One Token: Efficient Generative World Modeling with Delta Tokens
Apr 6, 2026Tommie Kerssies, Gabriele Berton, Ju He +5Video World Models World Model Learning
Amazon · Eindhoven University of Technology · Johns Hopkins University
ExploreVLA: Dense World Modeling and Exploration for End-to-End Autonomous Driving
Apr 3, 2026Zihao Sheng, Xin Ye, Jingru Luo +2World Model Learning VLMs for Autonomous Driving
Bosch Research North America & Bosch Center for Artificial Intelligence (BCAI) · University of Wisconsin–Madison
WildWorld: A Large-Scale Dataset for Dynamic World Modeling with Actions and Explicit State toward Generative ARPG
Mar 24, 2026Zhen Li, Zian Meng, Shuwei Shi +5World Model Learning World Models
Alaya Studio, Shanda AI Research Tokyo · Beijing Institute of Technology · Shenzhen MSU-BIT University +2
Language-Conditioned World Modeling for Visual Navigation
Mar 23, 2026Yifei Dong, Fengyi Wu, Yilong Dai +8World Model Learning Vision-Language Navigation
University of Washington · National University of Singapore · Clemson University +2
What Do World Models Learn in RL? Probing Latent Representations in Learned Environment Simulators
Mar 23, 2026Xinyu ZhangWorld Models World Model Learning
Anyscale
FluidWorld: Reaction-Diffusion Dynamics as a Predictive Substrate for World Models
Mar 22, 2026Fabien PollyReaction-Diffusion Systems World Model Learning
Independent Researcher
Structured Latent Dynamics in Wireless CSI via Homomorphic World Models
Mar 20, 2026Salmane Naoumi, Mehdi Bennis, Marwa ChafiiLatent Dynamics Modeling Channel State Prediction
NYU WIRELESS, NYU Tandon School of Engineering, New York, USA. · Center for Wireless Communications, University of Oulu, Finland. · Engineering Division, New York University (NYU), Abu Dhabi, UAE.
R2-Dreamer: Redundancy-Reduced World Models without Decoders or Augmentation
Mar 18, 2026Naoki Morihira, Amal Nahar, Kartik Bharadwaj +3Model-Based RL World Model Learning
Honda R&D Co., Ltd. · The University of Tokyo · RIKEN AIP
From Digital Twins to World Models:Opportunities, Challenges, and Applications for Mobile Edge General Intelligence
Mar 18, 2026Jie Zheng, Dusit Niyato, Changyuan Zhao +2World Models Edge AI
State-Province Joint Engineering and Research Center of Advanced Networking and Intelligent Information Services, College of Computer, Northwest University, Xian, 710127, Shaanxi, China. · College of Computing and Data Science, Nanyang Technological University, Singapore 639798. · the Automation of School, Guangdong University of Technology, Guangzhou 510006, China.
Predictive Statistics Shape Emergent World Representations of Grid Walkers
Mar 17, 2026Sasha Brenner, Thomas R. Knösche, Nico ScherfPredictive Representation Learning World Model Learning
Max Planck Institute for Human Cognitive and Brain Sciences, Leipzig, Germany · Leipzig University, Germany · ScaDS.AI, Dresden/Leipzig, Germany
Fast-WAM: Do World Action Models Need Test-time Future Imagination?
Mar 17, 2026Tianyuan Yuan, Zibin Dong, Yicheng Liu +1World Action Models Efficient Inference for World Action Models
IIIS, Tsinghua University · Galaxea AI
RS-WorldModel: a Unified Model for Remote Sensing Understanding and Future Sense Forecasting
Mar 16, 2026Linrui Xu, Zhongan Wang, Fei Shen +4World Model Learning Remote Sensing VLMs
Central South University · Zhejiang University · National University of Singapore +2
EyeWorld: A Generative World Model of Ocular State and Dynamics
Mar 14, 2026Ziyu Gao, Xinyuan Wu, Xiaolan Chen +10Ophthalmic Image Analysis Medical Image Analysis
School of Optometry, The Hong Kong Polytechnic University, Hong Kong. · Eye Center of Second Affiliated Hospital, School of Medicine, Zhejiang University, Hangzhou, 310009, Zhejiang, China · Zhejiang Provincial Key Laboratory of Ophthalmology, Zhejiang Provincial Clinical Research Center for Eye Diseases, Zhejiang Provincial Engineering Institute On Eye Diseases, Hangzhou, 310009, Zhejiang, China +5
Egocentric World Model for Photorealistic Hand-Object Interaction Synthesis
Mar 13, 2026Dayou Li, Lulin Liu, Bangya Liu +6HOI Generation World Model Learning
Texas A&M University · University of Minnesota · University of Wisconsin–Madison +4
LeWorldModel: Stable End-to-End Joint-Embedding Predictive Architecture from Pixels
Mar 13, 2026Lucas Maes, Quentin Le Lidec, Damien Scieur +2World Model Learning Joint-Embedding Predictive Architecture
Mila & Université de Montréal · New York University · Samsung SAIL +1
VGGT-World: Transforming VGGT into an Autoregressive Geometry World Model
Mar 13, 2026Xiangyu Sun, Shijie Wang, Fengyi Zhang +53D Scene Prediction World Model Learning
UQMM Lab, The University of Queensland · Beijing Jiaotong University
From Gradients to Riccati Geometry: Kalman World Models for Single-Pass Learning
Mar 13, 2026Andrew KirulutaGradient-Free Optimization State Space Models
UC Berkeley School of Information
ResWM: Residual-Action World Model for Visual RL
Mar 11, 2026Jseen Zhang, Gabriel Adineera, Jinzhou Tan +1Model-Based RL World Model Learning
University of California, San Diego USA · Texas A&M University-Commerce USA
RAE-NWM: Navigation World Model in Dense Visual Representation Space
Mar 10, 2026Mingkun Zhang, Wangtian Shen, Fan Zhang +3Visual Navigation World Model Learning
Department of Precision Instrument, Tsinghua University, Beijing, China · University of Rochester, Rochester, NY, USA · Beijing Information Science and Technology University, Beijing, China
Interactive World Simulator for Robot Policy Training and Evaluation
Mar 9, 2026Yixuan Wang, Rhythm Syed, Fangyu Wu +7Interactive World Models World Model Learning
Columbia University · Toyota Research Institute · Amazon +1
Brain-WM: Brain Glioblastoma World Model
Mar 8, 2026Chenhui Wang, Boyun Zheng, Liuxin Bao +4Mixture-of-Transformers Clinical Prediction
Institute of Science and Technology for Brain-inspired Intelligence, Fudan University, Shanghai 200433, China · Department of Electronic Engineering, The Chinese University of Hong Kong, Hong Kong SAR, China · School of Automation, Hangzhou Dianzi University, Hangzhou 310018, China +1
DreamSAC: Learning Hamiltonian World Models via Symmetry Exploration
Mar 8, 2026Jinzhou Tang, Fan Feng, Minghao Fu +3World Model Learning World Models
UC San Diego · Sun Yat-sen University
Kinematics-Aware Latent World Models for Data-Efficient Autonomous Driving
Mar 7, 2026Jiazhuo Li, Linjiang Cao, Qi Liu +1World Models for Autonomous Driving World Model Learning
Key Laboratory of Road and Traffic Engineering, Ministry of Education, Tongji University, Shanghai, China
Dreamer-CDP: Improving Reconstruction-free World Models Via Continuous Deterministic Representation Prediction
Mar 7, 2026Michael Hauri, Friedemann ZenkeWorld Model Learning World Models
Friedrich Miescher Instiute for Biomedical Research, Fabrikstrasse 24, 4056 Basel, Switzerland
Self-Supervised Multi-Modal World Model with 4D Space-Time Embedding
Mar 7, 2026Lance Legel, Qin Huang, Brandon Voelker +9World Model Learning Geospatial Representation Learning
Ecological Intelligence Lab Ecodash.ai · School of Complex Adaptive Systems Arizona State University · Geosensing Systems Engineering & Sciences Lab University of Houston +6
WorldCache: Accelerating World Models for Free via Heterogeneous Token Caching
Mar 6, 2026Weilun Feng, Guoxin Fan, Haotong Qin +10Diffusion Model Inference Acceleration Diffusion Model Caching
State Key Laboratory of AI Safety, Institute of Computing Technology, Chinese Academy of Sciences · University of Chinese Academy of Sciences · ETH Zürich +4

Apr 15, 2026J/K move · Enter open · S save

Beyond State Consistency: Behavior Consistency in Text-Based World Models

Youling Huang, Guanqiao Chen, Junchi Yao, Lu Wang+7

Dalian University of Technology · Microsoft · MBZUAI+1

World Models World Model Learning World Model Evaluation Model-Based Planning

Open paper AskPDF arXiv

World models have been emerging as critical components for assessing the consequences of actions generated by interactive agents in online planning and offline evaluation. In text-based environments, world models are typically evaluated and trained with single-step metrics such as Exact Match, aiming to improve the similarity between predicted and real-world states, but such metrics have been shown to be insufficient for capturing actual agent behavior. To address this issue, we introduce a new behavior-aligned training paradigm aimed at improving the functional consistency between the world model and the real environment. This paradigm focuses on optimizing a tractable step-level metric named Behavior Consistency Reward (BehR), which measures how much the likelihood of a logged next action changes between the real state and the world-model-predicted state under a frozen Reference Agent. Experiments on WebShop and TextWorld show that BehR-based training improves long-term alignment in several settings, with the clearest gains in WebShop and less movement in near-ceiling regimes, while preserving or improving single-step prediction quality in three of four settings. World models trained with BehR also achieve lower false positives in offline surrogate evaluation and show modest but encouraging gains in inference-time lookahead planning.