World Model Learning

Follow this topic Ask about this topic

World model learning studies how agents can represent a changing environment and predict what may happen next, especially after an action. Common approaches include latent-state models such as RSSMs and JEPAs, video diffusion models, and structured representations based on objects, graphs, geometry, physics, or executable rules. These models are used for imagined rollouts and planning in settings such as robot manipulation, autonomous driving, games, and web or GUI agents, as well as for scientific forecasting and simulation. A central challenge is that strong one-step or visual predictions do not guarantee useful decisions: long-horizon consistency, action accuracy, uncertainty, and transfer to unfamiliar settings remain difficult to balance with computational cost.

Latest papers

419 papers88 papers in the last four weeks, up 83% on the four weeks before. 0.5% of all new papers.

world2vec: Mechanistic world models through latent dynamics learning
Oct 29, 2026Tobias Schmidt, Rodrigo Gonzalez Laiz, Krisztián Szuppinger +1World Model Learning World Models
Helmholtz Munich & Munich Center for Machine Learning (MCML)
Structural-Test Geometry as an Inductive Bias for Partially Observable World Models
Oct 28, 2026Todd Y. ZhouWorld Model Learning Latent World Models
Harvard University (Undergraduate Student), Cambridge, MA, USA
PulmoWorld: An Asynchronous Multimodal Medical World Model for Longitudinal Pneumonia Dynamics
Oct 28, 2026Cheng Tang, Xinyu Zhang, Jiawei Chen +14Disease Progression Modeling World Model Learning
Shanghai Artificial Intelligence Laboratory · University of the Chinese Academy of Sciences · Fudan University +7
Self-Supervised Latent Action-Conditioned World Modeling for Time Series
Oct 27, 2026Chia-Han Kuo, Chengshuai Zhao, Huan Liu +1Self-Supervised Time Series Representation Learning Action-Conditioned World Models
Arizona State University
Trace-WAM: Learning to Act via 3D Motion Prediction for World Action Models
Oct 27, 2026Xu Hu, Changjie Wu, Linzhe Shi +8World Model Learning Robotic Manipulation
The Hong Kong Polytechnic University · Amap, Alibaba Group · Tsinghua University +2
LabWorld: Learning Interactive 4D Worlds for Scientific Experimentation
Oct 27, 2026Suozhi Huang, Kelsey Fu, Binxu Li +124D Reconstruction Indoor Scene Reconstruction
Princeton University · Stanford University · University of Chicago +4
DriveUniTok: A Unified Multimodal Tokenizer for Driving World-Action Models
Oct 26, 2026Jingbo Xu, Hanchao Leng, JiaHui Huang +7World Models for Autonomous Driving World Model Learning
Tsinghua University · Xiaomi EV
Video World Models Need Predictable Latents
Oct 26, 2026Mattia Scardecchia, Sherry YangVideo World Models World Model Learning
New York University
How Humans Learn Unfamiliar Worlds: A Think-Aloud Dataset on ARC-AGI-3
Oct 25, 2026Shuzhi Liu, Archana Warrier, Dat Nguyen +3World Model Learning Human Behavior Modeling
Nanyang Technological University · TU Darmstadt · Basis Research Institute & Harvard University +1
TurboWAM: Learning What Matters for Fast World Action Models
Oct 24, 2026Guile Wu, Shuangzhi Li, Xingxin Chen +5Efficient Inference for World Action Models World Action Models
Huawei Noah’s Ark Lab · University of Alberta · Department of Foundation Model, Huawei
DRILL: Training World Models to Improve Policies, Not Predict Pixels
Oct 22, 2026Zhaolu Kang, Tailong Luo, Lei Wei +9World Models for VLA Models World Model Learning
Hunyuan, Tencent · Peking University
An Introduction to World Models for Generalist Agents
Oct 22, 2026Zhihui ZhuWorld Models World Model Learning
Department of Computer Science and Engineering The Ohio State University
ALPS: Visual State Encoders for World Models
Oct 21, 2026Yanqing Liu, Letian Zhang, Chengzhi Liu +5World Model Learning Vision-Language Pretraining
University of California, Santa Cruz · University of California, Santa Barbara · Google
How Representational Priors Shape Generalization in World Action Models
Oct 21, 2026Junha Chun, Youngjoon Jeong, Taesup KimWorld Action Models World Model Learning
Department of Electrical and Computer Engineering, Seoul National University · Graduate School of Data Science, Seoul National University
SuperMimic: Scaling Humanoid Whole-Body Control with One Differentiable World Model
Oct 20, 2026Qiayuan Liao, Yiyang Shao, Junfeng Long +4Humanoid Robot Control Humanoid Motion Tracking
UC Berkeley
World Models with Configurable Perception
Oct 20, 2026Yijun Shen, Delong Chen, Xianming Hu +2World Model Learning World Model-Based Planning
East China Normal University · HKUST
LaLa-World: Scaling Latent World Models with Natural Language Actions
Oct 20, 2026Junha Song, Geonmo Gu, Jeongeun Park +5World Model Learning Latent World Models
NAVER AI Lab · Georgia Institute of Technology · Brown University +1
VPFlow: Value-Predictive Planning with Imperfect World Models for Offline Goal Reaching
Oct 19, 2026Yunzhe Hu, Dong XuOffline RL Model-Based Planning
School of Computing and Data Science, The University of Hong Kong
Train An Imaginative Agent
Oct 19, 2026Jiaxin Ge, Junyi Zhang, Tianning Chai +9LLM Agent Planning Model-Based Planning
University of California, Berkeley · Impossible Research · University of Washington +1
PICA-WAM: Visual Perturbation Invariance via Cross-Appearance Self-Alignment in World Action Models
Oct 19, 2026Yude Zou, Yifei Yao, Guanbo Jiao +3World Action Models World Model Learning
Shanghai Jiao Tong University · Shanghai Artificial Intelligence Laboratory · Fudan University
Simulation Priors for Rapid Adaptation in Model-Based RL for Autonomous Racing
Oct 19, 2026Adrian Remonda, Jiajun Xi, Nicklas Hansen +5RL for Autonomous Driving Model-Based RL
Toyota Research Institute, Los Altos, CA, USA. · University of California San Diego (UCSD), La Jolla, CA, USA.
PrismWAM: Decoupling State Representations in World Action Models
Oct 19, 2026Tianyu Xin, Wanqi Zhong, Bowen Hu +8World Models for Robotics World Models
Tsinghua University · NVIDIA
See the Future, Classify the Present: Stochastic Latent World Modeling for Hyperspectral Images
Oct 19, 2026Muhammad Ahmad, Gemine Vivone, Ronny Hänsch +3Hyperspectral Image Classification Predictive Representation Learning
Khanpur Institute of Technology, Khanpur 64100, Pakistan · National Research Council - Institute of Methodologies for Environmental Analysis (CNR-IMAA), 85050 Tito, Italy · Department of SAR Technology, German Aerospace Center (DLR), Germany +2
Frechet Forcing: Training Long-Horizon Streaming World Models from Short Clips
Oct 19, 2026Yian Zhao, Yu Yan, Hongcan Guo +3World Model Learning Video World Models
Vector-Quantized World Model: Latent Feedback Motion Planning for Robot Manipulation
Oct 19, 2026Xuezhou Xu, Chung-Pang Wang, Zhenzhen Li +1Long-Horizon Robotic Manipulation World Model-Based Planning
New York University, New York, NY, USA. · NVIDIA.
CoRe-WM: Conditioned Response-State World Modeling for Longitudinal Treatment-Response Prediction in Breast DCE-MRI
Oct 18, 2026Lin Zhao, Bowen Chen, Qiaohui Gao +4Cancer Drug Response Prediction Clinical Outcome Prediction
Department of Biomedical Engineering, New Jersey Institute of Technology, Newark, NJ 07102, USA. · College of Engineering, Northeastern University, Boston, MA 02115, USA. · Department of Radiology, Massachusetts General Hospital, Boston, MA 02114, USA. +2
Beckmann World Models: Direct Terminal-Map Learning for One-Step Action-Conditioned Prediction
Oct 16, 2026Adam Lee, Shobhit Agarwal, June Pyo SuhWorld Model Learning Action-Conditioned World Models
UC Berkeley · Stanford University · Independent Researcher
LithoWorld: An Endpoint World Model for Lithography Process-Window Recovery
Oct 16, 2026Botong Zhao, Fang Yu, Xinyuan Chen +1Semiconductor Manufacturing World Model Learning
Shanghai Key Laboratory of Multidimensional Information Processing, East China Normal University, Shanghai, China · Shanghai Artificial Intelligence Laboratory, Shanghai, China
ORD-WM: Learning Object-Centric Rigid Dynamics for 3D World Modeling in Robotic Manipulation
Oct 16, 2026Yueyang Weng, Xiaopeng Zhang, Yuxi Ou +3World Model Learning Object-Centric World Models
Harbin Institute of Technology, Shenzhen
GeoState: Learning View-Decoupled Geometry States for 3D World Modeling
Oct 15, 2026Hongchang Liu, Jian Liu, Yujuan Ding +5World Model Learning 3D Scene Prediction
Tongji University · Hong Kong Polytechnic University · Tencent +1

Oct 29, 2026J/K move · Enter open · S save

world2vec: Mechanistic world models through latent dynamics learning

Tobias Schmidt, Rodrigo Gonzalez Laiz, Krisztián Szuppinger, Steffen Schneider

Helmholtz Munich & Munich Center for Machine Learning (MCML)

World Model Learning World Models Latent Dynamics Modeling System Identification

Open paper AskPDF arXiv

World models can achieve strong control performance without faithfully capturing the dynamics of their environment, limiting their potential for mechanistic interpretability and scientific discovery. Here, we introduce world2vec to learn mechanistic world models without reward information directly from observations and actions. World2vec learns latent dynamics that faithfully reproduce the true system over long prediction horizons, with at least one order of magnitude fewer parameters than DreamerV3, TD-MPC2 and LeWM. This enables state of the art performance in controlling continuous environments from pixels and accurate system identification across a benchmark of 71 attractors, demonstrating how world models are amenable to scientific discovery.