Latent World Models

Follow this topic Ask about this topic

Latent world models encode observations and histories into compact states, then predict how those states change over time or under actions, avoiding the need to reconstruct every pixel. Common approaches include recurrent state-space models such as Dreamer, joint-embedding predictive architectures (JEPAs), and diffusion or flow predictors, often shaped by inverse dynamics, temporal objectives, physical structure, or causal supervision. These models support imagined rollouts and planning in visual reinforcement learning, robot manipulation and navigation, autonomous driving, and other forecasting settings. A central challenge is that accurate short-term predictions or interpretable latents do not necessarily preserve action effects and goal-relevant structure over long horizons, leaving planning evaluation, uncertainty calibration, and robust transfer open.

Latest papers 198

50 papers in the last four weeks, up 194% on the four weeks before. 0.3% of all new papers.

All topics

QQWorld: Quantile-Quantile Matching for World Model Regularization
Jul 30, 2026Zhoushun Yu, Xiaoyu Hu, Xiangyu XuWorld Model Learning World Model-Based Planning
Xi’an Jiaotong University
ODEWorld: A Continuous Predictive Architecture via Physical-Time Flow
Jul 30, 2026Dongxiu Liu, Haoyi Niu, Peng Cheng +5Robotic Control Latent Dynamics Modeling
Institute for AI Industry Research (AIR), Tsinghua University, Beijing, China · 2Berkeley Artificial Intelligence Research (BAIR), University of California, Berkeley, CA, USA
What Can Latent World Models Know? Physical Parameter Identifiability in Multimodal Predictive Representations
Jul 29, 2026Kaizhen Tan, Xin Xu, Siru Tao +4Parameter Identifiability Predictive Representation Learning
New York University, New York, NY, USA · Carnegie Mellon University, Pittsburgh, PA, USA · Columbia University, New York, NY, USA
Temporal-Distance JEPA: Plan-Aware Representation Learning for Latent World Model Predictive Control
Jul 28, 2026Jiaxin Bai, Jiaxuan XiongWorld Model-Based Planning Joint-Embedding Predictive Architecture
Hong Kong Baptist University Hong Kong, China
LeapBot-WA: World-Anchor Action Models via Predictive Latent Alignments
Jul 27, 2026Pei Liu, Nan Zheng, Lang Zhang +8Robot Policy Learning World Models for Robotics
1The Hong Kong University of Science and Technology (Guangzhou) · 2The Hong Kong University of Science and Technology · 3Leapmotor +1
Dreamer-CPC: Message Learning with World Models for Decentralized Multi-agent Reinforcement Learning
Jul 22, 2026Taisuke Takayama, Naoto Yoshida, Tadahiro TaniguchiPredictive Coding Partially Observable RL
Graduate School of Informatics, Kyoto University, Kyoto, Japan · Research Organization of Science and Technology, Ritsumeikan University, Shiga, Japan
Koopman Dreamer: Spectrally Constrained Latent Dynamics for Stable World-Model Imagination
Jul 22, 2026Jiaqi Li, Xinglong Zhang, Haibin Xie +3Koopman Operator Learning Latent Dynamics Modeling
College of Intelligence Science and Technology, National University of Defense Technology, Changsha, China
Multi-Horizon Consistency as Geometry: When Latent Dynamics Contract, and When They Do Not
Jul 21, 2026Kavya Bhand, Aadi JoshiLatent Dynamics Modeling Latent World Models
Department of Computer Engineering Vishwakarma Institute of Technology, Pune, India
DWM: Separating World Effects from Actions in Latent World Models
Jul 21, 2026Yi-Ge Zhang, Tianqi Du, Qi Zhang +1World Model Learning Latent Dynamics Modeling
Peking University
SAGE: Subgoal-Conditioned Action Generation for Latent World Model Planning
Jul 20, 2026Letian Cheng, Qi Zhang, Yisen WangWorld Model-Based Planning Action-Conditioned World Models
Peking University
From Surface Forecasting to Observability Forecasting: A Latent World Model for Cloud-Aware EO Monitoring
Jul 15, 2026Mohanad AlbughdadiWorld Model Learning Remote Sensing
European Centre for Medium-Range Weather Forecasts
Mind the Gap: Promises and Pitfalls of Hierarchical Planning in LeWorldModel
Jul 14, 2026Niccolò Caselli, Francesco Massafra, Samuele Punzo +3Hierarchical Planning Long-Horizon Robotic Manipulation
University of Amsterdam
A Control Theory of Predictability in Latent World Models
Jul 11, 2026Hanzhe You, Yonggang Zhang, Maohao Ran +6Model Predictive Control Latent World Models
University of Science and Technology of China · The Hong Kong University of Science and Technology, HKGAI · Hong Kong Baptist University
Does Depth Survive Composition, and Can a Router Use It? Adaptive Compute in Latent World Models
Jul 11, 2026Achyuthan SivasankarLatent World Models World Model Evaluation
New York University
The Rank-One Corner: How Much Value Equivalence Does a Task Need from a World Model?
Jul 7, 2026Donna VakalisWorld Model Learning Predictive Representation Learning
Mila – Quebec AI Institute Université de Montréal Montréal, Québec
Multiplayer Interactive World Models with Representation Autoencoders
Jul 6, 2026Anthony Hu, Václav Volhejn, Adrien Ramanana Rahary +24Action-Conditioned World Models Latent World Models
1General Intuition · 2Kyutai · ‡École nationale des ponts et chaussées +1
InternVLA-A1.5: Unifying Understanding, Latent Foresight, and Action for Compositional Generalization
Jul 6, 2026Haoxiang Ma, Junhao Cai, Xiaoxu Xu +26Generalization in Robotic Manipulation Visuomotor Policy Learning
Physical Intelligence Team, Shanghai AI Laboratory
Qantara: Bridge-Flow Training for Multi-Paradigm JEPA Inference
Jul 6, 2026Ruslan Rakhimov, George Bredis, Yuriy Maksyuta +1Joint-Embedding Predictive Architecture Feedback Control
T-Tech
Operator-on-F complements value-equivalence: a planning-time diagnostic for latent world models
Jul 5, 2026Donna VakalisWorld Model-Based Planning Latent World Models
1Mila – Quebec AI Institute · 2Université de Montréal
Reward Observability and the Limits of Offline Checkpoint Selection in RSSM World Models
Jul 2, 2026Nikolai Smolyanskiy, Jonathan ShockReinforcement Learning Latent World Models
Valdi: Value Diffusion World Models
Jul 1, 2026Christopher Lindenberg, Kashyap ChittaWorld Model Learning Latent Dynamics Modeling
University of Tübingen, Germany · 2KE:SAI – Kyutai ELLIS Scalable Autonomous Intelligence, Germany
AdaJEPA: An Adaptive Latent World Model
Jun 30, 2026Ying Wang, Oumayma Bounou, Yann LeCun +1Continual Test-Time Adaptation World Model-Based Planning
New York University · AMI Labs
Delta-JEPA: Learning Action-Sensitive World Models via Latent Difference Decoding
Jun 30, 2026Zhenghao Zhang, Yuanxiang Wang, Zhenyu Guan +11World Models World Model-Based Planning
School of Computer Science and Technology, University of Chinese Academy of Sciences, Beijing · Institute of Information Engineering, Chinese Academy of Sciences, Beijing · School of Computer Science and Technology, Harbin Institute of Technology, Weihai +2
Orca: The World is in Your Mind
Jun 29, 2026Yihao Wang, Yuheng Ji, Mingyu Cao +54World Model Learning Multimodal Pretraining
From Tokens to States: LLMs as a Special Case of World Models and the Continuous Path Beyond
Jun 26, 2026Paul DuboisLLM World Models World Models
Event-Conditioned Diagnostics of Kinematic, Contact, and Object-Permanence Fields in Passive Object-State World Models
Jun 26, 2026Yang Liu, Yuming ChenLatent Dynamics Modeling Latent World Models
College of Intelligent Robitcs and Advanced Manufacturing, Fudan University · Shanghai, China
Textual Belief States for World Models: Identifiable Representation Learning Under Strict Mediation
Jun 26, 2026Xiang Gao, Kaiwen Dong, Yuguang Yao +2Representation Learning Group Relative Policy Optimization
Intuit AI Research
Internalizing the Future: A Unified Agentic Training Paradigm for World Model Planning
Jun 25, 2026Xuan Zhang, Zhijian Zhou, Lingfeng Qiao +6World Model-Based Planning Model-Based Planning
Fudan University · Shanghai Innovation Institute · Tencent Youtu Lab
Fast LeWorldModel
Jun 24, 2026Yuntian Gao, Xiangyu XuLatent Dynamics Modeling World Model-Based Planning
Xi’an Jiaotong University
Conformal Orbit-Valid Trust Horizons for Equivariant World Models
Jun 23, 2026Hongbo WangEquivariant Neural Networks Conformal Prediction
Department of Mathematics, Stony Brook University, Stony Brook, NY 11794, USA

Jul 30, 2026J/K move · Enter open · S save

QQWorld: Quantile-Quantile Matching for World Model Regularization

Zhoushun Yu, Xiaoyu Hu, Xiangyu Xu

Xi’an Jiaotong University

World Model Learning World Model-Based Planning Distribution Matching Latent World Models Model-Based RL

Open paper AskPDF arXiv

Latent world models enable efficient planning by predicting future states in a compact representation space, but their performance depends critically on the quality of the learned latent distribution. LeWorldModel (LeWM) regularizes its latents toward an isotropic Gaussian using the Epps-Pulley (EP) objective. We show that the corrective gradients of EP rapidly vanish for isolated tail samples, leaving heavy-tailed deviations insufficiently controlled. To address this limitation, we propose QQWorld, which replaces EP with a quantile-quantile matching objective that directly aligns projected latent samples with rank-matched Gaussian quantiles, thereby maintaining effective corrective gradients in the tails. We further develop cross-batch QQ, which enlarges the effective ranking pool using detached samples from previous batches, and characterize its bias-variance trade-off. Across four control environments, QQWorld effectively improves the average planning success rate of LeWM, while consistently yielding better Gaussian alignment and thinner latent tails.