Latent World Models

Follow this topic Ask about this topic

Latest papers 189

36 papers in the last four weeks, up 38% on the four weeks before. 0.2% of all new papers.

All topics

LeWorldModel: Stable End-to-End Joint-Embedding Predictive Architecture from Pixels
Mar 13, 2026Lucas Maes, Quentin Le Lidec, Damien Scieur +2World Model Learning Joint-Embedding Predictive Architecture
World Model for Battery Degradation Prediction Under Non-Stationary Aging
Mar 11, 2026Kai Chin Lim, Khay Wai SeeLithium-Ion Batteries RUL Estimation
Probabilistic Dreaming for World Models
Mar 5, 2026Gavin WongWorld Model Learning World Models
Beyond Pixel Histories: World Models with Persistent 3D State
Mar 3, 2026Samuel Garcin, Thomas Walker, Steven McDonagh +5World Models Interactive World Models
University of Edinburgh · Microsoft Research
FRAPPE: Infusing World Modeling into Generalist Policies via Multiple Future Representation Alignment
Feb 19, 2026Han Zhao, Jingbo Wang, Wenxuan Song +5World Models for Robotics Latent World Models
Zhejiang University · Westlake University · HKUST (GZ) +3
Factored Latent Action World Models
Feb 18, 2026Zizhao Wang, Chang Shi, Jiaheng Hu +4Latent World Models Latent Action Learning
University of Texas at Austin · Sony AI
CLARITY: Medical World Model for Guiding Treatment Decisions by Modeling Context-Aware Disease Trajectories in Latent Space
Dec 8, 2025Tianxingjian Ding, Yuanhao Zou, Chen Chen +2Healthcare Clinical Decision Support
Institute of Artificial Intelligence, University of Central Florida
Geometry-Centered 3D Latent World Models for Growing Surfaces
May 29, 2025Xiaoyi Liu, Hao TangWorld Model Learning Latent World Models
Department of Computer Science Brown University · School of Computer Science Peking University
Mitigating Covariate Shift in Imitation Learning for Autonomous Vehicles Using Latent Space Generative World Models
Sep 25, 2024Alexander Popov, Alperen Degirmenci, David Wehr +9End-to-End Autonomous Driving Imitation Learning
NVIDIA

Mar 13, 2026J/K move · Enter open · S save

LeWorldModel: Stable End-to-End Joint-Embedding Predictive Architecture from Pixels

Lucas Maes, Quentin Le Lidec, Damien Scieur, Yann LeCun+1

World Model Learning Joint-Embedding Predictive Architecture Latent World Models

Open paper AskPDF arXiv

Joint Embedding Predictive Architectures (JEPAs) offer a compelling framework for learning world models in compact latent spaces, yet existing methods remain fragile, relying on complex multi-term losses, exponential moving averages, pre-trained encoders, or auxiliary supervision to avoid representation collapse. In this work, we introduce LeWorldModel (LeWM), the first JEPA that trains stably end-to-end from raw pixels using only two loss terms: a next-embedding prediction loss and a regularizer enforcing Gaussian-distributed latent embeddings. This reduces tunable loss hyperparameters from six to one compared to the only existing end-to-end alternative. With ~15M parameters trainable on a single GPU in a few hours, LeWM plans up to 48x faster than foundation-model-based world models while remaining competitive across diverse 2D and 3D control tasks. Beyond control, we show that LeWM's latent space encodes meaningful physical structure through probing of physical quantities. Surprise evaluation confirms that the model reliably detects physically implausible events.