Latent World Models

Follow this topic Ask about this topic

Latent world models encode observations and histories into compact states, then predict how those states change over time or under actions, avoiding the need to reconstruct every pixel. Common approaches include recurrent state-space models such as Dreamer, joint-embedding predictive architectures (JEPAs), and diffusion or flow predictors, often shaped by inverse dynamics, temporal objectives, physical structure, or causal supervision. These models support imagined rollouts and planning in visual reinforcement learning, robot manipulation and navigation, autonomous driving, and other forecasting settings. A central challenge is that accurate short-term predictions or interpretable latents do not necessarily preserve action effects and goal-relevant structure over long horizons, leaving planning evaluation, uncertainty calibration, and robust transfer open.

Latest papers 198

50 papers in the last four weeks, up 194% on the four weeks before. 0.3% of all new papers.

All topics

When Do Conservation Laws Survive Learned Representations? Certified Horizons for Latent World Models
Jun 23, 2026Hongbo WangDynamical Systems Latent World Models
Department of Mathematics, Stony Brook University, Stony Brook, NY 11794, USA
Beyond the Next Step: Variable-Length Latent World Models for Long-Horizon Planning
Jun 19, 2026Tianqi Du, Qi Zhang, Yifei Wang +1Latent Dynamics Modeling World Models
State Key Lab of General Artificial Intelligence, School of Intelligence Science and Technology, Peking University · Amazon AGI SF Lab · Institute for Artificial Intelligence, Peking University
Imitation from Heterogeneous Demonstrations using Grounded Latent-Action World Models
Jun 19, 2026Tianyou Wang, Anson Lei, Joe Watson +1Latent Action Learning Visuomotor Policy Learning
University of Oxford
Sensorimotor World Models: Perception for Action via Inverse Dynamics
Jun 18, 2026Petr Ivashkov, Randall Balestriero, Bernhard SchölkopfRepresentation Learning World Model-Based Planning
Max Planck Institute for Intelligent Systems, Tübingen, Germany · Department of Computer Science, Brown University, Providence, RI, USA · ELLIS Institute, Tübingen, Germany +1
Looped World Models
Jun 16, 2026Hongyuan Adam Lu, Z. L. Victor Wei, Qun Zhang +28World Model Learning Recurrent Transformers
FaceMind Research Asia
NarrativeWorldBench: A Frontier-Saturated Benchmark and a Latent World Model for Long-Horizon Co-Creative Audio Drama
Jun 16, 2026Logan Mann, Abdur Rahman, Mohammad Saifullah +2LLM World Models Multilingual Language Model Evaluation
University of California, Santa Barbara · Pocket FM
BRICKS-WM: Building Reusability via Interface Composition Kinetics for Structured World Models
Jun 15, 2026Shaowei Zhang, Jiahan Cao, Xunlan Zhou +2World Models Latent World Models
1National Key Laboratory for Novel Software Technology, Nanjing University, China · School of Artificial Intelligence, Nanjing University, China · School of Intelligence Science and Technology, Nanjing University, China
LaWAM: Latent World Action Models for Efficient Dynamics-Aware Robot Policies
Jun 14, 2026Jialei Chen, Kai Wang, Kang Chen +9Latent Action Learning Action-Conditioned World Models
Jilin University · Zhongguancun Academy · Nankai University +4
Pixels to Proofs: Probabilistically-Safe Latent World Model Control via Parallel Conformal Robust MPC
Jun 14, 2026Devesh Nath, Anutam Srinivasan, Haoran Yin +3Constrained Motion Planning World Model-Based Planning
Georgia Institute of Technology
DLWM: Diverse Latent World Models for Efficient Multimodal Reasoning
Jun 13, 2026David Huang, Lianlei ShanEfficient Multimodal Inference Multimodal Reasoning
University of Toronto · Tsinghua University
FlowMo-WM: A World Model with Object Momentum and Hidden Ambient Drift
Jun 11, 2026Yitao Jiang, Luyang Zhao, Muhao Chen +1World Model Learning World Models for Robotics
Department of Computer Science Dartmouth College, Hanover, NH · Department of Electrical and Computer Engineering Clemson University, Clemson, SC · Department of Mechanical and Aerospace Engineering University of Houston, Houston, TX
Certified World Models: Predictability Across Configuration, Horizon, and Resolution
Jun 11, 2026Hongbo WangEquivariant Neural Networks Latent World Models
Department of Mathematics, Stony Brook University, Stony Brook, NY 11794, USA
ReflectiChain: Epistemic Grounding in LLM-Driven World Models for Supply Chain Resilience
Jun 9, 2026Jia LuoConstrained RL LLM-Guided RL
School of Foreign Languages, Huazhong University of Science and Technology, Wuhan 430074, China.
Toward Compiler World Models: Learning Latent Dynamics for Efficient Tensor Program Search
Jun 8, 2026Haolin Pan, Lianghong Huang, Xvlin Zhou +2GPU Kernel Optimization Latent World Models
Hangzhou Institute for Advanced Study, University of Chinese Academy of Sciences, Hangzhou, China · Institute of Software, Chinese Academy of Sciences, Beijing, China · University of Chinese Academy of Sciences, Beijing, China
FF-JEPA: Long-Horizon Planning in World Models with Latent Planners
Jun 8, 2026Sergi Masip, Jonathan Swinnen, Yutong Hu +2Hierarchical Planning Long-Horizon Robotic Manipulation
PSI, ESAT, KU Leuven · RAM, MECH, KU Leuven
ATM: Why Latent World Models Can Fail to Plan
Jun 8, 2026Jiaheng Chen, Tinghe Zhang, Yucheng Xiao +5World Model-Based Planning Model-Based Planning
School of Software, Northeastern University
PRISM: PRior-guided Imagination Sampling in world Models
Jun 6, 2026Yuhai Wang, Jiawei Xia, Rongxuan Zhou +4World Models for Robotics World Model-Based Planning
1Northeastern University · University of California, Berkeley · 3Qiyuan Lab +1
PLAN-S: Bridging Planning with Latent Style Dynamics for Autonomous Driving World Models
Jun 4, 2026Xiaoyun Qiu, Jingtao He, Yijie Chen +4Roadmap-Based Motion Planning World Model-Based Planning
Intelligent Transportation Thrust, Systems Hub, and Center of Seamless Connectivity & Connected Intelligence, The Hong Kong University of Science and Technology (Guangzhou), Guangzhou 511453, China.
Autoregressive Diffusion World Models for Off-Policy Evaluation of LLM Agents
Jun 4, 2026Kaixuan Liu, Guojun Xiong, Weinan Zhang +1World Model Learning Off-Policy Evaluation
Department of Computer Science Emory University Atlanta, GA, USA · School of Computer Science Shanghai Jiao Tong University Shanghai, China
Generalization of World Models under Environmental Variability for Vision-based Quadrotor Navigation
Jun 3, 2026Luca Zanatta, Grzegorz Malczyk, Kostas AlexisWorld Models for Robotics World Models
Norwegian University of Science and Technology
CLAW: Learning Continuous Latent Action World Models via Adversarial Latent Regularization
Jun 2, 2026Tewodros Ayalew, Matthew Jeung, Samuel Wheeler +5Latent Action Learning Imitation Learning
University of Chicago · 3Argonne National Laboratory · 2Toyota Technological Institute at Chicago
Exact equivariance, kept through training, buys zero-shot generalisation across the symmetry group
Jun 2, 2026Hongbo WangWorld Models for Robotics World Models
Department of Mathematics, Stony Brook University, Stony Brook, NY 11794, USA
IMWM: Intuition Models Complement World Models for Latent Planning
Jun 1, 2026Baoqi Gao, Ruize Han, Miao Wang +1World Model-Based Planning AI Control
Beihang University · Shenzhen University of Advanced Technology
BRo-JEPA: Learning Modular Arithmetic in Latent Space
May 31, 2026Divyansh Jha, Yuanfang Xie, Varan Mehra +1Zero-Shot Learning Joint-Embedding Predictive Architecture
1Georgia Institute of Technology · 2NYU Langone Health
Dreaming Of Others: Latent Teammate Modeling In World Models For Multi-Agent Reinforcement Learning
May 29, 2026Tomas Leroy-StoneMulti-Agent Coordination Theory of Mind
Subspace-Decomposed JEPAs: Disentangling Progression and Content in Latent World Models
May 29, 2026Lucas Thil, Jesse Read, Rim Kaddah +1Disentangled Representation Learning Representation Learning
LIX, École Polytechnique Palaiseau, France · IRT SystemX Palaiseau, France · Safran Tech Chateaufort, France
World Models: A Comprehensive Survey of Architectures, Methodologies, Reasoning Paradigms, and Applications
May 28, 2026Arif Hassan Zidan, Yi Pan, Hanqi Jiang +23World Model Learning World Model-Based Planning
School of Computer and Cyber Sciences, Augusta University, Augusta, GA, USA · School of Computing, University of Georgia, Athens, GA, USA · Tandon School of Engineering, New York University, Brooklyn, NY, USA +9
Latent Goal Prediction from Language for Model-Based Planning
May 26, 2026Samuel Barbeau, Simon Roy, Giovanni Beltrame +2World Model-Based Planning Long-Horizon Planning
1École de Technologie Supérieure, Montréal · 2International Laboratory on Learning Systems (ILLS) · 3Polytechnique Montréal +4
Back to Parsimonious Latents: Learning Task-Centric World Models from Visual Foundations
May 25, 2026Minghao Fu, Fan Feng, Nicklas Hansen +1World Model Learning World Models
University of California, San Diego
SliceWorld: A Predictive and Controllable World-State Model for CT Report Generation
May 23, 2026Yuanhe Tian, Yan SongRadiology Report Generation World Model Learning
Zhongguancun Academy · University of Science and Technology of China

Jun 23, 2026J/K move · Enter open · S save

When Do Conservation Laws Survive Learned Representations? Certified Horizons for Latent World Models

Hongbo Wang

Department of Mathematics, Stony Brook University, Stony Brook, NY 11794, USA

Dynamical Systems Latent World Models Certified Robustness Hamiltonian Dynamics

Open paper AskPDF arXiv

We ask a representation-learning question about physical world models: when does a conservation law remain certifiable after a model learns a latent representation? A certified horizon bounds -- in advance, from measurable model defects -- how many steps a rollout provably stays on a physical invariant's level set. The key design choice is what is certified: not a learned latent Hamiltonian or a learned scalar witness (a model can conserve either while drifting in true energy), but the decoded physical invariant obtained by decoding the latent state and evaluating the known invariant. Around this object we derive shell-horizon certificates whose budget decomposes into representation, readout, and latent-dynamics defects, with a monotone alignment bridge through which a soft learned witness yields a certified horizon for the decoded invariant, and test them across state, learned-lift, and pixel observations on conservative systems. Conservation certificates can survive learned representation, but not all geometric priors survive equally. Hard canonical symplectic structure yields the longest horizons in known phase coordinates yet does not cross a learned chart, whereas a controlled-Lipschitz-aligned soft invariant survives in the nonlinear learned-representation settings we test -- two lift systems, with the gain growing with nonlinearity, and pixels. Pixel certification is recovered on a readout-stable sub-tube, and the Kepler problem exposes a geometric boundary. The central object is therefore not a latent Hamiltonian, but a decoded physical invariant whose robustness to representation learning can be measured, certified, and falsified.