Latent Dynamics Modeling

Follow this topic Ask about this topic

Latent dynamics modeling studies how to represent a changing system with hidden states and learn how those states evolve from observations such as video, sensor streams, or sparse measurements. Common approaches include autoencoders with state-space models, neural ODEs and SDEs for continuous-time or stochastic change, Koopman operators for approximately linear evolution, and JEPA-style world models for action-conditioned prediction. These models support forecasting and control in robotics and physical simulation, as well as analysis of biological, clinical, and neural time series. A central challenge is that good one-step prediction or reconstruction does not guarantee reliable long-horizon rollouts, useful control, or recovery of meaningful physical mechanisms; work therefore explores stability, identifiability, and physics-based structure while balancing these constraints against model flexibility.

Latest papers

254 papers48 papers in the last four weeks, up 92% on the four weeks before. 0.3% of all new papers.

Understanding the limits of latent world models
Oct 27, 2026Rodrigo Gonzalez Laiz, Steffen SchneiderWorld Models Latent World Models
Helmholtz Munich Munich Center for Machine Learning (MCML)
Data-driven Discovery of Lie Symmetries and First Integrals with Latent ODEs
Oct 25, 2026Matt L. Wiemman, Peter MelchiorSymmetry Discovery Lie Group Methods
Princeton University
Latent dynamics for reconstructing fine-scale PDE trajectories from coarse observations
Oct 24, 2026Chenhao Si, Zishan Qian, Shing-Tung Yau +2Latent Dynamics Modeling PDE Surrogate Modeling
School of Data Science, The Chinese University of Hong Kong, Shenzhen, Shenzhen, China. · Institute of Statistics and Big Data, Renmin University, Beijing, China. · Beijing Key Laboratory of Topological Statistics and Applications for Complex Systems, Beijing Institute of Mathematical Sciences and Applications, Beijing, China. +1
ZENO: Nonlinear filtering with Zakai Equation Neural Operators
Oct 23, 2026Aaditya Prasad, Steven W. Flavell, Brian DePasqualeLatent Dynamics Modeling Stochastic Differential Equations
Picower Institute for Learning & Memory, Department of Brain & Cognitive Sciences, Massachusetts Institute of Technology, Cambridge, MA 02139, USA · Howard Hughes Medical Institute, Picower Institute for Learning & Memory, Department of Brain & Cognitive Sciences, Massachusetts Institute of Technology, Cambridge, MA 02139, USA · Department of Biomedical Engineering, Boston University, Boston, MA 02215, USA
Predictive Latents for Autoregressive Language Models: Possibility, Gauge, and Obstruction
Oct 20, 2026Yuzhou HongAutoregressive Language Modeling Latent Dynamics Modeling
Zhejiang Sci-Tech University
Meta-iLaD: Identifiable Latent Dynamics via Meta-Learning of Dynamics Environments
Oct 19, 2026Yubo Ye, Sweekar Piya, Xiajun Jiang +1Latent Dynamics Modeling Time Series Forecasting
Department of Computing and Information Sciences, Rochester Institute of Technology, Rochester, New York, USA · Department of Computer Science, Rowan University, Glassboro, New Jersey, USA
Vision-Language-Delta-Action: Learning Compact Deltas as Actionable Transition Interfaces for Robotic Manipulation
Oct 19, 2026Lingsen Zhang, Wei Li, Jie He +5Robotic Manipulation Vision-Language-Action Models
Harbin Institute of Technology, Shenzhen · Baidu, Inc. · Shenzhen Loop Area Institute
See the Future, Classify the Present: Stochastic Latent World Modeling for Hyperspectral Images
Oct 19, 2026Muhammad Ahmad, Gemine Vivone, Ronny Hänsch +3Hyperspectral Image Classification Predictive Representation Learning
Khanpur Institute of Technology, Khanpur 64100, Pakistan · National Research Council - Institute of Methodologies for Environmental Analysis (CNR-IMAA), 85050 Tito, Italy · Department of SAR Technology, German Aerospace Center (DLR), Germany +2
Latent Projection for Latent Planning
Oct 19, 2026Yoo Sung Jang, Michael RyooLatent-Space Planning World Model-Based Planning
Stony Brook University
A discrete-continuous-time stochastic framework for attention-deficit/hyperactivity disorder-related symptom modeling and pharmacological intervention via methylphenidate
Oct 18, 2026Oana-Maria-Catalina CrisanMental Health Stochastic Optimal Control
Faculty of Mathematics and Computer Science, Babes,-Bolyai University, Str. Mihail Kog˘alniceanu, nr. 1, Cluj-Napoca, 400084, Romania.
Structured Latent Dynamics for Long-Horizon Embodied World Models
Oct 17, 2026Kevin Chen, Yu-Wei Zhan, Haoran Liang +2World Models for Robotics Embodied World Models
Tsinghua University
A low-dimensional dynamical basis for tongue shape control
Oct 16, 2026Sam Kirkham, Patrycja StrycharczukSpeech Production Latent Dynamics Modeling
Phonetics Laboratory, Lancaster University, United Kingdom · Linguistics and English Language, University of Manchester, United Kingdom
Where Latents Belong and How They Flow: Joint Identification of Structural Roles
Oct 16, 2026Xintong Li, Kun Zhang, Yujia ZhengLatent Dynamics Modeling Representation Identifiability
CMU · MBZUAI · UIUC
Learning Intrinsic Dynamics in Latent World Models
Oct 15, 2026Jiamin Shi, Yuanhong Zhang, Zhiyang Liu +2Latent Dynamics Modeling Latent World Models
Xi’an Jiaotong University · National University of Singapore · Beijing Zhongguancun Academy
RAN-JEPA: An Action-Conditioned World Model for Energy-efficient Mobile Network Control
Oct 13, 2026Binghan Wu, Kai Zeng, Kai Yang +3Action-Conditioned World Models Wireless Communications
AsiaInfo Technologies, Beijing, China · School of Computer Science and Technology, Tongji University, China · Shenzhen Loop Area Institute, China +2
Directional Pullback Regularization for Long-Horizon Rollout of Latent World Models in Autonomous Driving
Oct 12, 2026Shatadal Mishra, Ahmadreza Moradipari, Nejib AmmarLatent-Space Regularization World Models for Autonomous Driving
Toyota Motor North America R&D - InfoTech Labs
LeHami: Symplectic Latent Dynamics for World Models
Oct 11, 2026Romina Yalovetzky, Randall Balestriero, Yann LeCunLatent Dynamics Modeling Latent World Models
Courant Institute, New York University · Brown University · AMI Labs
Learn the Dynamics, Enforce the Physics: Generative Latent Operators for High-Dimensional Beam Evolution
Oct 10, 2026Haohan Zou, Alexander Scheinker, Yuanyuan ShiLatent Dynamics Modeling Physics-Informed Generative Modeling
University of California San Diego · Los Alamos National Laboratory
Bilevel optimization for data-driven learning of Koopman embeddings using kernel-based autoencoders
Oct 8, 2026Joel-Pascal Ntwali N'konzi, Feliks Nüske, Stefan KlusExtended Dynamic Mode Decomposition Koopman Operator Learning
Maxwell Institute for Mathematical Sciences, The University of Edinburgh & Heriot–Watt University, Edinburgh, United Kingdom · Max Planck Institute for Dynamics of Complex Technical Systems, Magdeburg, Germany · LAAS-CNRS, Université de Toulouse, Toulouse, France +1
Understanding Latent-Dimension Scaling in Dynamical-System Learning through Spectral Reliability
Oct 8, 2026Itsushi Sakata, Yuta Miyauchi, Yoshinobu KawaharaLatent Dynamics Modeling Dynamical Systems
RIKEN Center for Advanced Intelligence Project, Tokyo, Japan. · Graduate School of Information Science and Technology, The University of Osaka, Suita, Osaka, Japan.
Causal-fate dynamics of unrealized influence
Oct 8, 2026Yiwei Liu, Luwei Yang, Shunbo LeiLatent Dynamics Modeling Causal Inference
School of Science and Engineering, The Chinese University of Hong Kong, Shenzhen, Guangdong, 518172, China. · Shenzhen Research Institute of Big Data (SRIBD), Shenzhen, 518172, China.
Invariant-Measure Reasoners: Stable Representations for Latent Reasoning
Oct 7, 2026Yuto Inui, Takuya Konishi, Yoshinobu KawaharaLatent Reasoning Continuous Latent Reasoning
The University of Osaka · RIKEN
Similar Predictive Fit but Different Latent Dynamics: Characterizing Learned Dynamical Structure in Personalized Models of Brain Disorders
Oct 7, 2026Rita Huan-Ting Peng, Nhat BuiLatent Dynamics Modeling Electroencephalography
University of Illinois Urbana-Champaign, Urbana, IL, USA · Carle Foundation Hospital, Urbana, IL, USA
Learning PDE solution operators with variable initial conditions via Latent Dynamics Networks
Oct 6, 2026Stefano Maria Pizzamiglio, Stefano Pagani, Francesco RegazzoniLatent Dynamics Modeling Meta-Learning
MOX Laboratory, Department of Mathematics, Politecnico di Milano, Italy
Identifiable World Models from Pretrained Diffusion Representations
Oct 4, 2026Ruchi Sandilya, Conor Liston, Logan GrosenickLatent Dynamics Modeling World Models
Department of Psychiatry and Brain & Mind Research Institute Weill Cornell Medicine, New York, NY, USA
BeliefGraph-JEPA: Structured Latent World Models for Action-Conditioned Time Series
Oct 4, 2026Yue Li, Kangqi Ni, Zhen Tan +1Multivariate Time Series Forecasting Latent Dynamics Modeling
Carnegie Mellon University · University of North Carolina at Chapel Hill · Stevens Institute of Technology
Discrete Action Matching: Learning Stochastic Dynamics from Samples via State Graphs
Oct 4, 2026Mikhail Persiianov, Alexander KorotinLatent Dynamics Modeling Optimal Transport
Applied AI Institute Moscow, Russia
Latent-Lagrangian Neural Networks for Reduced Order Modeling of Non-autonomous Nonlinear Dynamical Systems
Oct 3, 2026Anand Kumar Agrawal, Anders ThorinLagrangian Neural Networks Reduced-Order Modeling
Université Paris-Saclay, CEA List, Palaiseau, France
Gaussian Flow Dynamics: Simulation-Free Neural SDE Learning Beyond One-Time Marginals
Oct 3, 2026Grigory Bartosh, Christian A. NaessethStochastic Differential Equations Latent Dynamics Modeling
Informatics Institute University of Amsterdam
LD-EnFF: Latent-Dynamics Ensemble Flow Filtering for Data Assimilation with Sparse Observations
Oct 2, 2026Ziyu Tian, Kaichen Shen, Wenbo Hao +3Data Assimilation Bayesian Filtering
School of Mathematics, Georgia Institute of Technology · School of Computational Science and Engineering, Georgia Institute of Technology

Oct 27, 2026J/K move · Enter open · S save

Understanding the limits of latent world models

Rodrigo Gonzalez Laiz, Steffen Schneider

Helmholtz Munich Munich Center for Machine Learning (MCML)

World Models Latent World Models Latent Dynamics Modeling World Model Evaluation Action-Conditioned World Models

Open paper AskPDF arXiv

World models are widely proposed as core components for enabling agents to reason over long temporal and spatial horizons. Yet current evaluations often emphasize task performance while providing limited insight into whether learned world models acquire predictive latent dynamics that reflect the underlying structure of the environment. Here we introduce the concept of faithful latent world models as an equivariance condition between the environment's transition map and the learned latent dynamics. We propose latent rollout consistency (LRC) as a metric to evaluate action-conditioned latent prediction. Strikingly, we find that across the studied state of the art latent world models, none solves the task by actually faithfully modeling the dynamics of the environment.