Latent Dynamics Modeling

Follow this topic Ask about this topic

Momentum

35 papers in the last four weeks, up 483% on the four weeks before. 0.3% of all new papers.

Jul 13Week of Sep 28

Latest papers 172

Oct 6, 2026cs.LG

Learning PDE solution operators with variable initial conditions via Latent Dynamics Networks

In many-query scenarios, data-driven surrogate models provide an efficient alternative to high-fidelity solvers for simulating physical systems governed by Partial Differential Equations (PDEs). In this context, the Latent Dynamics Network (LDNet) has recently demonstrated remarkable performance in predicting the response of spatio-temporal systems, combining Neural Ordinary Differential Equations with nonlinear dimensionality reduction. However, the original formulation assumes a fixed initial condition, limiting its applicability to many real-world applications where a system evolves from varying starting states. In this work, we overcome this limitation while keeping the end-to-end training procedure of the original LDNet and its encoder-free nature, which preserves its intrinsic independence from spatial resolution and grid topology. We infer the initial latent state directly from a small set of early-time observations, treating latent-state initialization as an adaptation problem, and investigate two strategies: an auto-decoding formulation and a meta-learning approach in which the initial latent state acts as a task-specific context variable. We demonstrate the accuracy of the proposed methods across diverse physical phenomena, spanning advection-diffusion, fluid dynamics, and solid mechanics. Meta-learning markedly accelerates latent-state inference and induces smoother, better-conditioned optimization landscapes, and spontaneously organizes the latent space into a structured representation that reflects physically meaningful features of the underlying dynamics. The coordinate-based decoder enables training from spatially subsampled data while recovering high-resolution solution fields at inference. The resulting approach provides an efficient and resolution-independent surrogate modeling framework for many-query simulations of time-dependent PDEs with varying initial conditions.

Stefano Maria Pizzamiglio, Stefano Pagani, Francesco Regazzoni

MOX Laboratory, Department of Mathematics, Politecnico di Milano, Italy

Latent Dynamics Modeling Meta-Learning PDE Surrogate Modeling PDE Operator Learning

Oct 4, 2026cs.LG

BeliefGraph-JEPA: Structured Latent World Models for Action-Conditioned Time Series

Action-conditioned time-series forecasting requires accounting for how future actions and exogenous forcings influence multiple targets through partially observed effects with different delays and persistence. Direct conditioning leaves the evolution and target-specific influence of these effects implicit in the predictor, while static relational graphs specify connections without tracking evolving effects. This motivates representing future-driver influence through structured latent states that evolve over the forecast horizon and route information to individual targets. We introduce BeliefGraph-JEPA, a structured latent world model that factorizes driver influence into typed latent-effect states. These states are rolled forward under future drivers and routed through a graph to target-specific nodes, forming the predictive base of a joint-embedding predictive architecture. A capacity-controlled residual supplements this base with direct driver information. On four multi-target clinical, agricultural, environmental, and industrial systems, the framework outperforms a range of pretrained and supervised known-future-covariate baselines. Matched controls isolate latent dynamics, future rollout, graph routing, and residual capacity; future rollout and graph-first residual routing improve forecasting across all four systems.

Yue Li, Kangqi Ni, Zhen Tan +1

Carnegie Mellon University · University of North Carolina at Chapel Hill · Stevens Institute of Technology

Multivariate Time Series Forecasting Latent Dynamics Modeling Joint-Embedding Predictive Architecture Time Series Forecasting Latent World Models

Oct 4, 2026cs.LG

Discrete Action Matching: Learning Stochastic Dynamics from Samples via State Graphs

Learning population dynamics from unpaired temporal marginals is an ill-posed inverse problem that requires structural assumptions on the underlying dynamics. We introduce

\textit{Discrete Action Matching}

(DAM), a finite-state counterpart of Action Matching based on discrete Wasserstein geometry. For a prescribed marginal path and transport geometry, we derive an action-minimization objective for its canonical minimum-kinetic-energy current. Our key observation is that the density dependence of the discrete action reduces to neighboring density ratios. Along an empirical interpolation of the snapshots, DAM first estimates these ratios and then learns an action potential. The learned fields also define a graph-supported Markov sampler. Experiments on controlled synthetic dynamics and real mouse gastrulation data evaluate marginal reconstruction and interpolation. Additional experiments approximate numerical surface-transport paths from paired samples.

Mikhail Persiianov, Alexander Korotin

Applied AI Institute Moscow, Russia

Latent Dynamics Modeling Optimal Transport

Oct 1, 2026physics.ao-ph

AI Emulation of Stochastic Sudden Stratospheric Warming with Interpretable Latent Structure

Rare weather regime transitions pose a challenge for data-driven modeling due to class imbalance. In this study, we develop a probabilistic deep learning emulator for a prototypical system with regime transitions, the stochastic Holton--Mass model of stratospheric variability, and analyze the structure of its learned latent space. The Holton--Mass model exhibits two metastable regimes, a strong and a weak polar vortex, maintained by nonlinear wave--mean flow interactions, with weak stochastic forcing intermittently triggering rare transitions between these regimes that qualitatively represent SSW events. We employ a ResNet-inspired Conditional Variational Autoencoder with six-layer encoder and decoder layers and explicit current-state conditioning to model the distribution of the system's state at the next time step (one day). The emulator accurately reproduces short-term dynamics, steady-state probability distributions, regime persistence statistics, rare transition rates, the transition committor function, and the transition expected lead time of the physical model. Beyond emulation fidelity, we interrogate the learned latent representation to understand how the model internalizes the underlying metastable structure of the dynamics. Principal Component Analysis of the 32-dimensional latent space reveals a clear and unsupervised separation into four physically interpretable clusters corresponding to strong versus weak vortex regimes and stable versus transition-prone configurations. Such emergent regime separation in latent space is hard to identify for deep generative models applied to high-dimensional stochastic systems. Our results show that carefully designed probabilistic emulators can uncover physically meaningful manifolds governing extreme-event dynamics, potentially aiding the development of improved operational advanced warning systems.

C. Daniel Boscu, Daniel Hernandez, Fabio Alvarez Ventura +4

Department of Geophysical Sciences, University of Chicago, Chicago, 60637, IL · Data Science Institute, University of Chicago, Chicago, 60637, IL · Department of Applied Mathematics, University of California, Santa Cruz, Santa Cruz, 95064, CA +1

Dynamical Systems Surrogate Modeling Latent Dynamics Modeling Climate Science Conditional Variational Autoencoder

Oct 1, 2026cs.LG

Inferring Multi-Timescale Neural Dynamics with Switching Linear Dynamical Systems

Neural activity often exhibits multiple timescales that can vary with behavioral states and task conditions. Identifying these timescales from neural recordings is important for better understanding neural computation and function. However, traditional approaches based on autocorrelation fitting are difficult to scale to high-dimensional population recordings and can become unreliable when neural dynamics change with behavior. State-space models have been a powerful framework for modeling high-dimensional neural population activity through latent dynamical systems, but standard formulations and inference methods do not explicitly account for multiple timescales and therefore do not guarantee accurate recovery of the underlying temporal structure. Motivated by these questions, we introduce the Multi-Timescale Switching Linear Dynamical System (MTS-SLDS), a framework for identifying regime-specific latent timescales from continuous or spiking neural observations. MTS-SLDS combines a multi-lag moment initialization, which captures temporal structure across multiple observation lags, with \textit{regime-conditioned} Laplace-EM inference, which reduces mixing of dynamical statistics across uncertain regimes. Characteristic timescales can then be extracted directly from the eigenvalues of the learned latent transition matrices. In synthetic and neural experiments with Gaussian and Poisson spike observations, MTS-SLDS accurately recovers timescales and switching structure over multiple datasets.

Lulu Gong, Yongxu Zhang, Shreya Saxena

Center for Neurocomputation and Machine Intelligence, Wu Tsai Institute, Department of Biomedical Engineering, Yale University, New Haven, CT 06510, USA

Stochastic Linear Dynamical Systems Latent Dynamics Modeling Regime-Switching Dynamical Systems Multiscale Time-Series Modeling

Oct 1, 2026cs.LG

Langevin-Informed Transfer Learning: Replacing Target Samples by Black-Box Feedback

Many scientific and machine learning systems, from molecular dynamics to diffusion models and beyond, are governed by stochastic dynamics with low-dimensional structure, evolving on slow timescales. However, target trajectories, used to identify and interpret such dynamics, are often inaccessible: only biased or static samples that explore the underlying manifold are available. We introduce Langevin-Informed Transfer Learning (LITL), a framework for recovering target Langevin dynamics from biased source samples using only black-box feedback. LITL learns the leading spectral structure of the target infinitesimal generator and the projected drift through Dirichlet representation learning, enabling kinetic reconstruction in spectral form and slow-manifold gradient field estimation. We further introduce a spherical variant well suited to steering normalized latent representations commonly used in learning systems toward desired objectives. We establish finite-sample guarantees for eigenvalue, eigenfunction, and projected drift estimation in Sobolev norms, thereby ensuring generalization of these quantities and their first-order derivatives. Empirically, LITL recovers physical transition timescales from biased molecular simulations, builds kinetic structure from static samples of generative models, reconstructs spherical symmetries of physical systems, and enables post-hoc latent steering of trained neural networks under black-box feedback. Together, these results position spectral operator learning as a practical framework for recovering stochastic dynamics under distribution shift and unlock applications across machine learning and the physical sciences.

Vladimir R. Kostic, Karim Lounici, Hélène Halconruy +3

CSML, Istituto Italiano di Tecnologia · University of Novi Sad · CMAP-Ecole Polytechnique +5

Latent Dynamics Modeling Transfer Learning Operator Learning

Oct 1, 2026cs.LG

Learning Commute-Time-Preserving World Models for Planning

World models allow agents to plan in latent space by choosing a sequence of actions that most reduces the distance to a given goal state. Thus, planning can benefit from latent representations whose distances mirror commute-times in the environment. The spectral embedding space of the graph Laplacian provides such a representation, if it obeys a specific eigenvalue-dependent scaling. Unfortunately, instantiating the graph Laplacian is intractable in large, continuous environments. Self-supervised learning offers a natural route to such commute-time-preserving embeddings at scale. However, here we show that existing methods, which commonly encourage isotropic representations to prevent representational collapse, tend to degrade the "correct" eigenvalue-dependent scaling, leading to an inaccurate representation of commute times. To address this problem, we introduce Commute-Time-Preserving World Models (CTWMs), combining a latent displacement predictor and a log-determinant regularizer that prevents collapse, which provably recover the correctly scaled Laplacian representation under reversible deterministic dynamics and at the predictor's fixed point. In numerical simulations, CTWM matches or outperforms LeWM, a task-agnostic baseline, on several complex, continuous goal-reaching benchmarks, while using half the parameters.

Michael Hauri, Peter Buttaroni, Fabian A. Mikulasch +1

Friedrich Miescher Institute for Biomedical Research, Basel, Switzerland · Faculty of Science, University of Basel, Switzerland

Representation Learning World Model Learning Latent Dynamics Modeling World Model-Based Planning Model-Based Planning Graph Embedding

Oct 1, 2026cs.LG

Variational Streaming Flow: Probabilistic Forecasting in Physical Time

Probabilistic forecasting is important for predicting complex dynamical systems because intrinsic randomness and incomplete observations can cause the same observed state to evolve into multiple plausible futures. While flow matching is a flexible approach for probabilistic forecasting, it is computationally expensive. Streaming flow (SF) reformulates this approach to model temporal evolution efficiently by learning a continuous velocity field directly in physical time. However, SF learns a deterministic velocity field. Thus, it provides only a single future trajectory for a given fixed initial state and observation history. To overcome this limitation, we introduce Variational Streaming Flow (VSF). Our approach learns a latent distribution that is conditioned on the dynamics of interest. In turn, this enables probabilistic forecasting. Importantly, we retain the computational efficiency of SF by generating in physical time. Across deterministic and stochastic dynamical systems, VSF demonstrates superior predictive accuracy and distributional fidelity. We demonstrate the advantage for both long-horizon rollouts exceeding 1,000 steps, and settings with bifurcating dynamics. Moreover, VSF can be integrated into existing Joint-Embedding Predictive Architecture (JEPA)-based world models as a plug-and-play predictor to improve temporal dynamics and goal-directed success rate in navigation, motion planning, and manipulation.

Hans Hao-Hsun Hsu, Minseon Gwak, Soon Hoe Lim +2

Georgia Institute of Technology · Lawrence Berkeley National Lab · International Computer Science Institute +2

Dynamical Systems Flow Matching Latent Dynamics Modeling Probabilistic Forecasting

Sep 30, 2026cs.SD

Where the Body Keeps the Beat: Structured Motion Conditioning and Music Dynamics Supervision for Dance-to-Music Generation

Dance-to-music (D2M) generation aims to synthesize musically plausible soundtracks whose temporal structure aligns with a given dance performance. Representative D2M methods do not explicitly distinguish motion cues across body parts and frequency bands, while standard flow matching lacks a dedicated objective for supervising local music-latent dynamics. To address these issues, we propose Dyna2Music, a latent flow-matching framework that combines structured motion conditioning with explicit supervision of music-latent dynamics. An empirical analysis on AIST++ quantifies how spatial partitioning and frequency separation affect raw music-to-kinematic beat alignment and motion-reference density, informing the conditioning design. Accordingly, Dyna2Music decomposes joint velocities into slow and fast components and hierarchically fuses the resulting part-wise motion energy with pretrained joint features to condition music generation. To complement this representation, we introduce latent dynamics consistency (LDC), an auxiliary objective that matches adjacent-frame change magnitudes between a single-step clean-latent estimate and the paired reference. LDC makes local music-latent variation an explicit training target without adding trainable parameters or inference computation. Dyna2Music supports variable-length music generation, and experiments on AIST++ and TikTok demonstrate improved rhythmic alignment and audio quality over representative D2M baselines.

Changchang Sun, Lu Cheng, Yan Yan

University of Illinois Chicago · Penn State University

Flow Matching Latent Dynamics Modeling Controllable Music Generation

Sep 30, 2026cs.LG

JEPA-TTT: Persistent Test-Time Training of Latent World Models for Planning under Dynamics Shifts

World models enable agents to plan by predicting future states of the environment, but their predictions can become unreliable when test-time dynamics differ from those seen during training. We present JEPA-TTT, which adapts the latent dynamics predictor of a pretrained action-conditioned Joint-Embedding Predictive Architecture world model throughout test time. Self-supervised updates accumulate across episodes, while the visual encoder and reward head remain fixed, preserving the pretrained representation and task objective. Planning requires neither a goal image nor online environment reward. JEPA-TTT uses dense replay, which forms prediction windows at every temporal offset, retains them in a growing buffer, and samples minibatches from that buffer for predictor updates. Across eight dynamics shifts in four continuous-control environments, JEPA-TTT improves planning on every shift. After 500 test-time episodes, it reduces autoregressive latent prediction error by 83% on average and improves planning performance by 153% over the frozen JEPA world model. These results show that persistent self-supervised test-time training can adapt a pretrained latent world model under changed dynamics.

Zheyuan Zhang, Suyu Ye, Nakul Agarwal +5

Honda Research Institute USA · Johns Hopkins University

Test-Time Adaptation Latent Dynamics Modeling World Model-Based Planning Joint-Embedding Predictive Architecture Action-Conditioned World Models Model-Based Planning

Sep 30, 2026cs.LG

Learning Continuous Neural Representation of Stochastic Hybrid Systems

A stochastic hybrid system (SHS) is governed by a stochastic differential equation (SDE) describing the continuous dynamics and a Markov reset kernel triggered on the guard surface. Its probability evolution can be described by a hybrid Fokker-Planck (HFP) equation with a partial differential term corresponding to the SDE and an integral term arising from the reset kernel. This work shows that such an SHS can be approximated by an SDE in a higher-dimensional latent space where the sample paths are continuous. The key to this result is to encode different branches of the reset kernel using auxiliary variables, transforming the resets into deterministic ones that enable topological gluing. By the embedding theorem, the glued manifold can then be embedded into a higher-dimensional Euclidean space. We show that the probability evolution on the embedded image no longer requires explicit reset terms in the HFP equation. Building on this theorem, we design a loss that matches the evolving state distributions, enabling a single latent SDE to recover the probability evolution of the SHS without mode labeling, trajectory segmentation, or event-based simulations.

Sangli Teng, Hang Liu, Koushil Sreenath

Dynamical Systems Stochastic Differential Equations Latent Dynamics Modeling State Representation Learning

Sep 29, 2026cs.LG

Predictive Self-Supervised Learning Provably Identifies Stochastic Signals under Nuisance

Self-supervised learning (SSL) by predicting in latent space, without generating the input data itself, learns highly abstract, useful representations. Intuitively, this success is often attributed to its ability to discard nuisance information that is irrelevant to prediction. However, this poses a conundrum: both stochastic variation in a prediction-relevant latent signal and true nuisance make observations partly unpredictable; how could they be distinguished? Surprisingly, we prove that common SSL methods can achieve exactly this, by implicitly instantiating a latent-variable model with stochastic dynamics and observation-private nuisance. We trace their ability to recover the stochastic signal to two complementary principles: Predictive mutual information maximization ensures that representations retain the information needed for prediction, while latent distribution matching constrains how this information is encoded, thereby making the retained signal identifiable. We confirm this identifiability result in simulations for Gaussian predictors, which recover the true signal up to an affine transformation even in dynamic, nuisance-laden environments.

Fabian A. Mikulasch, Friedemann Zenke

Friedrich Miescher Institute for Biomedical Research, Basel, Switzerland · Faculty of Science, University of Basel, Switzerland

Latent Variable Models Representation Identifiability Latent Dynamics Modeling Predictive Representation Learning Invariant Representation Learning

Sep 29, 2026cs.AI

Generative Interactions: Weaving Multiparty Human Motion with Bilevel Latent Dynamics

Human social behaviour is not a collection of independent motions, but a jointly organised process in which group dynamics and individual variation continuously shape one another. Yet existing social motion models often prioritise plausible trajectories while leaving interaction state implicit, limiting their ability to transfer across groups, tasks, and partial-observation regimes. To address this gap, we introduce Bilevel Representations for Agent Interaction Dynamics (BRAID), a hierarchical sequential latent-variable model for generative multi-person interaction. BRAID explicitly formulates social motion generation as a meta-transfer learning problem: shared interaction priors are learned across datasets and adapted through arbitrary context sets of observed people and joints. The model represents each scene through a group-level latent state that captures shared interaction dynamics and person-level latent states that capture individual behaviour conditioned on the evolving group context. This modelling choice enables coherent generation under full, sparse, or partial observations while exposing compact social-state vectors that can serve as an interface for downstream embodied-agent systems. We evaluate BRAID under a unified SMPL-based representation on social forecasting, tracking and in-filling, and response generation, using metrics that assess not only reconstruction accuracy but also realism, diversity, temporal alignment, and interpersonal coordination. We further analyse the hierarchical latent space, showing that it captures separable group- and individual-level structure.

Ojas Shirekar, Yash Surange, Agustinas Jučas +1

TU Delft Delft, The Netherlands · University of Oxford Oxford, UK

Latent Dynamics Modeling Human Motion Generation Human Motion Prediction

Sep 29, 2026cs.AI

Accelerated surrogate dynamics for dynamical, stochastic system evolution

Dynamic simulations are an entrenched way of gaining insight into the evolution of system dynamics. Their computational cost however is often prohibitively high, especially in cases of stochastic frameworks. Machine learning algorithms are especially suited as simulation surrogates. Nevertheless, they face some very distinct limitations. Firstly, the sheer dimensionality of these systems, however, precludes the use of traditional time series models who struggle with high dimensional feature spaces. Additionally, traditional time series focus exclusively on either long or short range effects, causing local or global drift given enough time. In this paper, we propose a framework that addresses those limitations. Our framework combines a Variational Autoencoder, with a convolutional or graph basis that reduces the dimensionality of the system. This latent vector is propagated in time using a Temporal Fusion Transformer model, which includes both long range and short range effect encoding, as well as static covariate support. We test our framework on three distinct cases, to prove its robustness and in all three we have achieved practically identical to the simulation results at a fraction of the time. Further, our framework is flexible enough to be adapted to any new system and provides an inbuilt uncertainty quantification for targeted experiment design.

Marco Jochum, Ioannis Kouroudis, Gohar Ali Siddiqui +3

Chair of Simulation of Nanosystems for Energy Conversion, Department of Electrical Engineering, TUM School of Computation, Information and Technology, Atomistic Modeling Center (AMC), Munich Data Science Institute (MDSI), Technical University of Munich, Hans-Piloty-Straße 1, 85748 Garching, Germany

Neural Surrogate Modeling Dynamical Systems Stochastic Linear Dynamical Systems Latent Dynamics Modeling

Sep 28, 2026stat.ML

One-Step Next-Latent Prediction Is Not a World Model

Next-latent prediction fits a map from the current embedding to the next one. LeNEPA carries this objective to time series, replacing the stop-gradient of next-embedding prediction with the isotropy penalty of LeJEPA. A world model is a transition kernel that can be rolled out. The one-step regression identifies a conditional mean, and a mean is a kernel only in special cases. For a linear-Gaussian Markov latent, the mean transition and the innovation covariance are fixed by the one-step problem, and the open-loop squared error at horizon

K

equals the trace of the sum of the pushed-forward innovation covariances. That error grows with

K

after the one-step fit is exact. If the conditional mean is nonlinear, composing it is not the multi-step conditional mean. If the observation is a non-injective function of a Markov state, a memoryless one-step map does not determine future observations, while a short window can. An isotropy penalty is a function of the embedding marginal, so its partial derivative in the transition weights is zero. On a scalar autoregression with coefficient

0.9

, the one-step mean squared error is

0.998

and the

16

-step open-loop error is

5.10

. On a hidden rotation, an eight-step window reaches

16

-step error

0.056

, while the current scalar alone reaches

0.778

. Raising the isotropy weight from

0.1

to

10

leaves eight-step latent error inside

[0.78,0.85]

on three seeds.

Shitong Wang, Zhongang Cai, Yuzhou Hong

World Model Learning Latent Dynamics Modeling

Sep 28, 2026eess.SY

Statistical Learning of Contractive Dynamical Representations for Composite Adaptive Control

We present a representation-learning framework for composite adaptive tracking control under dynamically coupled disturbances. The framework connects classical disturbance-accommodating control (DAC) to recent last-layer adaptive disturbance-rejection methods. Specifically, we introduce a statistically principled hard expectation-maximization (hard-EM) procedure, with a Kalman smoother in the hard E-step, to identify dynamical representations of disturbance whose latent evolution is uniformly contractive. The learned representation evolves a latent disturbance-excitation state from measured plant features and control inputs and decodes that state into the time-varying disturbance acting on the nominal plant, thereby extending prior "fixed-decay" last-layer adaptive methods to a learned, predictive DAC-style formulation. Combined with Bayesian filtering of the learned latent state, this representation yields a composite adaptive tracking controller with predictive capability and provable exponential convergence to a bounded neighborhood. We validate our approach experimentally on a slippery ground vehicle carrying a liquid-sloshing tank and a pendulum load, and we further assess its robustness on a system of coupled Duffing oscillators. Across both settings, the method achieves accurate disturbance prediction and improved overall tracking performance relative to fixed-decay representation-learning ablations, LTI disturbance-accommodating baselines, and model-based PD baselines.

Min Kim, José Leonardo Brenes, Fred Hadaegh +1

California Institute of Technology (Caltech), Pasadena, CA 91125 USA. · Jet Propulsion Laboratory (JPL), Pasadena, CA 91109 USA.

Representation Learning Latent Dynamics Modeling Adaptive Control

Sep 28, 2026cs.LG

Identifying Neural Source Dynamics from Unknown Local Interventions

Electroencephalography (EEG) records mixtures of brain-source activity. Even with a known anatomical forward model, experiments that excite only part of the source-state space leave the dynamics unidentified, and repetition cannot resolve the ambiguity. We show that unknown local mechanism changes can supply the missing information. We consider linear dynamics among fixed anatomical sources with known source-state initialization patterns. Changing one source's update rule for one transition leaves a rank-one, source-specific signature in subsequent EEG: subtracting matched baseline responses isolates it, and the forward model identifies the source and calibrates its response history. Combining these histories with initialization responses recovers source interactions without baseline reachability and without first identifying the intervention coefficients. We establish sufficient recovery conditions, a direct estimator, and a noise-sensitivity bound conditional on correct source labels. Simulated EEG on anatomy derived from magnetic resonance imaging confirms the information gain: with baseline excitation confined to four of twelve source coordinates, eight unknown changes recover all dynamics in 32/32 systems, whereas baseline realization, baseline regression through an invertible forward model, and changes that leave the tested states unexposed all fail, and explicitly constructed alternative dynamics reproduce every baseline mean. Where baseline information suffices, direct reconstruction is also more reliable than a matched-information spectral estimator. Nonlocal changes and forward-model error limit accuracy even when source labels are correct.

Ayana Mussabayeva, Jiaqi Sun, Anuar Aimoldin +2

Mohamed bin Zayed University of Artificial Intelligence (MBZUAI), Abu Dhabi, UAE · Carnegie Mellon University (CMU), Pittsburgh, PA, USA

Dynamical Systems Electroencephalography Latent Dynamics Modeling System Identification

Sep 28, 2026cs.AI

PDE-JEPA: Predictive Representation Learning of Latent Dynamics Modeling for Parametric PDEs

Physical trajectories contain more than snapshots of a system: they also reveal how its states evolve under governing conditions. However, representation learning for parametric partial differential equations (PDEs) has largely relied on reconstruction-based objectives that emphasize recovering observed physical fields. In this paper, we investigate predictive representation pretraining as an alternative to reconstruction-based learning. We find that predictive representations preserve rich physical information, yet this advantage alone does not ensure accurate field evolution. Based on these observations, we introduce PDE-JEPA for parametric PDE dynamics. Specifically, we first train an encoder using a masked-latent prediction to capture the underlying regularities of PDE dynamics. To explicitly adapt the pretrained representation toward a more dynamics-aligned state space, we then introduce a geometry projector that aligns latent trajectory geometry with the evolution geometry of physical fields. Finally, building on this geometry-aligned latent space, we further develop a physics-structured latent predictor that decomposes the dynamics into parameter-independent evolution and parameter-dependent response components. Extensive experiments on nine widely used PDE benchmarks demonstrate that our framework outperforms existing state-of-the-art methods by an average of 33.4% in-distribution, while achieving an average improvement of 51.4% when extrapolating to unseen governing parameters. The project page is available here.

Zhentao Tan, Jianrong Zhang, Ruijie Quan +1

Zhejiang University.

Latent Dynamics Modeling Joint-Embedding Predictive Architecture PDE Surrogate Modeling Predictive Representation Learning Physics-Informed ML

Sep 28, 2026cs.LG

DisKO: Deep Koopman Learning in Distribution Space from Unpaired Snapshots

Many complex systems are observed only through temporally unpaired distribution snapshots, making trajectory-based dynamical learning difficult without additional assumptions. We therefore formulate the problem directly in distribution space, treating the distribution itself as the dynamical state. The challenge is that distribution space is infinite-dimensional, making compact and approximately closed representations difficult to learn from finite snapshots. We introduce DisKO, which extends deep Koopman learning to distribution dynamics by jointly learning predictive distributional observables, a finite-dimensional Koopman representation, and a generative map back to the full distribution. Across seven diverse benchmarks, DisKO achieves state-of-the-art extrapolation performance, with substantially slower error accumulation on long-horizon prediction tasks. DisKO further recovers leading Koopman eigenvalues and eigenfunctions on systems with analytic spectra, revealing meaningful dynamical structure in the learned representation.

He Ma, Xiaochen Liu, Wanfeng Lu +3

School of Mathematical Sciences, Fudan University, China. · Research Institute of Intelligent Complex Systems, Fudan University, China. · Shanghai Center for Mathematical Sciences, Fudan University, China. +2

Koopman Operator Learning Latent Dynamics Modeling Predictive Representation Learning

Sep 28, 2026cs.LG

Shaping Persistent Representations from Independent Interactions

World models learn environment dynamics from interaction experience. These dynamics depend on the current state and actions, as well as on properties that persist across interactions. Yet standard predictive training can reduce error using local evidence alone, without organizing persistent information into reusable context. We introduce SPRII, a training principle that uses relations between interactions as weak supervision for persistent context while retaining the learner's native objective. For example, different trajectories of the same system share persistent properties even when their states and actions differ. SPRII uses such relations to guide context learning without numerical property labels. Two composable components encourage contexts from related interactions to agree (Align) and use one interaction's context to predict another's future (Cross). Our analysis distinguishes three linked questions: what persistent information is accessible in the learned context (Formation), how that context influences a fixed predictor (Use), and whether it reduces task error (Value). Success at one stage does not guarantee success at the next. Controlled experiments show that more reliable relations improve representation organization, but adding a shared-property constraint can reduce access to a property that remains shared. Context substitutions change predictions at fixed model weights, while the benefit from history depends on prediction horizon and readout. Evaluations span thirteen settings, including controlled physical systems, public dynamics tasks, robotic and tactile data, and partner interaction, across multiple learner families. Relative to the corresponding baselines, SPRII yields average gains of over 10% in downstream task performance and over 15% in persistent-property readout. The project page is available at https://persistent-learning-review.netlify.app/interactive.html.

Ji Dai, Quan Fang, Junyu Gao +4

Beijing University of Posts and Telecommunications · Institute of Automation, Chinese Academy of Sciences · Shenzhen University +1

Representation Learning Latent Dynamics Modeling World Models Predictive Representation Learning

Sep 28, 2026cs.LG

Retracing Hodgkin and Huxley: State Recovery Does Not Certify Mechanism

Predicting observed dynamics does not establish recovery of the underlying physical mechanism. Can machine learning retrace the hidden-state reasoning behind the Hodgkin-Huxley (HH) model? We train structured latent models on simulated current and voltage, withholding gate identities and trajectories from training and model selection. We then test response prediction, state recovery, protocol transfer, and agreement with HH dynamics. Prediction error and its cross-seed spread both drop sharply at three latent dimensions under the tested protocols, while gate recovery under new protocols improves through five to six coordinates. State recovery depends on which observations the chart uses. Observed voltage improves current-clamp decoding relative to freely predicted voltage. Under voltage clamp, adding latent state to command voltage raises m-state

R^2

from 0.976 to above 0.99, yet the transported field disagrees with HH on identical smooth samples. Known invertible HH coordinates achieve high fast-m field agreement under the same audit procedure. An exact HH identity decomposes the discrepancy into time-scale-weighted state error and a residual in the transported field; these terms can cancel or reinforce. These findings concern the tested models and charts. They support evaluating state and dynamics recovery separately, including chart inputs and transported-field agreement across interventions.

Peiyu Zang, Jiayi Hao, Yongqiang Cai

School of Mathematical Sciences, Beijing Normal University

Dynamical Systems Latent Dynamics Modeling System Identification State Representation Learning

Sep 27, 2026cs.RO

Beyond One-Step Accuracy: State-Affine Latent Transition for Reliable Visual Planning

Joint-embedding world models enable visual planning by learning action-conditioned dynamics in latent space. Yet they are commonly trained for one-step prediction on encoded states, while planning recursively applies the learned transition to its own predictions. One-step accuracy therefore does not capture how prediction errors propagate under recursive rollout. We decompose multi-step rollout error into the errors introduced at individual steps and their propagation through subsequent transitions. We show that state-affine dynamics are precisely the differentiable transitions with state-independent Jacobians, eliminating the nonlinear propagation residual and making the error propagation operators depend only on the action sequence. Guided by this result, we introduce SALT (State-Affine Latent Transition), an action-conditioned state-affine dynamics model in which the action modulates both the state transformation and the additive update. We train SALT through recursive multi-step rollout supervision, feeding each predicted latent state back into the transition so that training matches how the model is used during planning. Across four visual planning environments, SALT exhibits

1.48

--

2.19\times

higher one-step prediction error than the matched LeWM baseline, yet improves closed-loop success in every environment by

10.0

percentage points on average. On OGBench-Cube, the fraction of episodes that fail with a sharp rise in model-predicted cost after execution decreases from

23.3%

to

2.0%

.

Boyuan Zhang, Yingjun Du, Xiantong Zhen +1

UCAS-Terminus AI Lab, School of Engineering Science, University of Chinese Academy of Sciences · VIS Lab, University of Amsterdam · Central Research Institute, United Imaging Healthcare, Co., Ltd.

Dynamical Systems Latent Dynamics Modeling World Model-Based Planning Model-Based Planning Latent World Models

Sep 27, 2026cs.LG

Correct then Forecast: Observer State-Space Models for Time Series Forecasting

Time series forecasting requires extrapolating the dynamics of an observed process beyond the last available measurement. Yet recurrent forecasting models typically treat observations as inputs that directly control their latent dynamics. It leads to a regime change when these observations become unavailable at prediction time. Following a state-estimation perspective, we introduce Observer State-Space Models (OSSMs), a class of recurrent models that interprets the observed input time series as measurements of an underlying autonomous dynamical system. OSSMs explicitly separate latent-state propagation from measurement assimilation: a single transition governs the dynamics across both context and forecasting intervals, while available observations correct the estimated state through an observer. This formulation naturally exposes classical control-theoretic properties, including observability and convergence of the state estimation error. We further show that conventional and recent SSMs can be recovered as particular instances of our OSSM framework, thereby providing a unified interpretation of their recurrent dynamics and revealing modeling inconsistencies. We perform experiments across several benchmarks showing that OSSM achieves substantial improvements while maintaining the same parameter count and training setup as the corresponding SSM baseline. These results support a simple principle for recurrent forecasting: observations should correct the estimated latent state, rather than control the dynamics used to propagate it.

Alexis-Raja Brachet, Guillaume Clavier--Frémond, Abdelhakim Ziani +2

MICS, CentraleSup´elec, Universit´e Paris-Saclay, France · CentraleSup´elec, IETR UMR CNRS 6164, France · Universit`a di Torino, Torino, Italy.

Latent Dynamics Modeling Time Series Forecasting State Space Models

Sep 27, 2026math.ST

Identifying the Predictable Drift of a Semimartingale from Marginal Laws

A special semimartingale admits a unique decomposition

X=X_0+M+A

into a local martingale

M

and a predictable finite-variation part

A

. We consider the identification of

A

when

X

is observed only through repeated cross-sections. The estimand is then the projection of the sampled predictable compensator onto the observable feature filtration, namely the current state together with whatever randomness is shared across the population, so that at a fixed diffusion coefficient the marginal flow identifies the drift only up to a Markovian projection. If the drift is an affine functional of an observed lag window, the joint problem is a convex quadratic programme whose solution is the pseudo-panel regression of econometrics. Our principal concern is the case, which we believe not to have been treated before, in which the drift is the output of a hidden linear dynamical system whose dynamics are themselves to be identified from the marginals. The joint problem is then a bilinear quadratically constrained programme, which we solve to certified global optimality by spatial branch and bound; with unpenalised state disturbances and a drift basis growing with the grid it is NP-hard already in latent dimension one, by reduction from

\ell^1

rank-one matrix approximation, whereas the complexity of the deterministic system at fixed latent dimension remains open. A block-coordinate decomposition offers a cheaper alternative. For the estimator itself, we obtain rates at a fixed mesh, separated into Monte-Carlo, estimation and grid contributions.

Jakub Marecek, Enrico Biffis, Abigail Langbridge +1

Department of Computer Science, Czech Technical University in Prague, Czechia · Department of Finance, Imperial College London, UK · Dyson School of Design Engineering, Imperial College London, UK

Latent Dynamics Modeling Constrained Optimization State Space Models

Sep 27, 2026cs.LG

KoopCell: Koopman-Based Generative Model for Learning Single-Cell Dynamics from Distribution Snapshots

Learning population dynamics from temporally sparse, unpaired distribution snapshots is a fundamental challenge in developmental biology. Recent approaches based on neural differential equations and flow matching can interpolate between observed population snapshots, but may struggle to extrapolate beyond the training horizon and often lack an explicit mechanism for modeling developmental branching. We propose KoopCell, a unified generative framework based on Koopman-Mori-Zwanzig theory that jointly learns representations and predictive linear latent dynamics. Theoretically, using the weak continuity equation, we derive a closed-form least-squares estimator for the Koopman generator from distribution snapshots and establish convergence guarantees under suitable assumptions. To model branching dynamics, we further develop KoopCell-M, which incorporates non-Markovian memory into the latent Koopman dynamics through a Markovian embedding. Experiments on synthetic systems and three scRNA-seq datasets demonstrate the ability of our framework to recover Koopman spectra, model branching through memory, and scale to predicting high-dimensional gene expression distributions, achieving state-of-the-art performance among the evaluated methods.

Wanfeng Lu, Yutong Zhang, Keyi Zhou +3

School of Mathematical Sciences, SCMS, SCAM, and CCSB, Fudan University, China · Research Institute of Intelligent Complex Systems, Fudan University, China · State Key Laboratory of Medical Neurobiology and MOE Frontiers Center for Brain Science, Institutes of Brain Science, Fudan University, China +1

Koopman Operator Learning Latent Dynamics Modeling Generative Modeling Single-Cell Transcriptomics

Sep 26, 2026cs.LG

Adaptive Latent Capacity for World Models

We introduce Adaptive LeWorldModel (ALeWM), a world model based on a joint-embedding predictive architecture (JEPA) that learns to concentrate predictive information in compact prefixes of a wide latent representation. To encourage this ordering, ALeWM learns a sequence-conditioned distribution over prefix lengths and trains the predictor to estimate the full next embedding from a sampled input prefix. As standard anti-collapse objectives encourage variation across latent coordinates and do not organize them by predictive importance, we also introduce MixSIGReg. MixSIGReg regularizes the masked embeddings against a prior-weighted mixture with Gaussian active prefixes and zeros in the remaining coordinates. As a result, the ALeWM objective encourages early coordinates to retain information useful for prediction and recursive planning. Our analysis shows that the mixture distribution used by MixSIGReg assigns higher variance to earlier coordinate blocks and lower variance to later ones. In addition, we show that, under specified assumptions, prediction error is minimized by placing the information most useful for prediction in earlier blocks. Empirically, we study the behavior of ALeWM in a controlled dynamical system with known state variables and in goal-conditioned visual control. We show that ALeWM consistently achieves higher mean success rates than tuned fixed-width LeWM, with lower planning capacity on average.

Idan Achituve, Lior Dikstein, Idit Diamant +2

Arm Research Israel

World Model Learning Latent Dynamics Modeling World Model-Based Planning Joint-Embedding Predictive Architecture Model-Based Planning

Sep 24, 2026cs.LG

Beyond Compression: Training Latent Representations for Stable Long-Horizon Rollout in Neural Surrogate Solvers

Latent neural surrogate solvers, or latent dynamics models, accelerate simulations of time-dependent physical systems by evolving a compressed latent space rather than resolving full-resolution fields directly. In principle this reduces computational cost and simplifies learning, but in practice errors often accumulate rapidly during long autoregressive rollouts, limiting predictive utility. We show that this instability does not stem from the latent representation itself, but arises when it is trained solely for reconstruction, producing representations poorly suited to long-horizon forecasting. We systematically evaluate training-level interventions that align latent representations with long-horizon rollout: Koopman operator learning and Hamming noise injection during autoencoder training to improve compression, together with noise injection and multi-step rollout fine-tuning to improve dynamics. Interventions that improve long-horizon rollout stability often degrade conventional training metrics, including reconstruction and one-step prediction accuracy. Collectively, these interventions reduce long-rollout error by approximately 40% and match or exceed the accuracy of full-resolution models on two physics benchmarks, while requiring 2 orders of magnitude fewer floating point operations and half the GPU memory. Applied to mesoscale crystal-plasticity simulations of high-cycle fatigue, the resulting surrogate achieves stable extrapolation over horizons orders of magnitude beyond those observed during training. More broadly, these results show that neural compression should be designed not merely to reduce dimensionality, but to restructure the solution space for stable dynamical evolution, a key requirement for reliable, efficient neural surrogates in scientific applications.

Andreas E. Robertson, Ashley T. Lenau, John D. Shimanek +5

Center for Integrated Nanotechnologies, Sandia National Laboratories, New Mexico, USA · Center for Computing Research, Sandia National Laboratories, Albuquerque, NM, USA · School of Engineering, Brown University, Providence, RI, USA +2

Neural Surrogate Modeling Latent Dynamics Modeling

Sep 23, 2026cs.LG

Beyond Static Graph World Models: Learning Stochastic Latent Dynamics over Evolving Topologies

Graph-based world models have recently emerged as a means of learning transitions over relational state representations. However, existing approaches are largely limited to fixed-topology graphs or deterministic, fully observable environments. We propose the Graph Dynamics Model (GDM), a world model for graph-structured observations that is designed to handle the more general setting of evolving topologies in stochastic and partially observable environments. The GDM uses a sparse recurrent adjacency matrix to model topology updates and perform message passing, together with a recurrent state-space architecture for modelling stochastic transitions. Furthermore, we identify a gap in the evaluation of graph-based world models, as existing methods do not provide a means of comparing predicted and true distributions over the joint graph state comprising the interdependent topology, node features, and graph features. We therefore introduce the Graph Distribution Distance (GDD) metric, which uses maximum mean discrepancy with a graph kernel to comprehensively compare joint next-state distributions. We evaluate the GDM across several environments, including stochastic and partially observable settings. We demonstrate that GDM outperforms baseline models and displays zero-shot generalisation on large graphs.

Alex Schutz, Nick Hawes, Victor-Alexandru Darvariu

Oxford Robotics Institute University of Oxford

Latent Dynamics Modeling Dynamic Graph Learning Latent World Models

Sep 23, 2026cs.CV

Frozen Flows Forget: Diagnosing and Restoring Lost Motion in a Latent-flow World Model

Latent world models that integrate a flow in a frozen self supervised latent space train stably and cheaply, yet silently lose the property manipulation depends on most: motion. The pretrained flow never moves the manipulated object; retraining it with latent-only losses only trades stillness for teleport-like motion. We trace the failure to the training signal, not the representation: anchor-sparse, latent-only supervision never says where along the horizon change belongs. Decode-augmented rollout training (DART) repairs this while keeping the representation frozen, retraining only the flow with decode-path supervision. DART outperforms its latent only parent on the full protocol, restores the temporal structure of motion, and re-couples predicted motion to the scene; at larger scale it further improves prediction quality, closing nearly half the remaining gap to an oracle-informed interpolation reference. Finally, we report an unexpected finding about evaluation: pixel error alone rewards frozen predictions.

Xiwen Chen, Rigaudiere Z. Li, Zhiruo Zhou +2

Shenzhen International Graduate School, Tsinghua University · Shanghai Jiaotong University · Wuhan University of Technology

World Model Learning Latent Dynamics Modeling Latent World Models Flow-Based Generative Modeling

Sep 20, 2026cs.CV

Algebraic Consistency Alone Does Not Certify Temporal Structure in Latent Action Models

Latent action models infer a code for the transition between two frames of action-free video. Recent methods regularise this code to compose additively and reverse antisymmetrically, and report order-of-magnitude reductions in the resulting errors as a label-free certificate that the code has captured temporal structure. We show that this conclusion does not follow. Reconstruction drives the decoded transition toward a difference of state features, for which both identities hold for any pairing, a solution the metric cannot distinguish from one encoding nuisance state or a coordinate convention. Across five source domains, a trained but unconstrained counterpart already achieves 83-97% of the reduction relative to an untrained anchor. The residual fold is governed as much by the decoder family as by what is learned. A constrained model retrained after its temporal pairing is destroyed still reaches, in each domain, a lower error than the unconstrained model on real data. Downstream, preserving the temporal pairing yields no consistent advantage on LIBERO-GOAL or LIBERO-SPATIAL, and across the tested arms the code's mean linear action decodability falls as the algebraic error improves. We also test the most direct repair, a violation-contrastive objective that requires the algebra to fail on destroyed pairings: in the tested configurations it yields only a marginal separation within the reconstruction budget, on training and test triples alike. We recommend a validation protocol that these methods currently lack: a baseline-corrected metric, retraining on destroyed pairings, and a seed-budget analysis.

Di Wen, Ruodi Zhang, Kailun Yang +1

Karlsruhe Institute of Technology, Karlsruhe 76131, Germany · Hunan University, Changsha 410012, China

Latent Action Learning Latent Dynamics Modeling

Sep 17, 2026cs.LG

Seismic Site Response Prediction from Sparse Observations Using Finite-Element-Pretrained Latent Dynamics

Numerical site-response predictions often deviate from observations, yet correcting these discrepancies is difficult because records are limited in both sensor coverage and number of events. This study proposes the Transfer-Enabled Forced Latent Autoencoder for Response Equations (FLARE-T) to improve these predictions by learning and calibrating low-dimensional latent dynamics that connect the base acceleration input to acceleration outputs at multiple depths. FLARE-T learns a low-dimensional response manifold and input-driven dynamics from dense finite-element simulations. It then trains a sparse encoder to map simulated sensor responses into the learned coordinates and uses limited records to calibrate the dynamics within them. A short response window initializes each prediction, while the complete base motion drives the response. The framework was evaluated using a layered-soil centrifuge test and the Lotung field vertical array. Test-set results show that FLARE-T improved multi-depth acceleration histories and 5%-damped pseudoacceleration response spectra relative to the original finite-element models, reducing errors at every evaluated sensor for motions of different intensities and, at Lotung, for both horizontal components. Two Lotung source models with different constitutive parameters achieved comparable test-set accuracy, indicating reduced dependence on precise prior calibration. FLARE-T therefore provides a data-efficient means of combining dense numerical response information with limited field records to improve future site-response predictions.

Yi Zhu, Su Chen, Xiaojun Li

State Key Laboratory of Bridge Safety and Resilience, Beijing University of Technology Beijing 100124, China

Neural Surrogate Modeling Latent Dynamics Modeling Seismology

Sep 15, 2026cs.LG

Walking the Score Manifold: Continuous-time Generative Dynamics on Learned Data Manifolds

Generative modeling of time-dependent data is typically formulated on a discrete temporal grid, restricting supervision to the observed timestamps in the training data. We instead frame generation as continuous-time evolution on a learned data manifold. To this end, we leverage pretrained score-based models as geometric priors and learn a vector field that evolves data along score-induced interpolation paths. Because these dynamics follow transitions that respect the geometry learned by the score model, they support generation at arbitrary timestamps and temporal super-resolution beyond the discretization of the training data. Moreover, this geometric formulation allows us to train the vector field simulation-free through a regression objective. To improve long-horizon rollout robustness, we introduce an objective that promotes path-relative transverse exponential stability. While motivated by stability theory, it admits a practical interpretation as denoising score matching transverse to the interpolation path. Further, we extend the framework to a probabilistic setting that models a distribution over plausible future trajectories. We demonstrate the method on natural video and scientific dynamical data, including temporal super-resolution, PDE-based spatiotemporal fields, and molecular dynamics. Our results show that score-based priors provide a strong foundation for learning stochastic continuous-time generative dynamics.

Jan Tauberschmidt, Brian B. Moser, Stanislav Frolov +3

DFKI Kaiserslautern · RPTU Kaiserslautern · Imperial College London

Dynamical Systems Latent Dynamics Modeling Generative Modeling Score-Based Generative Modeling

Sep 15, 2026cs.LG

Stable by Construction: Variational Latent Markov Operators for Long-Horizon PDE Prediction

Neural PDE solvers provide efficient surrogates for time-dependent physical systems, but autoregressive prediction over long horizons remains challenging because local errors can induce distribution shift and accumulate under recursive deployment. We develop a variational approach to this problem by introducing latent Markov dynamics in which physical states are represented by latent distributions and evolved through probabilistic transitions. The framework is formulated directly on function spaces and specialized to functional Gaussian models, where structured latent perturbations induce a spectral geometry and variational transition alignment regularizes the learned dynamics. We further analyze how these mechanisms affect autoregressive error propagation, providing a theoretical connection between variational training and long-horizon prediction. We instantiate the framework as the Variational Autoencoding Markov Operator (VAMO), which combines spatially resolved latent fields, structured Gaussian perturbations, and a neural-operator transition. Empirically, we demonstrate the effectiveness of VAMO on several fluid-dynamics benchmarks with prediction horizons extending substantially beyond those represented during training, where it consistently reduces error accumulation and improves rollout stability over several deterministic and noise-injection baselines. Overall, these results highlight variational modeling as a complementary approach to robust long-horizon neural PDE dynamics.

Junyi Liao, Johann Guilleminot, Vahid Tarokh

†Department of Electrical and Computer Engineering, Duke University · ‡Department of Mechanical Engineering and Materials Science, Duke University

Variational Autoencoders Latent Dynamics Modeling PDE Operator Learning Spatiotemporal Forecasting Neural Operators

Sep 14, 2026cs.CV

Reconstructing Is Not Acting: Action-Centric Latent Dynamics Modeling

Latent action models (LAMs) learn action representations from unlabeled videos by inferring latent actions from visual transitions and reconstructing future states. However, we identify a fundamental

\textbf{reconstruction-action mismatch}

: lower reconstruction error does not necessarily yield better latent dynamics or downstream performance. We attribute this mismatch to two underconstrained aspects of reconstruction-based latent dynamics modeling: (i) the inverse dynamics model (IDM) is not explicitly encouraged to distinguish action-related transitions from nuisance appearance, and (ii) the forward dynamics model (FDM) can underutilize the inferred latent action by exploiting predictive shortcuts from the current state. To address both limitations, we propose

\textbf{ACT-LAM}

, a lightweight action-centric framework that strengthens both action extraction and action utilization. Specifically, its Action Query IDM (AQ-IDM) employs learnable action queries and gated aggregation to selectively extract rich action-related transition cues without strong information bottlenecks. And its Action Token FDM (AT-FDM) projects latent actions into action tokens that progressively interact with evolving state representations, enabling continuous state-aware action conditioning. ACT-LAM further streamlines feature processing to concentrate model capacity on latent dynamics modeling. Extensive experiments on several robotic datasets and the VP

^2

benchmark demonstrate stronger latent action consistency, forward dynamics, and downstream visual planning performance with fewer trainable parameters and lower computational overhead. In particular, ACT-LAM surpasses the previous state of the art by

\textbf{7.6%}

on the aggregated VP

^2

success rate. Codes at

[url](https://github.com/DingjieFu/ACT-LAM)

.

Dingjie Fu, Dianxing Shi, Yangyang Xu +1

Harbin Institute of Technology (Shenzhen) · Beihang University

Latent Action Learning Latent Dynamics Modeling Action-Conditioned World Models Robotics

Sep 9, 2026cs.LG

Semigroup-JEPA: Latent Dynamics Consistency for Zero-Shot Physics Generalization

Joint-Embedding Predictive Architecture (JEPA) world models learn a compact latent representation of the world that supports prediction and planning, but their capability to learn physics and generate physically realistic dynamics remains hitherto untested. In this work, we introduce SemiGroup-JEPA (SG-JEPA), which extends the LeWorldModel framework by supplying the parameter governing the physics to the temporal model via action-conditioning and jointly training an encoder and predictor through an autoregressive latent rollout. To evaluate the model's ability to generalize out of distribution, we design dynamical tasks under different gravitational fields that, despite obeying the same physical law, exhibit qualitatively different dynamics, ranging from floating motion in weak gravitational fields to rapid bouncing in strong ones. In contrast to DINO-WM, SG-JEPA reduces open-loop prediction error by up to 2 times on two-dimensional datasets, and increases control success rate up to 2.5 times for three-dimensional robotic datasets, for which we train independent diffusion policies. To explain this advantage, we develop a linear feature model that separates local law-conditioned error from its recursive amplification under rollout. Guided by this model, we find that back-propagating the multi-step rollout loss into the representation trains the encoder to keep the features that the predictor can carry forward, and that those are the features the dynamics depend on, so most of the gain comes from the encoder learning better features rather than from the predictor learning better dynamics. See project page at https://sg-jepa.github.io.

Andy Zeyi Liu, Haoran Sun, Lucas Baker +2

Yale University · Jump Trading · Brown University

World Models for Robotics Latent Dynamics Modeling Joint-Embedding Predictive Architecture

Sep 9, 2026stat.ME

Dynamical Non-compensatory Multidimensional IRT Model Using Variational Approximation

Multidimensional item response theory (MIRT) is a statistical test theory that precisely estimates multiple latent skills of learners from the responses in a test. Both compensatory and non-compensatory models have been proposed for MIRT: the former assumes that each skill can complement other skills, whereas the latter assumes they cannot. This non-compensatory assumption is convincing in many tests that measure multiple skills; therefore, applying non-compensatory models to such data is crucial for achieving unbiased and accurate estimation. In contrast to tests, latent skills will change over time in daily learning. To monitor the growth of skills, dynamical extensions of MIRT models have been investigated. However, most of them assumed compensatory models, and a model that can reproduce continuous latent states of skills under the non-compensatory assumption has not been proposed thus far. To enable accurate skill tracing under the non-compensatory assumption, we propose a dynamical extension of non-compensatory MIRT models by combining a linear dynamical system and a non-compensatory model. This results in a complicated posterior of skills, which we approximate with a Gaussian distribution by minimizing the Kullback-Leibler divergence between the approximated posterior and the true posterior. The learning algorithm for the model parameters is derived through Monte Carlo expectation maximization. Simulation studies verify that the proposed method is able to reproduce latent skills accurately, whereas the dynamical compensatory model suffers from significant underestimation errors. Furthermore, experiments on an actual data set demonstrate that our dynamical non-compensatory model can infer practical skill tracing and clarify differences in skill tracing between non-compensatory and compensatory models.

Hiroshi Tamano, Daichi Mochihashi

the graduate university for advanced studies, sokendai · the institute of statistical mathematics

Latent Dynamics Modeling Knowledge Tracing Item Response Theory

Aug 31, 2026cs.LG

Geometry-aware Latent Autoregressive Generative Model for PDEs in Complex Domains

Solving multiphysics partial differential equations (PDEs) remains a major challenge in scientific computing, especially for highly complex

μ

m-scale tortuous geometries critical to energy and chemical engineering. We address this challenge by proposing a Geometry-aware Latent Autoregressive generative Model for PDEs (GeoLAMP), which solves physics within highly irregular and tortuous structures by decoupling flow and transport physics. GeoLAMP introduces a dual-encoder architecture on graph representations to jointly capture global topology and fine-scale geometric features, enabling an effective transition from real-space fields to compact latent representations. In the latent space, we propose a causal self-attention transformer with flow matching to model temporal dynamics, allowing stable and scalable block-wise autoregressive prediction. In addition, we propose a grid-graph data fusion scheme that projects low-resolution grid-based approximate priors onto graph representations, improving prediction of flow in tortuous structures. We establish three multiphysics benchmark datasets in complex geometries, covering reactive flow, heat convection, and elasticity. GeoLAMP consistently achieves the most stable autoregression performance on these datasets. Our results provide a systematic study of geometry-aware learning for PDEs in

μ

m-scale complex geometries and offer new insights into block-wise time marching of latent autoregressive PDE modeling via a flow matching framework.

Zi Wang, Minghui Xu, Tapan Mukerji

Department of Energy Science and Engineering Stanford University Stanford, CA 94305

Graph Neural Networks Neural PDE Solvers Latent Dynamics Modeling PDE Surrogate Modeling Autoregressive Generation

Aug 29, 2026cs.LG

Flow-JEPA: Robust Latent Dynamics for JEPA World Models via Flow Matching

Joint-Embedding Predictive Architectures (JEPAs) provide a powerful framework for latent world modeling and planning in a reconstruction-free manner. Although numerous JEPA-based approaches have been proposed to mitigate representation collapse, our experiments on localized, out-of-distribution visual noise reveal that performance degradation remains pronounced and unresolved. We propose Flow-JEPA (F-JEPA), a flow-based latent dynamics model that jointly generates a sequence of future latent states conditioned on the current observation and actions. A Gaussian distribution serves as the flow source, exposing the vector field to perturbed latent trajectories as it learns to transport them toward clean future representations. This formulation retains the reconstruction-free JEPA framework while switching from pointwise transition regression to stochastic trajectory-level prediction. F-JEPA raises mean success from

86\%

to

92\%

under clean observations and from

67\%

to

86\%

under noisy conditions. Further evaluations over varying perturbation severity and inference settings show that the performance advantage persists across a broad range of conditions. These results suggest that conditional flow matching provides a promising alternative to deterministic autoregressive prediction as a dynamics formulation in JEPA world models.

Yanchen Huo, Ziying Song, Yadan Luo

Nanyang Technological University · The University of Queensland

Flow Matching Latent Dynamics Modeling Joint-Embedding Predictive Architecture Latent World Models Flow-Based Generative Modeling

Aug 17, 2026cs.CV

LaGSplat: Inferring Physics-Governed Interactive Simulation from Monocular Video Using Latent Lagrangian Gaussian Splatting

We present LaGSplat (Latent Lagrangian Gaussian Splatting), a framework that infers interactive, physics-governed dynamics from one or a few monocular videos. At inference it lets a user push on the filmed object, rigid or deformable, with an external force that was never measured, annotated, or seen during training. This is possible because a low-dimensional latent state

\mathbf{q} \in \mathbb{R}^d

plays two roles at once: it is the generalised coordinate of a learned dissipative Lagrangian and the conditioning variable of a Gaussian Splatting decoder. The inductive bias of this decoder, whose primitives are explicit points

μ_i(\mathbf{q})

that move with the object, is what lets a force

f

applied in the image pull back into a latent generalised force

J(\mathbf{q})^\top f

and enter the equations of motion, which pixel-space (CNN) or neural-field (NeRF) decoders cannot do. We validate LaGSplat on test cases of increasing difficulty, from rigid to deformable and from autonomous to forced real systems, combining monocular video and sensor measurements. We further demonstrate interactive use: forces of arbitrary magnitude and direction can be applied to the reconstructed object at any time, its response rendered in real time, in 2D or 3D. Assuming a dissipative Euler-Lagrange equation over a few generalised coordinates trades generality for a bounded, plausible response to unseen forces, where an unconstrained predictor diverges.

Louen Pottier

Université Paris-Saclay, CEA, List, F-91120 Palaiseau, France

3D Gaussian Splatting Latent Dynamics Modeling Dynamic Scene Reconstruction Physical Reasoning in Video Deformable Object Simulation

Aug 13, 2026cs.LG

History-informed Lagrangian Neural Networks

Forecasting the long-horizon evolution of mechanical systems from position-only observations is a pivotal yet difficult task, as hidden velocities and trajectory-specific physical properties must be inferred simultaneously. Although physics-guided neural networks like Lagrangian Neural Networks (LNNs) guarantee physical plausibility, they generally require complete state inputs and lack adaptability to changing system parameters. To break these limitations, we introduce History-informed Lagrangian Neural Networks (HiLNN). Grounded in the insight that temporal position sequences implicitly encode underlying dynamics, HiLNN employs a recurrent encoder to extract a latent context from history. This context not only reconstructs the unobserved initial velocity but also adaptively modulates the mass matrix, potential energy, and damping coefficients of a structured Lagrangian system. By leveraging a differentiable RK4 rollout scheme, the entire pipeline is optimized end-to-end under multi-step trajectory supervision and energy-consistency regularization. Empirical evaluations across conservative, dissipative, and heterogeneous variable-parameter systems show that HiLNN delivers superior long-term prediction accuracy and maintains precise energy profiles compared to state-of-the-art baselines. The source code is publicly available at https://github.com/yingtian22/History-informed-LNN.

Tianshuo Zhang, Xianglei Xing, Wenzhe Zhai +2

College of Intelligent Systems Science and Engineering, Harbin Engineering University, Harbin 150001, China

Latent Dynamics Modeling Physics-Informed ML

Aug 10, 2026cs.CV

Learning How the World Evolves: Extrapolative Video World Models via Latent Dynamics Reasoning

The world evolves following its dynamics, i.e., its laws of motion. However, leading video diffusion models largely fit the pixels without modeling how the pixels transit over time. Thus, they render visually plausible frames but may not accurately obey the laws. To capture the dynamics purely from pixels, we introduce Latent Dynamics Reasoning (LDR). LDR casts the latent transition as an explicit kinematic integration, where the lower-order dynamics are integrated numerically and the model regresses only the third- and higher-order residual that drives the rollout. For this integration to extrapolate better, LDR runs it on a structured latent rather than dense convolutional features. Following PhyWorld, we validate LDR on a controlled white-box physics benchmark spanning five tasks (uniform motion, parabola, collision, bouncing, looming), focusing on out-of-distribution scenarios that reveal whether a model has truly learned the underlying dynamics. LDR extrapolates the learned dynamics far better: the gap between its in- and out-of-distribution error is over 20

\times

smaller than the video diffusion baseline's, under both single- and joint-task training at 256

^2

resolution, while using 26

\times

fewer parameters and running 143

\times

faster. LDR can even generalize under severe shift: for example, trained only on red balls moving left-to-right, it correctly predicts the motion of a blue square moving right-to-left. To our knowledge, this is the first video world model that extrapolates learned dynamics beyond its training distribution. Project page: https://lat-dyn-reason.github.io/

Haodong Li, Shaoteng Liu, Tianyu Wang +7

1UCSD 2Adobe

Physical Consistency in Video Generation Physical Reasoning Latent Dynamics Modeling OOD Generalization Video World Models

Aug 10, 2026cs.RO

SLIM-0.5B: Learning Action-Grounded Predictive Latents for Robot Manipulation

Vision-language-action policies rely on large multimodal backbones to jointly perform perception, language conditioning, and action generation at every control step. Much of this capacity supports open-domain semantics, whereas continuous robot manipulation primarily requires compact representations of observations, actions, and the transitions induced by actions. Pixel-level world models provide another route, but predicting visual details irrelevant to control can be unnecessarily expensive. We propose SLIM (Self-supervised Latent Interaction Model), a compact 0.5B-parameter latent interaction policy. SLIM learns action-grounded predictive latents that capture both action-conditioned future transitions and the actions that explain observed changes. SLIM learns these representations through self-supervised masked trajectory prediction, combining action reconstruction with future-latent prediction. A compact Mixture-of-Transformers (MoT) backbone models interactions between observation latents and action tokens. The resulting policy is trained with flow matching for language-conditioned action generation. Across simulation benchmarks and real-world evaluation, SLIM matches or exceeds representative large-scale VLA and world-action-model baselines with fewer parameters, no additional embodied pretraining, lower inference latency, and substantially lower GPU memory usage.

Jingkai Wang, Zihan Tang, Gu Zhang +7

1Fudan University · 2Beijing Academy of Artificial Intelligence · 3Tsinghua University +1

Latent Dynamics Modeling Language-Conditioned Robot Manipulation Vision-Language-Action Models Latent World Models

Aug 9, 2026cs.LG

Path-dependent Discrete Amortized Inference

We consider the problem of sampling compositional and discrete objects from a given unnormalized posterior distribution. Notably, recent studies have shown that this problem can be efficiently solved by learning a deterministic Markov Decision Process (MDP) that progressively builds each object in proportion to the posterior. In this work, however, we demonstrate that the Markovian assumption can both hamper signal propagation during training and catastrophically reduce the learned sampler's expressivity due to state aliasing. To address these issues, we propose lifting the MDP with a learnable latent dynamical system that allows the underlying policy to depend on the entire past trajectory---and not only on the current state. In view of this, we refer to the resulting method as path-dependent discrete amortized inference. Importantly, we provably extend existing learning algorithms for discrete amortized samplers to our setting. In experiments on standard benchmark problems, we also show that our approach often leads to faster learning convergence and improved state space exploration relatively to prior techniques.

Tiago da Silva, Esmeralda S. Whitammer, Salem Lahlou

1MBZUAI · University of Edinburgh · 3CIFAR Fellow

Markov Decision Processes Latent Dynamics Modeling Amortized Inference Posterior Sampling

Aug 6, 2026cs.LG

Flowing Through States: Neural ODE Regularization for Reinforcement Learning

Neural networks applied to sequential decision-making tasks typically rely on latent representations of environment states. While environment dynamics dictate how semantic states evolve, the corresponding latent transitions are usually left implicit, creating a potential misalignment between the two. We propose to model latent dynamics explicitly by drawing an analogy between Markov decision process (MDP) trajectories and ordinary differential equation (ODE) flows: in both cases, the current state fully determines its successors. Building on this view, we introduce a neural ODE-based regularization method that enforces latent embeddings to follow consistent ODE flows, thereby aligning representation learning with environment dynamics. Although broadly applicable to deep learning agents, we demonstrate its effectiveness in reinforcement learning by integrating it into Actor-Critic algorithms. Our approach yields major performance gains across various standard Atari benchmarks for A2C and gridworld environments for PPO.

Mohamed Ghanem, Bernd Finkbeiner

CISPA Helmholtz Center for Information Security · Technical University of Munich

Latent Dynamics Modeling Representation Learning for RL State Representation Learning Neural ODEs

Aug 5, 2026cs.LG

Quantum-Structured World Models (QSWMs) for Predictive Latent Dynamics

World models learn latent states that summarize interaction histories, evolve over time, and support prediction, simulation, or planning. Most existing world models represent these states using classical vectors, probability distributions, recurrent hidden states, or transformer activations. In this paper, we introduce Quantum-Structured World Models (QSWMs), a quantum-inspired framework for predictive world modeling with structured latent states, latent transition operators, and measurement-inspired decoding maps. We study whether mathematical structures inspired by quantum theory, such as complex-valued representations and density-matrix-like latents, provide useful inductive biases for world modeling. We establish three foundational properties: classical inclusion, predictive sufficiency, and structured compactness. We then instantiate complex-valued and density-matrix-like QSWM variants and evaluate them on elementary cellular automata against strong classical baselines. Results show promising local predictive potential for complex-valued QSWMs, while also revealing limitations in long-horizon rollout, density-matrix variants

Hailong Jiang, Emran Hossain, Feng Yu +3

Youngstown State University, Youngstown, OH, USA · Kent State University · George Washington University

Latent Dynamics Modeling Quantum-Inspired ML Latent World Models

Aug 5, 2026cs.LG

Beyond Linear Dynamics: Neural Bilinear Dynamical Models for Time Series Forecasting

Time series in real-world applications are often generated by nonlinear dynamical systems, making accurate forecasting challenging. Existing approaches that explicitly model system dynamics typically rely on linear assumptions or Koopman-based linearizations, which may inadequately capture complex nonlinear behaviors and lead to error accumulation in long-horizon prediction. To address this limitation, we propose the Neural Bilinear Dynamical Model (NBDM), which models nonlinear system dynamics through a bilinear latent dynamical formulation. Specifically, NBDM leverages Koopman theory to lift the original nonlinear dynamics into a higher-dimensional latent space, where a bilinear dynamical model is constructed to characterize state evolution. To mitigate the approximation error introduced by bilinear representations, we further incorporate a parameterized error compensation term. Within this formulation, control inputs are explicitly integrated into the dynamics, using auxiliary variables when available and learned feedback signals otherwise. To handle scenarios with missing control inputs, we design a memory-enhanced controller that infers latent controls through multiplicative interactions between historical states and control signals. Experiments on five real-world datasets demonstrate that NBDM consistently outperforms competitive baselines in both given-control and missing-control settings, particularly for multi-step and long-horizon forecasting.

Mengzhou Gao, Huangqian Yu, Pengfei Jiao

Hangzhou Dianzi University Hangzhou, China

Dynamical Systems Long-Term Time Series Forecasting Latent Dynamics Modeling Time Series Forecasting

Aug 4, 2026cs.LG

SJEPA: Learning Elegant Latent Dynamics with Hybrid Symbolic-Neural Predictors

Joint-embedding predictive architectures learn abstract states by predicting target embeddings from context embeddings, but their transition models are typically opaque neural maps. We introduce SJEPA, a reconstruction-free JEPA framework that learns predictive representations whose induced dynamics admit compact symbolic descriptions. Its hybrid transition combines a symbolic law with a regularised neural correction for dynamics outside the selected grammar. The central principle is to learn the simplest adequate dynamics: representation constraints preserve informative, non-collapsed predictive coordinates, while operator compression favours low-complexity symbolic-neural transitions that remain predictively adequate. We formalise this principle through induced-dynamics complexity, analyse predictive-coordinate non-identifiability, and show that unconstrained operator compression creates a direct shortcut to representation collapse. The framework supports both alternating representation-equation learning and symbolic dynamics fitted to fixed representations. In controlled pendulum experiments, joint learning discovers substantially simpler symbolic dynamics with lower long-horizon rollout error and divergence than post-hoc fitting, while an unconstrained one-step diagnostic realises the predicted collapse shortcut. Under grammar misspecification, correction regularisation preserves the representable symbolic mechanism and directs the neural component towards residual dynamics. The results expose a controllable trade-off among predictive fidelity, representation quality, symbolic parsimony, and symbolic-neural allocation.

Yongchao Huang

Latent Dynamics Modeling Joint-Embedding Predictive Architecture Predictive Representation Learning Neurosymbolic Learning

Aug 3, 2026cs.LG

WorldDynCache: Risk-Controlled Latent Dynamics Approximation for Diffusion World Model

Diffusion world models generate high-quality futures, but re- peated transformer evaluations make inference prohibitively slow. Existing caches reuse intermediate features, selectively update tokens, or reuse and extrapolate denoising outputs ac- cording to local drift or short native-space histories. These criteria can miss both approximation-induced latent transition defects that accumulate across skipped steps and phase- or condition-dependent changes in the direction of latent evo- lution. We propose WorldDynCache, a risk-controlled latent dynamics approximation framework with two core compo- nents. First, a lightweight latent-transition risk estimator tracks the accumulated future impact of approximation defects and calibrates its predictions against counterfactual defects ob- served at exact anchors. Second, a condition- and phase- aware lifted latent surrogate approximates latent evolution without extra transformer evaluations. On HunyuanVoyager- 13B and Aether-5B, WorldDynCache achieves 4.92 times and 2.15 times speedups, respectively, while attaining the best gen- eration quality among the compared caching methods across WorldScore, PSNR, SSIM, and LPIPS.

Leyang Chen, Junyi Wu, Shaoqiu Zhang +1

Shanghai Jiao Tong University

World Model Learning Latent Dynamics Modeling Diffusion Model Caching Diffusion Model Inference Acceleration

Aug 3, 2026stat.ML

The Label Defines the Timescale: Trait-State Limits of Temporal-Aggregate Learning

Machine-learning benchmarks often pair a label that aggregates a long temporal horizon with input observed through one or a few short windows. Their apparent performance ceiling may therefore be an acquisition-protocol ceiling rather than a model-capacity ceiling. We study labels of the form

Θ_{g,T}=T^{-1}\int_0^T g\{Z(t)\}\,\mathrm{d}t

when the latent Gaussian process contains both a stable individual trait and a correlated within-individual state. An exact protocol-conditioned Bayes-risk identity provides a common tool. First, we decompose label variance into an

O(1)

trait component and an

O(T^{-1})

state component, explaining why a snapshot can retain cross-sectional predictability while poorly tracking within-person change. Second, we derive task-dependent effective temporal spans: mean labels depend on the ordinary correlation time, whereas occupation-time labels depend on an entire spectrum of higher-order correlation times. Third, state-driven occupation-label variance is maximal when the stable trait lies at the threshold; window efficiency decays much more slowly away from that boundary. Under an equal segment budget, exact risks and Monte Carlo experiments show that repeated segments at one time rapidly saturate, whereas temporally dispersed observations continue to increase state explainability. The trait ceiling uses quantities available from ordinary test-retest data; only the state ceiling requires short-lag temporal calibration. The results distinguish architectural limits from protocol limits and show that the label, rather than duration or segment count alone, defines the relevant timescale.

Xizhe Zhang

Latent Dynamics Modeling State Tracking Statistical Learning Theory

Jul 31, 2026cs.LG

Latent Lie-Poisson Neural Networks (LLPNNs): Discovering the motion of Lie-Poisson systems through observable data and latent dynamics

Structure-preserving neural networks are essential for the long-term prediction of Hamiltonian systems from data. Many important Hamiltonian systems in mechanics and control admit symmetry reduction to Lie--Poisson systems, including rigid bodies, underwater vehicles, fluids, plasmas, and optimal control problems. A fundamental challenge in learning such systems is that their dynamics evolve in momentum variables that are typically unobservable, while available data consist only of observable quantities such as configurations and velocities. In optimal control applications, the situation is further complicated because the latent variables contain unobservable co-states and the Hamiltonian may be degenerate, preventing the existence of a corresponding Lagrangian and rendering the encoder-decoder approaches inapplicable. We introduce Latent Lie--Poisson Neural Networks (LLPNNs), a structure-preserving framework for learning Lie--Poisson dynamics directly from observable data. The proposed approach exploits three geometric ingredients: (i) learning either a Hamiltonian decoder or a pseudo-Lagrangian encoder on the active variables, (ii) constructing latent trajectories through a universal Noether invariant arising from Lie--Poisson symmetry reduction, and (iii) reconstructing observable and latent dynamics through Lie--Poisson flows combined with Magnus-based Lie-group updates. The resulting method preserves the geometric structure and is applicable to both regular and degenerate Hamiltonian systems. We demonstrate the method on three examples: a generalized rigid body on SO(3), Kirchhoff's underwater vehicle on SE(3), and an optimal-control problem for interacting vehicles on

SE(2)^N

. Numerical experiments show excellent long-term predictive accuracy, strong robustness to noise, and competitive performance using only modest datasets and lightweight neural-network architectures.

Vakhtang Putkaradze

Department of Mathematics, The University of Alabama, Gordon Palmer Hall, Tuscaloosa, AL, 35401, USA

Latent Dynamics Modeling Hamiltonian Neural Networks Hamiltonian Dynamics

Jul 30, 2026cs.LG

ODEWorld: A Continuous Predictive Architecture via Physical-Time Flow

In the physical world we inhabit, space and time are fundamentally continuous. However, existing machine learning paradigms for world modeling are largely confined to discrete-time prediction, thereby exhibiting significant inefficiency in capturing the dynamics of physical world. We introduce Physical-Time Flow (PT-Flow), a novel approach that learns a continuous latent velocity field operating in physical time. Crucially, the underlying dynamics of sequential data are parameterized by an ordinary differential equation (ODE) embedded in a well-structured representation space. Under this paradigm, the prediction of future can be recast as temporal integration via an ODE solver in the compressed latent space. Building upon PT-Flow, we construct ODEWorld, a continuous-time latent world model that is both efficient and versatile. By extracting time-variant features and enforcing ODE properties on both the dynamical representation space and the latent velocity field, ODEWorld effectively addresses the long-standing representation collapse issue in latent world model literature. This also enables high-quality image reconstruction even after long-horizon prediction. Moreover, its continuous nature allows for arbitrary temporal resolution and even backward prediction, which is impossible for most discrete-time models. Lastly, ODEWorld can provide rich planning-oriented information to facilitate downstream policy learning. Comprehensive experiments demonstrate that ODEWorld successfully reconciles planning-conducive dynamics abstraction with visual realism, excelling in both video generation and robotic control. Project page: https://odeworld.github.io/.

Dongxiu Liu, Haoyi Niu, Peng Cheng +5

Institute for AI Industry Research (AIR), Tsinghua University, Beijing, China · 2Berkeley Artificial Intelligence Research (BAIR), University of California, Berkeley, CA, USA

Robotic Control Latent Dynamics Modeling World Models Video Prediction Latent World Models

Jul 29, 2026cs.RO

DLAM: Distributional Latent Actions with Temporal Constraints

Vision-language-action (VLA) models remain constrained by scarce action-labeled robot data, whereas action-free videos offer abundant observations of physical change. Latent action models can extract such priors, but reconstruction-trained codes may predict future observations without the structure required for joint generation with robot actions. Existing structured methods add temporal constraints but retain deterministic transition points, so residual errors in locally inferred transitions may propagate and compound under recursive composition. We introduce DLAM, a distributional latent-action model that represents each transition as a diagonal Gaussian. Reconstruction conditioned on the reference frame grounds the mean in observed visual change, while normalized composition and reversal over equal-gap triplets constrain both the mean and dimension-wise variance. Variance composition uses a lightweight shared-correlation coefficient to account for dependence between adjacent transitions that share an intermediate frame, whereas reversal negates the mean and preserves the variance. For downstream policy learning, we freeze the encoder and train a flow-matching policy to jointly generate mean transition sequences and robot actions. On held-out transitions, DLAM learns more temporally consistent latent dynamics than existing latent-action baselines and achieves stronger direct and cumulative reconstruction on held-out videos. Under the same controlled

π_0

transfer protocol, it also improves policy performance on MetaWorld MT50, LIBERO, and real-world manipulation tasks. Controlled ablations show that normalized mean constraints account for most of the reconstruction gain, while learned variance and correlation-aware composition provide complementary improvements in downstream control.

Zuojin Tang, Feifan Luo, Haoyun Liu +10

Zhejiang University · Nanjing University · Shanghai Jiaotong University +4

Latent Action Learning Latent Dynamics Modeling Visuomotor Policy Learning Vision-Language-Action Models

Jul 27, 2026physics.flu-dyn

The balance between compactness and forecast accuracy of data-driven latent-space reduced-order models in controlled wake flows

Model-based active flow control requires predictive models that are accurate, stable, and fast enough for real-time optimisation. In controlled wake flows, this is often achieved through Reduced-Order Models (ROMs) that first compress high-dimensional velocity snapshots into a latent space and then learn a time- stepping predictor for the dynamics in the latent space. Here, we study how the choice of the spatial encoder affects the predictability of the resulting latent coordinates for wake flows under control inputs. Using two actuated 2D wake configurations, a simplified truck wake and the fluidic pinball, we compare Proper Orthogonal Decomposition (POD) against nonlinear Convolutional Autoencoders (CAEs) and two types of variational autoencoders for compression, and evaluate several temporal predictors based on Long Short-Term Memory networks. CAEs achieve higher compression efficiency and sharper short-term reconstructions, but they produce latent dynamics that are more irregular and with broadband spectral content. As a consequence, long-horizon forecasts degrade faster and show a higher probability of catastrophic divergence than POD-based models. POD yields smoother latent trajectories that are easier to learn and extrapolate, leading to more reliable predictions beyond the short- term regime. These results reveal a clear trade-off between compactness and forecast accuracy, and suggest that the stability of the latent dynamics prediction can outweigh maximal compression. This is particularly relevant for control strategies rooted in forecasts of the dynamics, such as model predictive control and reinforcement learning. The findings provide practical guidance for designing actuation-aware, hardware-feasible predictive ROMs for real-time flow control.

Alberto Solera-Rico, Patricia García-Caspueñas, Carlos Sanmiguel Vila +1

Sub-directorate general of aeronautical systems, Spanish National Institute for Aerospace Technology (INTA), ctra. M-301, km. 10.5, 28330 San Mart´ın de la Vega, Madrid, Spain. · Departamento de Ingenier´ıa Aeroespacial, Universidad Carlos III de Madrid, Avenida de la Universidad 30, 28911 Legan´es, Madrid, Spain. · Department of Mechanical Engineering, University of Washington, 1410 NE Campus Parkway, Seattle, WA 98195, USA.

Variational Autoencoders Latent Dynamics Modeling Autoencoders Proper Orthogonal Decomposition Convolutional Autoencoder Feedback Control+2

Jul 27, 2026cs.SD

Modeling Stylistic Co-evolution in Symbolic Music Heritage Collections

Digitized musical heritage collections offer new opportunities to examine how stylistic traditions change over historical time, but computational analyses often reduce musical works to static classifications or similarity scores. This article proposes a representation-to-dynamics framework for studying cross-cultural harmonic change in Western art music. Starting from symbolic chord sequences, we derive contextual chord embeddings, project work-level representations into a shared harmonic space, and reconstruct country-level trajectories through temporally causal Kalman filtering. These trajectories are then modeled with DeGroot and Friedkin--Johnsen dynamics, yielding interpretable influence-like networks and estimates of stylistic anchoring. We apply the framework to a curated corpus of 480 dated works from 1875 to 1940 across Russia, France, Germany, Austria, and a heterogeneous ''Others'' group. Models fitted on 1875--1925 are evaluated through recursive forecasts for 1930--1940, testing whether the estimated dependency structure remains informative across a potentially changing historical and stylistic regime. The estimated trajectories and influence patterns are broadly consistent with established music-historical accounts of late Romantic and early modernist exchange, including the close relationship between German and Austrian traditions and historically plausible cross-currents between Russian and French traditions. PCA-variance-weighted estimation provides modest improvements while preserving a single interpretable influence network. Rather than treating the estimated matrices as direct causal evidence, the framework offers a reproducible quantitative layer for cultural heritage research, complementing archival and musicological interpretation.

Yulong He, Ivan Smirnov, Yanming Li

St. Petersburg State University, St. Petersburg, Russia · ITMO University, St. Petersburg, Russia · St. Petersburg State Institute of Culture, St. Petersburg, Russia

Latent Dynamics Modeling Cultural Evolution Music Information Retrieval Cultural Heritage

Jul 25, 2026cs.LG

Neural operator discovery from heterogeneous trajectories

Neural operators provide data-driven mappings for modeling dynamical systems. Extending them to families of systems typically requires explicit conditioning variables such as physical parameters, geometries, or boundary conditions. In many real-world settings, these quantities are unobserved. Here, we formulate neural operator discovery (NOD) as the problem of learning both shared solution operators and system-specific variation directly from heterogeneous trajectories without access to labeled governing factors. We introduce a factorized latent-conditioning formulation that jointly learns a neural operator and a low-dimensional latent representation through factorized prediction, trajectory-decoupled sampling, and dimension selection. Across diverse systems, the learned latent representation captures the intrinsic dimensionality of system variation and organizes system instances in a smooth and approximately invertible latent structure aligned with the underlying governing factors. This organization enables generalization to previously unseen system instances, including zero-shot extrapolation across regimes and stable long-horizon prediction. These results establish an interpretable paradigm for operator learning in the absence of explicit factor supervision.

Zituo Chen, Qiaofeng Li, Jiaxin Hu +1

Department of Mechanical Engineering, Massachusetts Institute of Technology, 77 Massachusetts Avenue, Cambridge, 02139, MA, USA. · State Key Laboratory of Fluid Power & Mechatronic Systems, Zhejiang University, 866 Yuhangtang Road, Hangzhou, 310058, Zhejiang Province, China.

Latent Dynamics Modeling Nonlinear System Identification Operator Learning Neural Operators

Jul 25, 2026cs.CV

Inverse Bayesian Inference for Extracting Lesion Dynamics from Longitudinal Spectral CT

Longitudinal medical imaging captures temporal evolution of lesions, yet extracting the underlying dynamical parameters governing this evolution remains challenging. We propose an inverse Bayesian framework for inferring lesion dynamics from longitudinal spectral CT. We decompose spectral feature (

x

) evolution into three components: \begin{equation*} \frac{dx_i}{dt} = A_i x_i + B \cdot n + C \cdot Δx_{\text{sat}} \end{equation*} where

A_i

captures intrinsic dynamics (lesion-autonomous evolution),

B

captures local environment tumour burden (organ tumour burden through satellite count coupling), and

C

captures environment/satellite state change (i.e., whether surrounding lesions move similarly or not). We demonstrate the framework on photon-counting NSCLC CT data from metastases, recovering distinct dynamical regimes: lung lesions exhibit significant satellite count coupling (

B=-0.34

,

p<0.05

) suggesting competitive dynamics, while liver lesions show synergistic satellite behaviour coupling (

C\approx+1.0

,

p<0.05

). Synthetic validation confirms parameter recovery, and cross-coupling analysis validates that our method detects non-zero coupling when present. This work establishes inverse dynamical inference as a principled methodology for extracting interpretable parameters from longitudinal imaging, moving beyond static feature extraction toward mechanistic characterisation of lesion behaviour. The code and data are available at: https://github.com/lukasf98/inverse-bayesian-inference

Lukas Förner, Melina Wördehoff, Julian Steffens +6

Dept. of diagnostic and interventional Radiology and Neuroradiology, University Hospital Augsburg, Germany · Digital Medicine, University Hospital Augsburg, Germany · Chair for Computer-Aided Medical Procedures and Augmented Reality, Technical University of Munich, Germany +5

Latent Dynamics Modeling Bayesian Inverse Problems Longitudinal Medical Image Analysis Oncology

Jul 24, 2026cs.LG

On the Identifiability of Controlled World Models

World model serves as a promising tool to infer environment dynamics under high-dimensional observations and candidate actions. Recently, LeCun's JEPA provides a compelling framework for learning such models in representation space. Its action-conditioned extension plays a central role in visual control and latent-space planning, but leaves a fundamental question: can it recover the controlled dynamics from nonlinear observations? This paper presents a joint identifiability condition for controlled world models with Gaussian latent states, which consists of two coupled components: (1) representation identifiability and (2) transition identifiability. The former depends on the spectral separation property while the latter is related to non-degenerate variation of conditional action. We prove that when this condition holds, minimizing the LeJEPA-style predictive objective can recover both latent states and controlled dynamics in the sense of orthogonal transformation. We further prove that the upper bound of transition prediction error is inversely proportional to the spectral separation margin. We also characterize an attainable amplification of counterfactual prediction error that scales inversely with the weakest conditional action-excitation margin. The theoretical predictions are empirically supported across four nonlinear observation settings.

Xiangteng Zhang, Yang Guan, Bo Zhang +3

School of Vehicle and Mobility, Tsinghua University · 2Didi Voyager Labs, Didi Autonomous Driving · School of Computing and Data Science, The University of Hong Kong +2

Parameter Identifiability Latent Dynamics Modeling Action-Conditioned World Models State Representation Learning

Jul 24, 2026stat.ML

Variational Low-rank Tensor Decomposition for Multisubject Spatiotemporal Data Analysis

Modeling shared and subject-specific structure in multisubject spatiotemporal data remains challenging, particularly in neuroimaging, where both spatial and temporal patterns exhibit rich variability across subjects. Existing matrix and tensor decompositions provide interpretable factorizations, but rely on fixed multilinear structures or coupling schemes that may limit their flexibility in capturing complex variability. In this work, we introduce a spatiotemporal variational tensor decomposition (ST-VTD) framework that combines a tensor factorization generative model with structured priors to jointly represent spatial maps and temporal dynamics. Spatial factors are regularized to promote a low-rank structure inspired by the LL1 decomposition, while temporal factors are modeled using a learned Long short-term memory (LSTM)-based prior, enabling flexible and adaptive dynamics. Posterior inference is performed using an amortized variational formulation by unrolling iterations of an optimization algorithm, leading to an interpretable and parameter-efficient architecture. The proposed inference framework employs a warm-start strategy based on group independent component analysis, which we found to improve optimization performance. Experiments on a realistic synthetic functional MRI (fMRI) dataset demonstrate that the proposed approach significantly improves latent factor recovery compared with representative classical and probabilistic decomposition benchmarks.

Laura M. Montaldo, Ricardo A. Borsoi, Sebastian Miron +1

CRAN, Université de Lorraine, CNRS, Vandoeuvre-l`es-Nancy, France. · Department of CSEE, University of Maryland Baltimore County, Baltimore, MD 21250, USA.

Latent Dynamics Modeling Low-Rank Matrix Decomposition Tensor Decomposition Variational Inference

Jul 22, 2026cs.LG

Koopman Dreamer: Spectrally Constrained Latent Dynamics for Stable World-Model Imagination

Latent world models improve sample efficiency in continuous control by optimizing policies over imagined latent trajectories, but common neural transitions offer limited direct control over modal persistence and error accumulation in long rollouts. We propose Koopman Dreamer, a Dreamer-style world model with a spectrally constrained deterministic latent dynamics core. Its Koopman-inspired backbone uses two-dimensional rotation--scaling blocks with bounded radii to represent damping, rotation, and near-periodic modes. Linear and low-rank bilinear action terms capture global and state-dependent control effects, while stochastic-state modulation supplies local correction information. To reduce the mismatch between posterior-conditioned training and prior-only imagination, the model combines posterior-conditioned EMA teacher targets with one-step consistency, multi-step rollout, and open-loop observation-prediction objectives. We further derive a multi-step rollout-error bound that separates amplification by the spectral backbone and bilinear interaction from the additive effects of stochastic-state mismatch and modeling residuals, clarifying the trade-off between error attenuation and long-term information retention. Experimental results on proprioceptive continuous-control tasks from the DeepMind Control Suite and UAV-LiDAR autonomous navigation demonstrate that Koopman Dreamer improves the stability of long-horizon latent rollouts and achieves stronger closed-loop control performance on tasks that rely on high-quality multi-step imagination.

Jiaqi Li, Xinglong Zhang, Haibin Xie +3

College of Intelligence Science and Technology, National University of Defense Technology, Changsha, China

Koopman Operator Learning Latent Dynamics Modeling World Models Latent World Models Model-Based RL

Jul 21, 2026cs.LG

Multi-Horizon Consistency as Geometry: When Latent Dynamics Contract, and When They Do Not

Multi-horizon latent consistency is a common training knob in video predictors and world models, but practitioners rarely know what it does to transition geometry. We treat lambda, the weight on multi-step latent agreement, as a diagnostic control and measure an empirical expansion proxy L20,q95 together with horizon-20 prediction error E20. On Moving-MNIST (n=6 seeds at the critical pair), raising lambda from 0 to 0.8 cuts L20 from 4.96 +/- 2.01 to 1.01 +/- 0.06 (paired t p=0.005, Wilcoxon p=0.031) and halves E20 (0.365 to 0.177, paired t p=1.1e-13). Four of six seeds cross L<1 at lambda=0.8. The same loss does not produce population L<1 on action-conditioned Pendulum-v1 or CartPole-v1, nor on KTH Actions video, even when E20 improves. An associational mediation analysis on MMNIST gives r-hat=0.94 (95% CI [0.88, 1.00], n=27, B=2000); lambda was not randomized. Defensive checks (architectural baselines, exogenous stress, WorldTest, MPC, scaling) mostly support a narrow claim: soft consistency can push passive video toward a near-contractive band, and that band is domain-limited. A stochastic-forcing law L20 ~ 1.23 + 1.82 eta at lambda=0.8 (bootstrap slope CI [1.73, 1.92], R^2=0.96) unifies control domains on the same curve via calibrated eta_eff. Complete joint slices at lambda in {0.4, 1.2} (30/30 cells, 5 eta x 3 seeds) show comparable linear L20(eta) slopes (~1.69 and ~2.00); we do not fit a continuous (lambda, eta) surface. We do not report DreamerV3 or TD-MPC2 returns.

Kavya Bhand, Aadi Joshi

Department of Computer Engineering Vishwakarma Institute of Technology, Pune, India

Latent Dynamics Modeling Latent World Models Consistency Regularization