Neural ODEs

ODE: Ordinary Differential Equation

Follow this topic Ask about this topic

Momentum

3 papers in the last four weeks, against 2 the four weeks before. 0.0% of all new papers.

Jul 13Week of Sep 28

Latest papers 67

Jun 7, 2026cs.CV

Learning to Solve Generative ODEs Beyond the Linear Span

Diffusion and flow generative models sample by integrating a learned ODE, but high quality still requires many sequential model evaluations. Solver learning reduces this cost by adapting scalar coefficients, timesteps, or both, while keeping the backbone model fixed. In this work, we identify a structural bottleneck in this update family: each step remains span-limited. Since the scalar-coefficient update lies in the span of buffered velocity evaluations, it can fit only the in-span component while leaving any out-of-span residual unreachable by scalar recombination alone. We propose SpanLift, a lightweight neural solver that augments scalar-coefficient updates with a spatial residual operator. SpanLift keeps a fixed base solver as an in-span prior and learns a spatial residual operator over the state and velocity buffer. The operator is trained by endpoint teacher matching, preserves the pretrained backbone, and adds no model NFEs. Empirically, the learned correction transfers across base solvers and is predominantly out-of-span. Across pixel-space diffusion, latent flow matching, and precipitation nowcasting, SpanLift achieves state-of-the-art few-step sampling. With only 3 NFE, it improves CIFAR-10 FID from 8.16 to 5.69 and ImageNet FID from 17.37 to 11.83.

Sihyeon Kim, Seunghun Lee, Vikas Singh +1

Korea University · KAIST · University of Wisconsin–Madison

Few-Step Diffusion Sampling Flow-Based Generative Modeling Neural ODEs Diffusion Model Inference Acceleration

Jun 4, 2026stat.ML

Function-Space Priors for Bayesian Neural ODEs with Application to Vessel Trajectory Prediction

Vessel trajectory prediction from Automatic Identification System (AIS) data is essential for maritime situational awareness, yet it remains challenging due to irregular sampling, missing reports, and complex dynamics. Beyond accurate point forecasts, maritime applications also demand well-calibrated uncertainty estimates for reliable decision-making. Bayesian Neural Ordinary Differential Equations (ODEs) offer a principled framework for continuous-time trajectory modeling with uncertainty quantification by placing a prior over the neural vector field parameters. However, the commonly used isotropic Gaussian weight prior fails to encode informative structural properties of vessel dynamics, such as smoothness and locality. Existing function-space Bayesian neural network methods address this limitation for static mappings, but do not transfer directly to Neural ODEs, where the primary quantity of interest is the trajectory rather than the vector field itself. In principle, one could place a Gaussian process (GP) prior directly over ODE solutions, but this requires propagating distributions through a nonlinear ODE solver, which is analytically intractable. To address this challenge, we adopt a practical approach that imposes a GP-kernel-based prior directly on the vector field evaluated at a finite set of measurement points. Specifically, we augment the standard weight-space variational objective with a kernel-based regularizer that penalizes deviations of the vector field from the structure implied by a GP prior. To handle long and irregular AIS trajectories, we further combine this function-space regularization with probabilistic multiple shooting, which decouples inference across temporal segments while maintaining global consistency.

Jaeyeong Lee, Wonmo Koo, Heeyoung Kim

aDepartment of Industrial and Systems Engineering, Korea Advanced Institute of Science and Technology (KAIST), Daejeon, Republic of Korea.

Bayesian Neural Networks Probabilistic Forecasting Vessel Trajectory Prediction Neural ODEs Trajectory Prediction

Jun 3, 2026cs.LG

Learning Manifold and Itô Dynamics with Branched Neural Rough Differential Equations

Neural rough differential equations (NRDEs) stay accurate under irregular sampling while taking far fewer integration steps than standard neural differential equations, summarising a finely sampled driver by its log-signature and advancing the hidden state over coarse intervals using the log-ODE method. This efficiency rests on the shuffle algebra, the algebraic counterpart of Stratonovich calculus. This reliance means NRDEs cannot expose the quadratic-variation terms Itô dynamics require, nor the ordered covariant derivatives that govern Itô flows on connection-equipped manifolds. Ameliorating this, we introduce Branched Neural Rough Differential Equations (B-NRDEs), a Hopf-algebraic framework that recasts the NRDE log-ODE step as geometric numerical integration on the state-space manifold, matching the driving algebra to the governing calculus: Grossman--Larson rooted trees for Euclidean Itô dynamics, Munthe-Kaas--Wright planar rooted trees for ordered covariant derivatives on manifolds, and the shuffle algebra in the classical Stratonovich case. This yields intrinsic coarse-step dynamics that exactly preserve manifold constraints. Finally, we introduce a branched signature-kernel objective to enable Itô-consistent law matching by making quadratic-variation terms visible during training. On rough Bergomi volatility, sim-to-real

\mathrm{SO}(3)

dynamics forecasting, and SPD covariance dynamics, B-NRDEs offer a unified, effective approach to stochastic and manifold-valued dynamics beyond the Euclidean--Stratonovich setting.

Luke Thompson, Dai Shi, Lequan Lin +2

University of Sydney, Australia · Cambridge University, United Kingdom.

Dynamical Systems Stochastic Differential Equations Neural ODEs

Jun 1, 2026cs.LG

Hybrid Neural Ordinary Differential Equations for Data-Efficient Polymerization Modeling with Incomplete Kinetics

Accurate prediction of polymerization dynamics is essential for process design, control, and optimization. Yet, purely mechanistic models require labor-intensive parameterization of partially characterized kinetics, while purely data-driven models demand large, diverse datasets that are costly to obtain, particularly in early-design stages. We propose a hybrid Neural Ordinary Differential Equation (NODE) framework for data-efficient modeling of free-radical polymerization. Using batch polymerization of methyl methacrylate (MMA) as a case study, the mechanistic mass balances are retained explicitly, and only the partially-characterized effective radical concentration governing monomer consumption is learned from data through a neural network surrogate, while established reactions such as initiator decomposition, propagation, and termination remain physically modeled. The hybrid NODE is evaluated against a discrete-time feedforward neural network and a purely data-driven NODE under sparse data conditions, with models trained on as few as ten measurements under both regular and irregular sampling. The hybrid NODE consistently achieves lower prediction errors and more physically consistent extrapolations than both purely data-driven baselines. In a generalization scenario with noisy data and unseen operating conditions, the hybrid NODE achieves an RMSE of 0.013, compared to 0.31 for the data-driven NODE and 0.68 for the discrete-time model, demonstrating that learning only a closure term rather than the full dynamics is sufficient for reliable prediction under limited data availability.

Marah Almanasreh, Alexander Mitsos, Eike Cramer

RWTH Aachen University, Process Systems Engineering (AVT.SVT), 52074 Aachen, Germany · JARA-CSD, 52425 Jülich, Germany · Energy Systems Engineering (ICE-1), Forschungszentrum Jülich, 52425 Jülich, Germany +1

Neural Surrogate Modeling Materials Science Neural ODEs Data-Efficient Learning

May 28, 2026cs.LG

Faithful Embeddings of Irregular and Asynchronous Data for Online Log-NCDEs

Continuous-time models are a natural choice for irregular and asynchronous data. A central design choice is how to embed discrete observations into continuous time. Interpolation- and imputation-based embeddings reconstruct a continuous observation path, making the model sensitive to the choice of reconstruction. We show that this reconstruction step is unnecessary; under mild conditions, compact-set universality on the model input space transfers to the data space whenever the embedding from data to input is continuous and injective. Guided by this result, and building on the rectilinear control path for Neural Controlled Differential Equations (NCDEs), we introduce a continuous and injective embedding for Log-NCDEs, a universal class of continuous-time models. Our approach records observations as increments and composes them over arbitrary query intervals to directly form log-signatures. This provides interval-level summaries without first interpolating the observed variables, while supporting online computation. Experiments on synthetic controlled dynamics and real-world time-series datasets show that the representation is accurate, efficient, and robust to irregular, asynchronous, and sparse observations.

Benjamin Walker, Alexandre Bloch, Lingyi Yang +2

Mathematical Institute University of Oxford · School of Mathematical Sciences University of Nottingham · Department of Mathematics, Imperial College London

Irregular Time-Series Modeling Neural Controlled Differential Equations Neural ODEs Time Series Representation Learning

May 27, 2026cs.LG

Universal Time Series Generation with Neural Controlled Differential Equations

Recent work on the sequence universality of State Space Models (SSMs) has introduced efficient, maximally expressive continuous-time approaches for time-series modelling. While these works focus on discriminative settings, we extend this perspective to generative time-series modelling by proving that maximally expressive Structured Linear Controlled Differential Equations (SLiCEs) are universal time-series generators, in the sense that they can approximate the induced path laws of continuous causal pushforwards on compact latent sets in

W_\infty

. Building on these theoretical results, we propose Generative SLiCEs (G-SLiCEs), a maximally expressive continuous-time model for flow matching on path-space. Empirically, we show that expressivity improves performance in probabilistic forecasting and downstream tasks, while retaining the advantages of continuous-time models such as generalising to arbitrary observation grids. This is particularly beneficial for irregular grids, where fixed-grid models often struggle.

Torben Berndt, Elyes Farjallah, Leif Seute +3

Heidelberg Institute for Theoretical Studies, Heidelberg, Germany · IAR, Karlsruhe Institute of Technology, Karlsruhe, Germany · Max Planck Institute for Polymer Research, Mainz, Germany +4

Neural Controlled Differential Equations Time Series Generation Neural ODEs Probabilistic Time Series Forecasting

May 22, 2026math.DS

Learning dynamical systems with biochemically informed neural ordinary differential equations

Ordinary differential equation models of biochemical reactions are often formulated as stoichiometric systems in which the dynamics arise from a collection of interacting processes. A central challenge is that the functional form of each process is rarely known a priori and may be difficult to infer from data. We propose biochemically informed neural ordinary differential equations (BINODEs), a neural-ODE framework that retains the stoichiometric structure of mechanistic models while representing individual processes by neural networks. In BINODEs, the outputs of neural network processes (NNPs) are mapped to state derivatives through a linear layer analogous to a stoichiometric matrix. This architecture allows biological side information, such as process-specific inputs, sign constraints, and monotonicity assumptions, to be built directly into the model. We characterize the approximation properties of NNPs for several standard biochemical rate laws and show that the proposed framework recovers both trajectories and process-level structure in Monod, Lotka--Volterra, pharmacokinetic, and ultradian endocrine models. These results suggest that BINODEs offer a useful compromise between mechanistic interpretability and data-driven flexibility for modeling partially known biochemical or biological dynamical systems.

Luis L. Fonseca, Reinhard C. Laubenbacher, Lucas Böttcher

Laboratory for Systems Medicine, Department of Medicine, University of Florida, Gainesville, FL, USA · Department of Computational Science and Philosophy, Frankfurt School of Finance and Management, Frankfurt am Main, 60322, Germany

Dynamical Systems Neural Processes Nonlinear System Identification Neural ODEs

May 22, 2026cs.LG

Learning partially observed systems with neural Hamiltonian ordinary differential equations

When learning dynamical systems from data, embedding physical structure can constrain the solution space and improve generalization, but many physics-informed models assume access to the full system state. This limits their use in partially observed settings, where some state variables are completely unobserved and must be inferred without direct supervision. Here, we present neural Hamiltonian ordinary differential equations (NHODE), a framework that combines Hamiltonian neural networks (HNNs) with neural ordinary differential equations (neural ODEs) to learn partially observed dynamical systems from data. The Hamiltonian structure enforces energy conservation by construction, while the neural ODE framework enables a flexible training procedure that allows the loss to be defined only on observed variables. We also incorporate additional physical constraints through symmetry-aware coordinate transformations and separable energy formulations. The framework is evaluated on systems of increasing complexity, from linear and nonlinear mass-spring systems to the chaotic three-body problem. Across all examples, increasing the amount of embedded physical structure improves the accuracy and long-horizon stability of the predictions. Even in the most challenging regimes, the NHODE framework captures both observed and latent dynamics, whereas purely data-driven baselines become unstable.

Sunniva Meltzer, Sølve Eidnes, Alexander Johannes Stasik

Department of Mathematics and Cybernetics, SINTEF Digital, Norway · Department of Physics, University of Oslo, Norway · Department of Data Science, Norwegian University of Life Science, Norway

Dynamical Systems Latent Dynamics Modeling Nonlinear System Identification Hamiltonian Neural Networks Physics-Informed ML Neural ODEs+1

May 21, 2026cs.LG

Holomorphic Neural ODEs with Kolmogorov-Arnold Networks for Interpretable Discovery of Complex Dynamics

Complex dynamical systems governed by holomorphic maps such as

z^2 + c

exhibit fractal boundaries with extreme sensitivity to initial conditions. Accurately modelling these structures from data requires methods that respect the underlying complex-analytic geometry, yet Multi-Layer Perceptrons (MLPs) within Neural Ordinary Differential Equations (Neural ODEs) lack complex-analytic priors, violate the Cauchy--Riemann conditions, and function as opaque approximators incapable of yielding governing equations. We introduce Holomorphic KAN-ODE, a framework that replaces the MLP with a Kolmogorov-Arnold Network (KAN) whose learnable B-spline activations reside on network edges, and incorporates Cauchy--Riemann equations as a differentiable regularization to preserve holomorphic structure. We evaluate on six families of complex dynamical systems spanning polynomial and transcendental classes. With only 280 parameters (

16\times

fewer than the MLP baseline), the network achieves velocity-field

R^2 > 0.95

on all six systems, correctly identifies all six governing symbolic families through automatic spline-to-formula fitting, and reconstructs Julia set fractal boundaries with up to 98.0% agreement. Crucially, the model exhibits only 4% MSE degradation under 10% observation noise versus

15.2\times

for MLPs, and achieves 90.4% improvement in transfer learning from quadratic to cubic dynamics. While the MLP attains lower pointwise reconstruction error due to its larger capacity, the KAN uniquely provides interpretable symbolic equations, enforced holomorphic structure, and superior noise resilience, capabilities that are entirely absent in black-box architectures. These results establish KANs as a parameter-efficient, interpretable alternative to MLPs for physics-informed discovery of holomorphic dynamics.

Bhaskar Ranjan Karn, Dinesh Kumar

Nonlinear System Identification Complex-Valued Neural Networks Kolmogorov-Arnold Networks Chaotic Dynamical Systems Neural ODEs ODE Discovery

May 19, 2026cs.LG

BrainDyn: A Sheaf Neural ODE for Generative Brain Dynamics

Efficient neural network models that generate brain-like dynamic activity can be a valuable resource for generating synthetic data, analyzing differences in brain transients under conditions such as testing perturbation activity or inferring the underlying generative dynamics. However, large language models (LLMs) or standard recurrent neural networks (RNNs) ignore the anatomical organization and therefore do not produce components that align with brain regions. On the other hand, graph-based networks often have very simple message passing rules that are not sufficiently expressive for brain-like dynamics. To address this, we introduce BrainDyn, a sheaf neural ordinary differential equation (neural ODE) model for continuous-time dynamics on structured brain graphs. BrainDyn encodes the recent activity history of each brain region using a long short-term memory (LSTM) model over a sliding temporal window to produce hidden states, or stalks, that are projected through learnable restriction maps into edge-specific shared spaces. Discrepancies between neighboring nodes in these shared spaces are characterized by a sheaf Laplacian that can facilitate message passing between neuronal units. The output of these messages is then fed to a neural ODE that governs the continuous-time evolution of neuronal activity. We evaluated BrainDyn on resting-state fMRI (PNC dataset), scalp EEG with focal epilepsy (TUSZ dataset), and simulated activity from the NEST spiking network simulator. BrainDyn achieves strong forecasting ability across modalities, and the resulting representations support downstream tasks including in silico perturbation prediction.

Siddharth Viswanath, Panayiotis Ketonis, Chen Liu +3

1Yale University · 2Boise State University · University of Wisconsin–Madison

Graph Neural Networks Sheaf Neural Networks Neuroimaging Neural ODEs Spatiotemporal GNNs Brain Network Analysis

May 13, 2026cs.LG

Toward AI-Driven Digital Twins for Metropolitan Floods: A Conditional Latent Dynamics Network Surrogate of the Shallow Water Equations

AI-driven flood digital twins demand fast hydrodynamic surrogates for ensemble forecasting and observation assimilation. Yet even GPU-accelerated two-dimensional shallow water equation (SWE) solvers still require

\sim 55

minutes per

96

-hour run on a

\sim 4.2

-million-active-cell metropolitan basin (the DesPlaines River basin at $30\,\mathrm{m}$ resolution), making such workloads prohibitive at native resolution. We present the Conditional Latent Dynamics Network (CLDNet): a low-dimensional latent neural ODE driven by rainfall, paired with a coordinate-based decoder conditioned on static terrain (elevation, slope, Manning roughness) that reconstructs depth and discharge at arbitrary query points. Pointwise decoding decouples memory from grid size and handles irregular watersheds natively, enabling metropolitan-scale training on a single compute node and direct queries at exact gauge coordinates without raster snapping. We evaluate CLDNet on a synthetic $250{,}000$ -cell Texas benchmark and on a new DesPlaines case study of

114

real-rainfall Stage~~IV storms whose reference simulator we validate against United States Geological Survey (USGS) gauges at the April~~2013 flood-of-record (Nash--Sutcliffe efficiency

0.57

--

0.94

on mean-recentered water-surface elevation). CLDNet roughly halves the relative root-mean-squared error of an unconditional baseline, outperforms regular-grid VAE--ConvLSTM and FNO baselines on the Texas benchmark (both presuppose a Cartesian grid and do not apply to the irregular Des~Plaines watershed), reaches a critical success index of

\approx 86\%

at the

0.5\,\mathrm{m}

inundation threshold, and produces a full

96

-hour basin-wide forecast in

\sim 29

seconds -- a

\sim 115\times

speedup.

Phillip Si, Yuan Qiu, Omar Sallam +4

School of Computational Science and Engineering, Georgia Institute of Technology, 756 West Peachtree Street Northwest, Atlanta, 30308, GA, USA · Environmental Science Division, Argonne National Laboratory, 9700 S Cass Ave, Lemont, 60439, IL, USA

Neural Surrogate Modeling PDE Surrogate Modeling Digital Twins Spatiotemporal Forecasting Neural ODEs Flood Forecasting

May 13, 2026cs.LG

MPINeuralODE: Multiple-Initial-Condition Physics-Informed Neural ODEs for Globally Consistent Dynamical System Learning

Neural ordinary differential equations (Neural ODEs) often fit training trajectories while generalizing poorly to unseen initial conditions and long horizons. We propose MPINeuralODE, which combines a soft physics-informed residual with a Multiple-Initial-Condition (MIC) multiple-shooting curriculum whose ingredients are structurally complementary: the physics term anchors the vector-field magnitude on the support that MIC enlarges. We evaluate along three axes: out-of-sample error, long-horizon stability, and Hamiltonian drift, which together expose whether the learned dynamics recover the underlying vector field. On Lotka-Volterra, MPINeuralODE achieves the lowest out-of-sample and long-horizon MSE among data-driven methods, with a 24% reduction over the baseline Neural ODE, while essentially matching the PINN ablation on Hamiltonian drift.

Lake Yang, Antonio Malpica-Morales, Frank Ioannis Papadakis Wood +1

Department of Chemical Engineering, Imperial College London, London SW7 2AZ, United Kingdom

Neural Network Generalization System Identification Physics-Informed ML Neural ODEs

May 11, 2026cs.LG

Steerable Neural ODEs on Homogeneous Spaces

We introduce steerable neural ordinary differential equations on homogeneous spaces

M=G/H

. These models constitute a novel geometric extension of manifold neural ordinary differential equations (NODEs) that transport associated feature vectors transforming under the local symmetry group

H

. We interpret features as sections of associated vector bundles over

M

, and describe their evolution as parallel transport. This results in a coupled system of ODEs consisting of a flow equation on

M

and a steering equation acting on features. We show that steerable NODEs are

G

-equivariant whenever the vector field generating the flow and the connection governing parallel transport are both

G

-invariant. Furthermore, we demonstrate how steerable NODEs incorporate existing NODE models and continuous normalizing flows on Lie groups. Our framework provides the geometric foundation for learning continuous-time equivariant dynamics of general vector-valued features on homogeneous spaces.

Emma Andersdotter, Daniel Persson, Fredrik Ohlsson

Department of Mathematics and Mathematical Statistics Umeå University Umeå SE-901 87, Sweden · Department of Mathematical Sciences2026 Chalmers University of Technology and Gothenburg University Gothenburg SE-412 96, Sweden · Department of Mathematics and Mathematical Statistics 11 Umeå University Umeå SE-901 87, Sweden

Equivariant Neural Networks Geometric Representation Learning Neural ODEs

May 11, 2026cond-mat.dis-nn

Exact Fixed-Point Constraints in Neural-ODEs with Provable Universality

We introduce a technique that enables Neural-ODEs to approximate arbitrary velocity fields with a priori planted fixed-points. Specifically, a recipe is given to explicitly accommodate for a finite collection of points in the reference multi-dimensional space of the Neural-ODE where the velocity field is exactly equal to zero. In this way, the gradient-based training is rigorously constrained inside the prescribed hypothesis class while leaving the expressive power of the Neural-ODE unaltered. We rigorously prove the universality of the Neural-ODE under any local constraints in the velocity field and give a computationally convenient way of imposing the fixed points. Our method is then tested on two paradigmatic physical models.

Feliciano Giuseppe Pacifico, Duccio Fanelli, Lorenzo Buffoni +3

Department of Informatics and Computer Science, University of Pisa, Italy · Department of Physics and Astronomy and INFN, University of Florence, Sesto Fiorentino, Italy

Constrained Optimization Neural Network Approximation Theory Neural ODEs

May 6, 2026math.OC

Meta-learning for sample-efficient Bayesian optimisation of fed-batch processes

The optimisation of fed-batch (bio)chemical process recipes is subject to inherent, underlying, and unmeasurable fluctuations across batches, whose trajectories are difficult to model and costly to measure. Bayesian Optimisation (BayesOpt) is a powerful tool for sampling and optimisation of expensive-to-measure functions. Gaussian Processes (GPs), the surrogate models used in BayesOpt, are static, forecast poorly, and lack generalisation across experiments, limiting their applicability to time-varying batch processes with stochastic parameters, i.e., process fluctuations. This work investigates System-Aware Neural ODE Processes (SANODEP) as a meta-learning model to overcome the limitations of GPs and increase few-shot optimisation performance in BayesOpt. Using a penicillin batch production case study, we find that SANODEP outperforms GP-based BayesOpt in the low-data regime, resulting in improved objectives when few experimental runs are performed. These improvements are observed in both on- and off-distribution batches, highlighting the generalisation capabilities of SANODEP. Using this approach, batch process operators can accelerate the initial optimisation steps in BayesOpt by deploying meta-learning or optimise the process with fewer experiments when the experimental cost is high.

Becky Langdon, Gabriel D. Patrón, Chrysoula D. Kappatou +6

Department of Computing, Imperial College London, United Kingdom · BASF SE, Ludwigshafen, Germany · School of Mathematical Sciences, Lancaster University, United Kingdom +1

Meta-Learning Bayesian Optimization Neural ODEs

May 5, 2026cs.LG

Physics-Modeled Neural Networks

We introduce \emph{Dynamical Physics-Modeled Neural Networks} (DynPMNNs), a continuous-time deep learning architecture in which each hidden layer is defined as the solution of an ordinary differential equation. Unlike classical feed-forward networks, this approach replaces static activation functions with time-evolving dynamical systems, providing a biologically inspired interpretation of hidden-layer behavior and enabling the integration of physically meaningful models. The framework is rigorously grounded in Reproducing Kernel Banach Spaces (RKBSs), allowing DynPMNNs to be characterized as finite-dimensional solutions of an abstract training problem and revealing structural connections with standard neural networks. We present a concrete implementation based on the FitzHugh--Nagumo model for neuronal activation, where numerical ODE solvers are embedded into the computational graph via Euler-type schemes. Both network weights and dynamical parameters are trained jointly. Through experiments on the California Housing dataset, we compare DynPMNNs with Neural ODEs (NODEs) and Closed-form Continuous-Time Networks (CfCs). Despite using fewer trainable parameters, DynPMNNs achieve competitive performance. These results position DynPMNNs as a principled bridge between dynamical systems and deep learning, with promising directions for further research in expressivity, stability, and physics-based modeling.

Raul Felipe-Sosa, Angel Martin del Rey, Maria Flores Ceballos

aDepartamento de Matemáticas, Escuela Politénica Superior, Universidad Nebrija, 28015 Madrid, Spain · bDepartment of Applied Mathematics, IUFFyM, Universidad de Salamanca, 37008 Salamanca, Spain · cUnidad de Excelencia en Luz y Materia Estructuradas (LUMES), Universidad de Salamanca, 37008 Salamanca, Spain

Dynamic Neural Networks Neural ODEs

May 5, 2026cs.LG

Local Truncation Error-Guided Neural ODEs for Large Scale Traffic Forecasting

Spatiotemporal forecasting in physical systems, such as large-scale traffic networks, requires modeling a dual dynamic: continuous macroscopic rhythms and discrete, unpredictable microscopic shocks. While Neural Ordinary Differential Equations (ODEs) excel at capturing smooth evolution, their inherent Lipschitz continuity constraints inevitably cause severe over-smoothing when confronting abrupt anomalies. Recent physics-informed methods attempt to bypass this by penalizing numerical integration errors to enforce manifold smoothness. However, we mathematically reveal that such rigid regularization inherently triggers gradient conflicts and ``attention collapse,'' stripping the model of its sensitivity to anomalies. To resolve this continuity-shock dilemma, we propose Local Truncation Error-Guided Neural ODEs (LTE-ODE). Rather than treating numerical error as a nuisance to be eliminated, we innovatively repurpose the Local Truncation Error (LTE) as an unsupervised forward inductive bias. By mapping the LTE into a dynamic spatial attention mask, our architecture gracefully preserves high-precision continuous ODE evolution in stable regions, while adaptively triggering a discrete compensation branch exclusively at shock points. Trained purely end-to-end without manifold penalties, LTE-ODE achieves state-of-the-art performance on multiple large-scale benchmarks, exhibiting exceptional robustness against highly non-linear fluctuations. Furthermore, our ablation on integration steps demonstrates high deployment flexibility, allowing the model to seamlessly adapt to varying hardware memory constraints in real-world applications.

Xiao Zhang, Yafei Li, Ruixiang Wang +3

Zhengzhou University · Northwestern Polytechnical University

Spatiotemporal Forecasting Neural ODEs Traffic Forecasting

May 5, 2026cs.RO

Robust Path Tracking for Vehicles via Continuous-Time Residual Learning: An ICODE-MPPI Approach

Model Predictive Path Integral (MPPI) control is a powerful sampling-based strategy for nonlinear autonomous systems. However, its performance is often bottlenecked by the fidelity of nominal dynamics. We propose ICODE-MPPI, a robust framework that leverages Input Concomitant Neural Ordinary Differential Equations (ICODEs) to learn and compensate for unmodeled residual dynamics. Unlike discrete-time learners, ICODEs maintain physical consistency and temporal continuity during the MPPI prediction horizon. High-fidelity simulations on complex trajectories demonstrate that ICODE-MPPI achieves up to a 69% reduction in cross-tracking error under persistent disturbances compared to standard MPPI control. Furthermore, our analysis confirms that ICODE-MPPI significantly suppresses control chattering, yielding smoother steering commands and superior robust performance.

Shugen Song, Wenjie Mei, Chengyan Zhao

School of Automation, Southeast University, Nanjing 210096, China · School of Automation and the Key Laboratory of MCCSE of the Ministry of Education, Southeast University, Nanjing 210096, China · Graduate School of Computer Science and Systems Engineering, Kyushu Institute of Technology, 680-4 Kawazu, Iizuka-shi, Fukuoka, Japan

Nonlinear MPC Residual Learning Autonomous Driving Robust Control MPPI Control Neural ODEs

Apr 30, 2026cs.LG

PINN-Cast: Exploring the Role of Continuous-Depth NODE in Transformers and Physics Informed Loss as Soft Physical Constraints in Short-term Weather Forecasting

Operational weather prediction has long relied on physics-based numerical weather prediction (NWP), whose accuracy comes at the cost of substantial compute and complex simulation workflows. Recent transformer-based forecasters offer efficient data-driven alternatives, however transformers are physics-agnostic models. Additionally, standard transformer encoders evolve representations through discrete layer updates that may be less suited to modeling smooth latent dynamics. In this work, we propose a continuous-depth transformer encoder for weather forecasting that integrates Neural Ordinary Differential Equation (Neural ODE) dynamics within each encoder block. Specifically, we replace discrete residual updates with ODE-based updates solved using adaptive numerical integration. We also introduce a two-branch attention module that combines conventional patch-wise self-attention with an auxiliary branch that applies a derivative operator to attention logits, providing an additional change-sensitive interaction signal. To further align forecasts with governing principles, we propose a customized physics-informed training objective that enforces physical consistency as a soft constraint. We evaluate the proposed method against a standard discrete transformer baseline and an existing continuous-time Neural ODE forecasting variant, demonstrating the importance of PINN-Cast in short term weather forecasting.

Hira Saleem, Flora Salim, Cormac Purcell

University of New South Wales, Sydney, NSW, Australia

Transformer Physics-Informed ML Spatiotemporal Forecasting Neural ODEs Weather Forecasting

Apr 28, 2026cs.LG

Observable Neural ODEs for Identifiable Causal Forecasting in Continuous Time

Causal inference in continuous-time sequential decision problems is challenged by hidden confounding and partially observed states. We show that, under explicit structural assumptions, observability of the latent state enables identification of dynamic treatment effects through a continuous-time conditional front-door adjustment, even in the presence of hidden confounding. We derive a general adjustment formula and show that it reduces to a tractable state-space formula when unobserved contemporaneous disturbances are temporally uncorrelated. This formula expresses potential-outcome distributions under alternative treatment trajectories through the measurement model, latent dynamics, and the filtering distribution over latent states. We propose Observable Neural ODEs (ObsNODEs), Neural ODE models in observable normal form that implement this tractable adjustment for causal forecasting. ObsNODEs learn continuous-time dynamics with states reconstructible from observations, enabling outcome prediction under alternative treatment paths. Experiments on synthetic, semi-synthetic, and real-world clinical data demonstrate strong performance over recent sequence models, including external validation.

Jennifer Wendland, Nicolas Freitag, Maik Kschischo

Department of Computer Science University of Koblenz 53424 Koblenz, Germany

Causal Effect Estimation Clinical Outcome Prediction Causal Identifiability Causal Inference Neural ODEs

Apr 27, 2026cs.LG

Latent-Hysteresis Graph ODEs: Modeling Coupled Topology-Feature Evolution via Continuous Phase Transitions

Graph neural ordinary differential equations (Graph ODEs) extend graph learning from discrete message-passing layers to continuous-time representation flows. While it supports adaptive long-range propagation, we show that Graph ODEs with strictly positive irreducible mixing operators face an inherent \emph{monostability trap}: in the long-time regime, information leakage is unavoidable and the dynamics converge to a single global consensus attractor. We propose the \textbf{Hysteresis Graph ODE (HGODE)}, which couples feature evolution with a latent topological potential driven by a learned pairwise force. A double-well edge potential and bipolarized gate allow edge states to polarize into connected or insulated phases while preserving differentiability. We provide asymptotic analysis of the collapse mechanism and the proposed hysteretic topology dynamics, and validate HGODE on theory-driven synthetic diagnostics and real-world graph benchmarks.

Qinhan Hou, Jing Tang

Doctoral Program of Computer Science, University of Helsinki, Helsinki, Finland · Research Program in Systems Oncology, Faculty of Medicine, University of Helsinki, Helsinki, Finland

Dynamical Systems Graph Structure Learning Graph Neural Networks Graph Representation Learning Phase Transitions Neural ODEs

Apr 27, 2026cs.LG

Time-varying Interaction Graph ODE for Dynamic Graph Representation Learning

Graph neural Ordinary Differential Equations (ODE) combine neural ODE with the message passing mechanism of Graph Neural Networks (GNN), providing a continuous-time modeling method for graph representation learning. However, in dynamic graph scenarios, existing graph neural ODEs typically employ a unified message passing mechanism, assuming that inter-node interactions share the same message passing function at any time, which makes it challenging to capture the diversity and time-varying nature of inter-node interaction patterns. To address this, we propose Time-varying Interaction Graph Ordinary Differential Equations (TI-ODE). The core idea of TI-ODE is to decompose the evolution function of a graph ODE into a set of learnable interaction basis functions, where each basis function corresponds to a distinct type of inter-node interaction. These basis functions are dynamically combined through time-dependent learnable weights, enabling inter-node interaction patterns to adaptively evolve over time. Experimental results on six dynamic graph datasets demonstrate that TI-ODE consistently outperforms existing methods and achieves state-of-the-art performance on attribute prediction tasks, and experiments on the \textit{Covid} dataset further verify the interpretability and generalizability of our TI-ODE. Furthermore, we demonstrate both theoretically and empirically that TI-ODE exhibits superior robustness compared to models utilizing a unified message-passing mechanism.

Xiaoyi Wang, Zhiqiang Wang, Jianqing Liang +4

Key Laboratory of Computational Intelligence and Chinese Information Processing of Ministry of Education, School of Computer and Information Technology, Shanxi University, Taiyuan, Shanxi 030006, China. · Department of System Engineering and Engineering Management, City University of Hong Kong, Kowloon 999077, Hong Kong, China. · Key Laboratory of Complex Systems and Data Science of Ministry of Education, Complex Systems Research Center, Shanxi University, Taiyuan 030006, Shanxi, China.

Temporal GNNs Graph Representation Learning Dynamic Graph Learning Neural ODEs

Apr 20, 2026cs.LG

Scalable Physics-Informed Neural Differential Equations and Data-Driven Algorithms for HVAC Systems

We present a scalable, data-driven simulation framework for large-scale heating, ventilation, and air conditioning (HVAC) systems that couples physics-informed neural ordinary differential equations (PINODEs) with differential-algebraic equation (DAE) solvers. At the component level, we learn heat-exchanger dynamics using an implicit PINODE formulation that predicts conserved quantities (refrigerant mass

M_r

and internal energy

E_\text{hx}

) as outputs, enabling physics-informed training via automatic differentiation of mass/energy balances. Stable long-horizon prediction is achieved through gradient-stabilized latent evolution with gated architectures and layer normalization. At the system level, we integrate learned components with DAE solvers (IDA and DASSL) that explicitly enforce junction constraints (pressure equilibrium and mass-flow consistency), and we use Bayesian optimization to tune solver parameters for accuracy--efficiency trade-offs. To reduce residual system-level bias, we introduce a lightweight corrector network trained on short trajectory segments. Across dual-compressor and scaled network studies, the proposed approach attains multi-fold speedups over high-fidelity simulation while keeping errors low (MAPE below a few percent) and scales to systems with up to 16 compressor-condenser pairs.

Hanfeng Zhai, Hongtao Qiao, Hassan Mansour +1

Department of Mechanical Engineering, Stanford University, Stanford, CA · Mitsubishi Electric Research Laboratories, Cambridge, MA

Neural Surrogate Modeling Physics-Informed ML Neural ODEs

Apr 3, 2026cs.RO

Goal-Conditioned Neural ODEs with Guaranteed Safety and Stability for Learning-Based All-Pairs Motion Planning

This paper presents a learning-based approach for all-pairs motion planning, where the initial and goal states are allowed to be arbitrary points in a safe set. We construct smooth goal-conditioned neural ordinary differential equations (neural ODEs) via bi-Lipschitz diffeomorphisms. Theoretical results show that the proposed model can provide guarantees of global exponential stability and safety (safe set forward invariance) regardless of goal location. Moreover, explicit bounds on convergence rate, tracking error, and vector field magnitude are established. Our approach admits a tractable learning implementation using bi-Lipschitz neural networks and can incorporate demonstration data. We illustrate the effectiveness of the proposed method on a 2D corridor navigation task.

Dechuan Liu, Ruigang Wang, Ian R. Manchester

Australian Centre for Robotics (ACFR), and the School of Aerospace, Mechanical and Mechatronic Engineering, The University of Sydney, Sydney, NSW 2006, Australia

Constrained Motion Planning Safe Motion Planning Motion Planning Neural ODEs Robot Motion Planning

Mar 16, 2026physics.flu-dyn

A convolutional autoencoder and neural ODE surrogate modeling framework applied to transient counterflow flames

A novel convolutional autoencoder and neural ODE (CAE-NODE) framework is proposed for a reduced-order model (ROM) applied to transient 2D counterflow flames, as an extension of AE-NODE methods in homogeneous reactive systems to spatially resolved flows. The multidimensional thermochemical fields (256 x 256 grid, 21 variables) obtained from direct numerical simulations (DNS) are used in training the CAE, where convolutional layers learned the underlying spatial correlations, allowing the CAE to construct an unsupervised 3D latent manifold that is physically meaningful, smooth, and continuous in time. This results in a compression ratio of over 400,000 times. The NODE then subsequently learns the continuous-time dynamics on the latent manifold, enabling the prediction of the full temporal evolution of the flames by integrating forward in time from an initial condition. The results demonstrate that the CAE-NODE can accurately capture the entire transient process, including ignition, flame propagation, and the gradual transition to a non-premixed condition, with excellent agreement with the DNS, while adhering to conservation principles at virtually no computational cost compared to the reference DNS. Predictions remain accurate at strain rates outside the training range. Moreover, despite being unsupervised, the learned latent manifold is highly correlated with the flame-state descriptors such as the progress variable, mixture fraction, and the scalar dissipation rate. This study, for the first time, highlights the potential of CAE-NODE for surrogate modeling of unsteady dynamics of multi-dimensional reacting flows.

Mert Yakup Baykan, Weitao Liu, Mohammad Rafi Malik +4

Department of Aerospace Engineering, KAIST, Daejeon, Republic of Korea · Clean Energy Research Platform (CERP), KAUST, Thuwal, Saudi Arabia · Institute for Reactive Flows (IRST), University of Stuttgart, Pfaffenwaldring 31, 70569 Stuttgart, Germany

Autoencoders PDE Surrogate Modeling Convolutional Autoencoder Reduced-Order Modeling Neural ODEs Computational Fluid Dynamics

Feb 6, 2026cs.LG

How (Not) to Hybridize Neural and Mechanistic Models for Epidemiological Forecasting

Epidemiological forecasting from surveillance data is a hard problem and hybridizing mechanistic compartmental models with neural models is a natural direction. The mechanistic structure helps keep trajectories epidemiologically plausible, while neural components can capture non-stationary, data-adaptive effects. In practice, however, many seemingly straightforward couplings fail under partial observability and continually shifting transmission dynamics driven by behavior, waning immunity, seasonality, and interventions. We catalog these failure modes and show that robust performance requires making non-stationarity explicit: we extract and extrapolate multi-scale structure from the observed infection series and use it as an interpretable control signal for a controlled neural ODE coupled to an epidemiological model. Concretely, we decompose infections into trend, seasonal, and residual components and use these signals to drive continuous-time latent dynamics while jointly forecasting and inferring time-varying transmission, recovery, and immunity-loss rates. Across early outbreak and multi-wave regimes, our approach attains the lowest RMSE on all five datasets (27-70% reduction over the strongest baseline), achieves the best peak detection accuracy, and recovers time-varying epidemiological rates within ground-truth ranges, without relying on auxiliary covariates.

Yiqi Su, Ray Lee, Jiaming Cui +1

Department of Computer Science, Virginia Tech, Alexandria, USA.

Epidemic Forecasting Non-Stationary Time Series Forecasting Neural ODEs Multiscale Time-Series Modeling

Feb 3, 2026cs.LG

Geometry-Preserving Neural Architectures on Manifolds with Boundary

A growing number of neural architectures have been proposed to enforce geometric constraints, including projection-based networks, exponential-map updates, constrained output layers, and manifold neural ODEs. We provide a unified framework for these geometry-preserving architectures by organizing them according to where and how constraints are enforced, either throughout the intermediate layers or only at the final output. This perspective reveals several gaps in the existing theory. To address these gaps, we prove high-level approximation theorems for projected neural ODEs, intermediate augmented architectures, and final augmented architectures on prox-regular constraint sets, including smooth manifolds with boundary. Numerical experiments on synthetic dynamics over S^2, the disk, SO(3), together with real-world protein backbone data on SE(3), demonstrate exact feasibility for analytic updates and show that the final augmentation have simpler architecture and outperform in most tasks considered. When the constraint set is unknown, we learn projections via small-time heat-kernel limits, showing diffusion/flow-matching can be used as data-based projections. Moreover, we also the demonstrate the usefulness of the architectures that enforce non-convex constraints for path planning on manifolds with boundary.

Karthik Elamvazhuthi, Shiba Biswal, Kian Rosenblum +4

Los Alamos National Laboratory · Boston College

Geometric Representation Learning Neural Network Approximation Theory Neural ODEs

Feb 2, 2026cs.LG

Efficient Neural Controlled Differential Equations via Attentive Kernel Smoothing

Neural Controlled Differential Equations (Neural CDEs) provide a powerful continuous-time framework for sequence modeling, yet the roughness of the driving control path often restricts their efficiency. Standard splines introduce high-frequency variations that force adaptive solvers to take excessively small steps, driving up the Number of Function Evaluations (NFE). We propose a novel approach to Neural CDE path construction that replaces exact interpolation with Kernel and Gaussian Process (GP) smoothing, enabling explicit control over trajectory regularity. To recover details lost during smoothing, we propose an attention-based Multi-View CDE (MV-CDE) and its convolutional extension (MVC-CDE), which employ learnable queries to inform path reconstruction. This framework allows the model to distribute representational capacity across multiple trajectories, each capturing distinct temporal patterns. Empirical results demonstrate that our method, MVC-CDE with GP, achieves state-of-the-art accuracy while significantly reducing NFEs and total inference time compared to spline-based baselines.

Egor Serov, Ilya Kuleshov, Alexey Zaytsev

Applied AI Institute · Risk AI Research Lab

Efficient Neural Network Inference Neural Controlled Differential Equations Attention Mechanisms Neural ODEs

Jan 20, 2026cs.LG

Diff-MN: Diffusion Parameterized MoE-NCDE for Continuous Time Series Generation with Irregular Observations

Time series generation (TSG) is widely used across domains, yet most existing methods assume regular sampling and fixed output resolutions. These assumptions are often violated in practice, where observations are irregular and sparse, while downstream applications require continuous and high-resolution TS. Although Neural Controlled Differential Equation (NCDE) is promising for modeling irregular TS, it is constrained by a single dynamics function, tightly coupled optimization, and limited ability to adapt learned dynamics to newly generated samples from the generative model. We propose Diff-MN, a continuous TSG framework that enhances NCDE with a Mixture-of-Experts (MoE) dynamics function and a decoupled architectural design for dynamics-focused training. To further enable NCDE to generalize to newly generated samples, Diff-MN employs a diffusion model to parameterize the NCDE temporal dynamics parameters (MoE weights), i.e., jointly learn the distribution of TS data and MoE weights. This design allows sample-specific NCDE parameters to be generated for continuous TS generation. Experiments on ten public and synthetic datasets demonstrate that Diff-MN consistently outperforms strong baselines on both irregular-to-regular and irregular-to-continuous TSG tasks. The code is available at the link https://github.com/microsoft/TimeCraft/tree/main/Diff-MN.

Xu Zhang, Junwei Deng, Chang Xu +2

Microsoft Research, Beijing, China · University of Illinois Urbana-Champaign, Illinois, United States · Imperial College London, London, United Kingdom

Irregular Time-Series Modeling Neural Controlled Differential Equations Time Series Generation Neural ODEs Diffusion Models

Nov 13, 2025cs.LG

FlowPath: Learning Data-Driven Manifolds with Invertible Flows for Robust Irregularly-sampled Time Series Classification

Modeling continuous-time dynamics from sparse and irregularly-sampled time series remains a fundamental challenge. Neural controlled differential equations provide a principled framework for such tasks, yet their performance is highly sensitive to the choice of control path constructed from discrete observations. Existing methods commonly employ fixed interpolation schemes, which impose simplistic geometric assumptions that often misrepresent the underlying data manifold, particularly under high missingness. We propose FlowPath, a novel approach that learns the geometry of the control path via an invertible neural flow. Rather than merely connecting observations, FlowPath constructs a continuous and data-adaptive manifold, guided by invertibility constraints that enforce information-preserving and well-behaved transformations. This inductive bias distinguishes FlowPath from prior unconstrained learnable path models. Empirical evaluations on 18 benchmark datasets and a real-world case study demonstrate that FlowPath consistently achieves statistically significant improvements in classification accuracy over baselines using fixed interpolants or non-invertible architectures. These results highlight the importance of modeling not only the dynamics along the path but also the geometry of the path itself, offering a robust and generalizable solution for learning from irregular time series.

YongKyung Oh, Dong-Young Lim, Sungil Kim

Medical & Imaging Informatics (MII) Group, University of California, Los Angeles (UCLA), CA, USA · Department of Industrial Engineering, Ulsan National Institute of Science and Technology (UNIST), Republic of Korea · Artificial Intelligence Graduate School, Ulsan National Institute of Science and Technology (UNIST), Republic of Korea

Irregular Time-Series Modeling Time Series Classification Neural ODEs