cs.LGSep 3, 2026

Data-Driven Learning of Unknown Nonlinear Differential Equations Using Functional Analysis

Authors: Seyyed Shaho Alaviani, Yongzhi Qu, Gregory W. Vogl

Organizations: Department of Mechanical Engineering University of Minnesota-Twin Cities Minneapolis, MN 55455, USA · Department of Mechanical Engineering University of Utah Salt Lake City, UT 84112, USA · National Institute of Standards and Technology Gaithersburg, MD 20899, USA

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

In this paper, the problem of data-driven discovery of nonlinear ordinary differential equations (ODEs) is recast, and a new interpretable machine learning (ML) method is proposed. The proposed method aims to learn the unknown vector field of nonlinear dynamics without prior knowledge of the system's physics from only one single state trajectory's data. The proposed method has two fundamental differences with existing methods: 1) the formulation presented in this method is derived based on Functional Analysis and Operator Theory, and 2) the cost function is constructed in the function space as a distance between two functions as an integral, instead of the discrete-sum of errors used in existing ML approaches. An incremental learning algorithm is proposed to learn the unknown vector field to handle new training samples in an online manner. The proposed method can discover the unknown vector field from both forced and unforced autonomous and non-autonomous (or time-varying) dynamical systems. The proposed method is able to simultaneously discover unknown external forces as a function of time and unknown underlying dynamics. Finally, numerical examples are given to demonstrate the advantages of the proposed method.

Figures & tables

Figure 1: Difference between FINS and existing methods for data-driven dynamics learning, with NODEs as an example. (Top: Cost evaluation and potential learned function of NODEs; Bottom: Cost evaluation and potential learned function of FINS).
Figure 2: Schematic of FINS method.
Dynamical systemsEquationsParameters
Duffing equation (Unforced)(x˙y˙)=(y−δy−αx−βx3)\begin{pmatrix}\dot{x}\\ \dot{y}\end{pmatrix}=\begin{pmatrix}y\\ -\delta y-\alpha x-\beta x^{3}\end{pmatrix}δ=α=β=1\delta=\alpha=\beta=1
Van der Pol(x˙y˙)=(y−αx+ε1y−ε2x2y)\begin{pmatrix}\dot{x}\\ \dot{y}\end{pmatrix}=\begin{pmatrix}y\\ -\alpha x+\varepsilon_{1}y-\varepsilon_{2}x^{2}y\end{pmatrix}α=1,ε1=ε2=0.05\alpha=1,\varepsilon_{1}=\varepsilon_{2}=0.05
Duffing equation with known force(x˙y˙)=(y−δy−αx−βx3)+(01000sin⁡(2t))\begin{pmatrix}\dot{x}\\ \dot{y}\end{pmatrix}=\begin{pmatrix}y\\ -\delta y-\alpha x-\beta x^{3}\end{pmatrix}+\begin{pmatrix}0\\ 1000\sin(2t)\end{pmatrix}δ=α=β=1\delta=\alpha=\beta=1
Van der Pol with unknown force(x˙y˙)=(y−αx+ε1y−ε2x2y+δt2)\begin{pmatrix}\dot{x}\\ \dot{y}\end{pmatrix}=\begin{pmatrix}y\\ -\alpha x+\varepsilon_{1}y-\varepsilon_{2}x^{2}y+\delta t^{2}\end{pmatrix}α=1,ε1=ε2=0.02,δ=0.01\alpha=1,\varepsilon_{1}=\varepsilon_{2}=0.02,\delta=0.01
Lorenz chaotic system(x˙y˙z˙)=(10(y−x)x(28−z)−yxy−83z)\begin{pmatrix}\dot{x}\\ \dot{y}\\ \dot{z}\end{pmatrix}=\begin{pmatrix}10(y-x)\\ x(28-z)-y\\ xy-\frac{8}{3}z\end{pmatrix}α=10,ρ=28,β=8/3\alpha=10,\rho=28,\beta=8/3
Mathieu-Hill’s equation(x˙y˙)=(y−(δ+ϵcos⁡(t))x)\begin{pmatrix}\dot{x}\\ \dot{y}\end{pmatrix}=\begin{pmatrix}y\\ -(\delta+\epsilon\cos(t))x\end{pmatrix}δ=1,ϵ=0.2\delta=1,\epsilon=0.2
Table 1: Dynamical systems considered in Examples 1-7
Figure 4
Duffing equationEDMDSINDyFINS
no noise13.72 ±\pm 11.300.48 ±\pm 0.68 (np) 0.09 ±\pm 0.03 (sp)0.03 ±\pm 0.06 (np) 0.05 ±\pm 0.09 (sp)
noisy8.87 ±\pm 5.344.88 ±\pm 3.84 (np) 6.36 ±\pm 5.02 (sp)2.91 ±\pm 2.11 (np) 5.20 ±\pm 3.97 (sp)
Table 2: Comparison of errors of prediction time (i.e., [3,25][3,25] and [6,10][6,10] seconds for no noise and noisy cases, respectively) for learned models of different methods for Duffing system in Example 1 under non-uniform sampling time
Figure 6
Parametersδ\deltaα\alphaβ\betaγ\gammaη\etaconstant1constant2
Duffing equation1111000
SINDy1.00510.73931.00070.99950-0.1005-1.2393
KLT0.99991.00020.99991.0000000.0005
FINS1.00040.97091.00031.0000-0.0006-0.0002-0.0077
FINS_s1.00010.99911.00001.0000000
Table 3: Learned models by several methods in Example 1 without noise under uniform sampling time
Duffing equationEDMDKLTSINDyFINS
no noise1.5141 ×1099±\times 10^{99}\pm 1.7720 ×10100\times 10^{100}0.0006 ±\pm 0.00200.8988 ±\pm 0.19490.0113 ±\pm 0.0326 (np) 0.0003 ±\pm 0.0040 (sp)
Table 4: Comparison of errors of learned models of different methods for Duffing system in Example 1 under uniform sampling time
Figure 7: 3D plot of trajectories of learned models by several methods for Van der Pol equation in Example 2
Parametersβ\betaα\alphaε1\varepsilon_{1}ε2\varepsilon_{2}const1const2
Van der Pol110.050.0500
SINDy0.99990.99910.05000.049900
SINDy_np0.99950.99830.05170.0501-0.00150.0068
FINS1.000110.05000.05000.00010
FINS_s0.999910.05000.050000
Table 5: Learned models by SINDy and FINS in Example 2 under non-uniform sampling time
 EDMDSINDyFINS
Van Der Pol4.0798 ±\pm 1.50800.0829 ±\pm 0.0224 (np) 0.0345 ±\pm 0.0110 (sp)0.0044 ±\pm 0.0010 (np) 0.0046 ±\pm 0.0010 (sp)
Table 6: Comparison of errors of learned models of different methods in Example 2 under non-uniform sampling time
Parametersβ\betaα\alphaε1\varepsilon_{1}ε2\varepsilon_{2}const1const2
Van der Pol110.050.0500
SINDy0.99940.99940.04940.049900
KLT1.00000.99990.04990.049900
FINS1.00001.00010.05000.050000
FINS_s0.99991.00000.04990.049900
Table 7: Learned models by several methods in Example 2 under uniform sampling time
 EDMDKLTSINDyFINS
Van der Pol0.7817 ±\pm 0.40960.0068 ±\pm 0.00170.0337 ±\pm 0.01050.0038 ±\pm 0.0009 (np) 0.0067 ±\pm 0.0016 (sp)
Table 8: Comparison of errors of learned models of different methods for Van der Pol system in Example 2 under uniform sampling time
Figure 14
Figure 10: Prediction error v.s. sampling time interval of learned models by SINDY_c, FINS, and FINS_s in Example 3
Figure 16
 FINS, N=2FINS_s, N=2FINS, N=3FINS_s, N=3
Training error0.0055 ±\pm 0.00330 ±\pm 05.4768 ±\pm 4.61487.2563 ±\pm 4.4988
Prediction error0.0156 ±\pm 0.01020 ±\pm 08.9985 ±\pm 4.678710.3255 ±\pm 4.5012
Table 9: Effect of the selection of polynomial order on training and prediction error for Example 4
MonomialsLorenz systemSINDyFINSFINS_s
100-0.00050
x-10-9.9986-9.9999-9.9999
y109.9991109.9999
z0000
xy0000
xz0000
Table 10: Coefficients of the monomials of x˙(t)\dot{x}(t) of learned models by SINDy, FINS, and FINS_s in Example 5 under non-uniform sampling time
MonomialsLorenz systemSINDyFINSFINS_s
100-0.00050
x2827.991328.000228.0013
y-1-0.9993-0.9999-1.0003
z0000
xy0000
xz-1-0.9996-1-1
Table 11: Coefficients of monomials of y˙(t)\dot{y}(t) of learned models by SINDy, FINS, and FINS_s in Example 5 under non-uniform sampling time
MonomialsLorenz systemSINDyFINSFINS_s
100-0.00100
x000.00020
y00-0.00010
z−83-\frac{8}{3}-2.6667-2.6666-2.6666
xy10.999911
xz0000
Table 12: Coefficients of monomials of z˙(t)\dot{z}(t) of learned models by SINDy, FINS, and FINS_s in Example 5 under non-unifrom sampling time
Lorenz SystemSINDyFINSFINS_s
Interpolation0.0101 ±\pm 0.00540.0016 ±\pm 0.00100.0010 ±\pm 0.0006
Extrapolation10.038 ±\pm 0.0160.008 ±\pm 0.0040.003 ±\pm 0.002
Extrapolation20.13 ±\pm 0.080.04 ±\pm 0.030.02 ±\pm 0.02
Table 13: Comparison of errors of learned models by SINDy, FINS, and FINS_s in Example 5 under non-uniform sampling time
Lorenz SystemSINDyFINSFINS_s
k = 26k, N = 1N/A0.0375 ±\pm 0.00630.0403 ±\pm 0.0103
k = 260k, N = 1N/A0.3527 ±\pm 0.01170.3500 ±\pm 0.0093
k = 2.6M, N = 1N/A3.5580 ±\pm 0.02403.5850 ±\pm 0.0157
k = 26k, N = 20.0139 ±\pm 0.00830.0787 ±\pm 0.01300.0805 ±\pm 0.0113
k = 260k, N = 20.1771 ±\pm 0.01150.7625 ±\pm 0.01080.7585 ±\pm 0.0099
k = 2.6M, N = 21.9897 ±\pm 0.02217.6285 ±\pm 0.04147.6977 ±\pm 0.0492
Table 14: Comparison of computational time (seconds) (for 25 independent runs) for discovering Lorenz system by SINDy, FINS, and FINS_s ( if the cardinality of Dt\mathcal{D}_{t} is fixed for the training time ) where N is highest degree of monomials in SINDy and of Hermite polynomials here
Lorenz SystemFINSFINS_s
k = 26k, N = 19.1228 ×10−6\times 10^{-6} ±\pm 3.2566 ×10−5\times 10^{-5}5.7904 ×10−5\times 10^{-5} ±\pm 2.0895 ×10−4\times 10^{-4}
k = 26k, N = 26.8872 ×10−4\times 10^{-4} ±\pm 9.7060 ×10−4\times 10^{-4}0.0015 ±\pm 0.0021
k = 26k, N = 37.7463 ×10−4\times 10^{-4} ±\pm 8.1156 ×10−4\times 10^{-4}0.0024 ±\pm 0.0028
k = 26k, N = 49.8189 ×10−4\times 10^{-4} ±\pm 8.3949 ×10−4\times 10^{-4}0.0028 ±\pm 0.0025
Table 15: Comparison of computational time (seconds) for discovering Lorenz system for incremental learning (i.e., update time per new datum) of FINS and FINS_s where N is highest degree of Hermite polynomials
 Duffing n=2Lorenz n=3Lorenz 96 n=4Method
k = 26k, N = 20.0114 ±\pm 0.00360.0139 ±\pm 0.00830.0209 ±\pm 0.0048SINDy
k = 260k, N = 20.0896 ±\pm 0.00420.1771 ±\pm 0.01150.2457 ±\pm 0.0111SINDy
k = 2.6M, N = 20.8026 ±\pm 0.00581.9897 ±\pm 0.02213.6259 ±\pm 0.0314SINDy
k = 26k, N = 20.0489 ±\pm 0.00580.0787 ±\pm 0.01300.3802 ±\pm 0.0394FINS
k = 260k, N = 20.4639 ±\pm 0.01080.7625 ±\pm 0.01083.8330 ±\pm 0.0402FINS
k = 2.6M, N = 24.7110 ±\pm 0.04597.6285 ±\pm 0.041437.8838 ±\pm 0.2629FINS
Table 16: Evaluation of computational time (seconds) ( if the cardinality of Dt\mathcal{D}_{t} is fixed for the training time ) w.r.t. the dimension of the states, i.e., n , (for 25 independent runs) where N is highest degrees of monomials in SINDy and of Hermite polynomials
 Duffing n=2Lorenz n=3Lorenz 96 n=4
FINS5.8375 ×10−4\times 10^{-4} ±\pm 7.8622 ×10−4\times 10^{-4}6.8872 ×10−4\times 10^{-4} ±\pm 9.7060 ×10−4\times 10^{-4}5.1372 ×10−4\times 10^{-4} ±\pm 4.4352 ×10−4\times 10^{-4}
FINS_s0.0014 ±\pm 0.00200.0015 ±\pm 0.00210.0081 ±\pm 0.0053
Table 17: Evaluation of computational time (seconds) for incremental learning (i.e., update time per new datum) w.r.t. the dimension of the states, i.e., n , where kk = 26k and N=2
Figure 13: Bar plot of Table 14
Figure 14: Bar plot of Table 16
More in this topic

ODE Discovery

Ordinary differential equation discovery seeks governing equations from observed trajectories to explain and predict dynamical systems.

See all 22 papers

Explore similar work

CardsList
May 8, 2026

Discovering Ordinary Differential Equations with LLM-Based Qualitative and Quantitative Evaluation

Discovering governing differential equations from observational data is a fundamental challenge in scientific machine learning. Existing symbolic regression approaches rely primarily on quantitative metrics; however, real-world differential equation modeling also requires incorporating domain knowledge to ensure physical plausibility. To address this gap, we propose DoLQ, a method for discovering ordinary differential equations with LLM-based qualitative and quantitative evaluation. DoLQ employs a multi-agent architecture: a Sampler Agent proposes dynamic system candidates, a Parameter Optimizer refines equations for accuracy, and a Scientist Agent leverages an LLM to conduct both qualitative and quantitative evaluations and synthesize their results to iteratively guide the search. Experiments on multi-dimensional ordinary differential equation benchmarks demonstrate that DoLQ achieves superior performance compared to existing methods, not only attaining higher success rates but also more accurately recovering the correct symbolic terms of ground truth equations. Our code is available at https://github.com/Bon99yun/DoLQ.

Sum Kyun Song, Bong Gyun Shin, Jae Yong Lee

Department of Artificial Intelligence, Chung-Ang University, Seoul, Republic of Korea · Department of AI Big Data, Daejin University, Pocheon-si, Republic of Korea.

AI Agents for Scientific DiscoveryAutonomous Scientific DiscoveryLarge Language Model-Guided Scientific DiscoveryODE DiscoverySymbolic RegressionOrdinary Differential Equations
AskPDF
Sep 29, 2026

Identifying ODEs from Unstructured Data with Causal Representation Learning

We study the problem of recovering the governing ODE of a dynamical system from unstructured, high-dimensional observations such as images. Existing methods for ODE discovery typically assume direct measurements of the variables, or do not provide theoretical guarantees on the learned variables and equations. While Causal Representation Learning (CRL) methods provide guarantees on identifying variables from high-dimensional observations up to component-wise diffeomorphisms, we show that in general these variables cannot be used directly as input to equation discovery methods, which typically assume that the variables will lead to sparse equations. So we introduce SParse Equivalent Equation Discovery AutoEncoder (SPEED-AE), a framework that combines a pretrained CRL method with a component-wise autoencoder that learns transformations of variables that are amenable to sparse ODE discovery. We show that for polynomial ODEs, this additional step allows us to restrict the identifiability of each variable from polynomial to monomial diffeomorphisms. Experiments on Lotka-Volterra, Lorenz, and a two-pendulum system show that SPEED-AE improves on the disentanglement of the CRL methods and that it recovers ODEs that are closest to the ground truth, while achieving state-of-the-art forecasting performance.

Alessandro Trenta, Riccardo Massidda, Davide Bacciu +1

Department of Computer Science University of Pisa · Saarland Informatics Campus Saarland University & University of Amsterdam

Causal Representation LearningRepresentation IdentifiabilitySparse Identification of Nonlinear DynamicsODE DiscoveryOrdinary Differential Equations
AskPDF
May 13, 2026

MPINeuralODE: Multiple-Initial-Condition Physics-Informed Neural ODEs for Globally Consistent Dynamical System Learning

Neural ordinary differential equations (Neural ODEs) often fit training trajectories while generalizing poorly to unseen initial conditions and long horizons. We propose MPINeuralODE, which combines a soft physics-informed residual with a Multiple-Initial-Condition (MIC) multiple-shooting curriculum whose ingredients are structurally complementary: the physics term anchors the vector-field magnitude on the support that MIC enlarges. We evaluate along three axes: out-of-sample error, long-horizon stability, and Hamiltonian drift, which together expose whether the learned dynamics recover the underlying vector field. On Lotka-Volterra, MPINeuralODE achieves the lowest out-of-sample and long-horizon MSE among data-driven methods, with a 24% reduction over the baseline Neural ODE, while essentially matching the PINN ablation on Hamiltonian drift.

Lake Yang, Antonio Malpica-Morales, Frank Ioannis Papadakis Wood +1

Department of Chemical Engineering, Imperial College London, London SW7 2AZ, United Kingdom

Neural Network GeneralizationSystem IdentificationPhysics-Informed MLNeural ODEsOrdinary Differential Equations
AskPDF