Sparse Equation Matching: A Derivative-Free Learning for General-Order Dynamical Systems
Authors: Jiaqiang Li, Jianbin Tan, Xueqin Wang
Organizations: School of Management, University of Science and Technology of China, Anhui, Hefei, China · Department of Biostatistics and Bioinformatics, Duke University, Durham, NC, USA
Equation discovery is a fundamental learning task for uncovering the underlying dynamics of complex systems, with wide-ranging applications in areas such as brain connectivity analysis, climate modeling, gene regulation, and physical simulation. However, many existing approaches rely on accurate derivative estimation and are limited to first-order dynamical systems, restricting their applicability in real-world scenarios. In this work, we propose Sparse Equation Matching (SEM), a unified framework that encompasses several existing equation discovery methods under a common formulation. SEM introduces an integral-based sparse regression approach using Green's functions, enabling derivative-free estimation of differential operators and their associated driving functions in general-order dynamical systems. The effectiveness of SEM is demonstrated through extensive simulations, benchmarking its performance against derivative-based approaches. We then apply SEM to electroencephalographic (EEG) data recorded during multiple oculomotor tasks, collected from 52 participants in a brain-computer interface experiment. Our method identifies active brain regions across participants and reveals task-specific connectivity patterns. These findings offer valuable insights into brain connectivity and the underlying neural mechanisms.
Figures & tables
Figure 1: True trajectories and their derivatives, along with corresponding estimates obtained from noisy observations using RKHS regression. In these examples, the signal-to-noise ratio is set to 0.15 , and the sample sizes are n=50,150,250 , with observation points evenly spaced over the time interval [0,10] . Blue triangles denote the observed data points.
Algorithm 1 Sparse Equation Matching for General-Order Dynamical Systems
Figure 2: Relative error rate (RER) of the estimated driving functions under various sample sizes n and contamination levels γ (main title of each subgraph).
Figure 3: True and average estimated vector fields for the nonlinear pendulum under the setting of n=150 and γ=0.05 . The color indicates the magnitude of the arrows.
Figure 4: (A) The directed interaction network; Each node represents a neural mass unit whose dynamics are governed by local excitation, inter-node directional coupling, and proportional damping that stabilizes oscillatory amplitudes. The direction between two nodes indicates that the trajectory of one node is affected by that of the other according to the dynamic directional model. (B) The trajectories generated from the dynamic directional model.
Figure 5: Mean accuracy of the estimated networks under different sample sizes n and noise levels γ (indicated in the title of each subfigure).
Figure 6: True and average estimated vector fields for X19 under three different values of b , based on 2000 simulation runs with n=150 and γ=0.05 . The color indicates the magnitude of the arrows.
Figure 7: Relative error rate (RER) of the estimated driving functions for different numbers of observed time points n=15,20,25,30 under contamination level γ=0.05 . SEM performs pre-estimation using an RKHS approach, whereas SEM-FPCA employs FPCA for pre-estimation.
Figure 8: Smoothed and predicted dynamics of the eye-blinking task for EEG channels C3, CP5, and FT7 from participants h=18,25,37 . The dashed vertical lines mark the transition from the training interval ( t<4 ) to the validation interval ( t≥4 ).
Figure 9: Boxplots of relative prediction errors (RPE) for 52 participants across three oculomotor tasks: eye blinking (blinking), horizontal eye movement (horizontal), and vertical eye movement (vertical). The RPE is evaluated on the held-out validation interval [4,5] .
Figure 10: Directed brain networks for the eye-blinking, horizontal eye movement, and vertical eye movement tasks. Edge color represents the empirical probability of occurrence of each edge across participants.
Appendix figures & tables5 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 11: True and average estimated vector fields for the pendulum system with nonlinear damping under the setting n=150 and γ=0.05 . The color indicates the magnitude of the velocity vectors.
Figure 12: Directed brain networks for different tasks: (A) resting state, (B) eye-blinking, (C) horizontal eye movement, (D) vertical eye movement. Edge color indicates the empirical probability of each edge occurring across participants.
Figure 13: A comparison of centrality measures. The diagrams display rank-ordered distributions of betweenness centrality, closeness centrality, in-degree, and out-degree for the network associated with the eye-blinking task, as shown in Figure 10 .
Figure 14: A comparison of centrality measures. Diagrams show rank-ordered distributions of betweenness, closeness, in degree and out degree for the network of horizontal eyeball movement task shown in Figure 10
Figure 15: A comparison of centrality measures. Diagrams show rank-ordered distributions of betweenness, closeness, in degree and out degree for the network of vertical eyeball movement task shown in Figure 10
Modeling neural population dynamics is crucial for foundational neuroscientific research and various clinical applications. Conventional latent variable methods typically model continuous brain dynamics through discretizing time with recurrent architecture, which necessarily results in compounded cumulative prediction errors and failure of capturing instantaneous, nonlinear characteristics of EEGs. We propose ODEBRAIN, a Neural ODE latent dynamic forecasting framework to overcome these challenges by integrating spatio-temporal-frequency features into spectral graph nodes, followed by a Neural ODE modeling the continuous latent dynamics. Our design ensures that latent representations can capture stochastic variations of complex brain states at any given time point. Extensive experiments verify that ODEBRAIN can improve significantly over existing methods in forecasting EEG dynamics with enhanced robustness and generalization capabilities.
Haohui Jia, Zheng Chen, Lingwei Zhu +6
Information Science and Techinology, Hokkaido University, Japan · SANKEN, The University of Osaka, Japan · Great Bay University, China +1
We develop the Sparse Orthogonal Regression Technique (SORT), a sparse spectral framework for learning orthonormal-basis expansions from noisy and irregularly sampled data. SORT estimates expansion coefficients directly from observations using L1-regularized regression, avoiding explicit quadrature or analytic inner-product evaluation. The central application is data-driven discovery of ordinary differential equations: vector fields are represented in chosen orthogonal bases and learned as sparse coefficient expansions. This provides a complementary route to symbolic regression, grammar-based discovery, and SINDy-style sparse identification by first recovering a compact spectral representation, which can later guide searches for simpler analytic forms. Across the dynamical-system experiments, SORT matches or improves upon library-based sparse-regression baselines when the basis is well adapted to the problem, and shows more stable degradation under sparse sampling, noisy derivative estimates, and representation mismatch. Specific examples illustrate why this representation is useful: if a finite library misses the problem-specific nonlinearity, the resulting model can fail. SORT is not immune to mismatch, but it shifts the problem away from brittle selection among generic terms to basis design adapted to the problem domain. The experiments also show that dominant low-order coefficients persist as model order increases, supporting order-consistent model growth. Beyond equation discovery, the same learned expansion supports nonlinear approximation and estimation of complex, high-dimensional integrals by coefficient readout. Overall, SORT provides a reusable intermediate representation for system identification, approximation, and integration, while making basis design an explicit part of the scientific modeling problem.
Sabin Roman, Ljupco Todorovski, Saso Dzeroski
Department of Knowledge Technologies, Jožef Stefan Institute · Faculty of Mathematics and Physics, University of Ljubljana
Discovering governing equations from observational data remains a fundamental challenge in scientific modeling, particularly when the underlying mathematical structure is unknown. Traditional sparse identification methods like SINDy excel at discovering parsimonious models but require researchers to specify candidate basis functions a priori, a limitation that often leads to model failure when critical terms are omitted or when systems exhibit unconventional dynamics. Purely symbolic regression approaches offer unlimited flexibility but struggle with noise sensitivity and frequently produce overly complex, unstable equations. We present AutoSINDy, a hybrid Discovery-then-Solve framework that combines the exploratory power of symbolic regression with the robust sparsity-promoting capabilities of SINDy. Our method operates in three stages: (1) PySR-based symbolic regression discovers candidate functional forms from bootstrapped data chunks; (2) a curation pipeline decomposes, expands, and filters these expressions using collinearity analysis to construct a minimal yet comprehensive library; and (3) SINDy identifies sparse governing equations from this custom-tailored library. Extensive experiments across canonical nonlinear systems demonstrate that AutoSINDy consistently recovers ground-truth equations even under high observational noise, achieving a ground-truth recovery rate of 92.8% across all trials. Compared with standard SINDy using enriched libraries and standalone symbolic regression, AutoSINDy achieves higher predictive accuracy, superior generalization to unseen trajectories, and substantially lower symbolic complexity.
Mohammad Amin Basiri, Charles Nicholson
Data Science and Analytics Institute, University of Oklahoma, Norman, OK, USA · School of Industrial and Systems Engineering, University of Oklahoma, Norman, OK, USA