Ill-Conditioning in Dictionary-Based Dynamic-Equation Learning: A Systems Biology Case Study

Yuxiang Feng; Niall M Mangan; Manu Jayadharan

q-bio.QMMar 11, 2026

Ill-Conditioning in Dictionary-Based Dynamic-Equation Learning: A Systems Biology Case Study

Authors: Yuxiang Feng, Niall M Mangan, Manu Jayadharan

Organizations: Department of Engineering Sciences and Applied Mathematics, Northwestern University, Evanston, IL 60208, USA. · NSF-Simons National Institute for Theory and Mathematics in Biology, Chicago, IL, USA; Northwestern University, Center for Synthetic Biology, Evanston, IL, USA

Abstract

Data-driven discovery of governing equations from time-series data provides a powerful framework for understanding complex biological systems. Library-based approaches that use sparse regression over candidate functions have shown considerable promise, but they face a critical challenge when candidate functions become strongly correlated: numerical ill-conditioning. Poor or restricted sampling, together with particular choices of candidate libraries, can produce strong multicollinearity and numerical instability. In such cases, measurement noise may lead to widely different recovered models, obscuring the true underlying dynamics and hindering accurate system identification. Although sparse regularization promotes parsimonious solutions and can partially mitigate conditioning issues, strong correlations may persist, regularization may bias the recovered models, and the regression problem may remain highly sensitive to small perturbations in the data. We present a systematic analysis of how ill-conditioning affects sparse identification of biological dynamics using benchmark models from systems biology. We show that combinations involving as few as two or three terms can already exhibit strong multicollinearity and extremely large condition numbers. We further show that orthogonal polynomial bases do not consistently resolve ill-conditioning and can perform worse than monomial libraries when the data distribution deviates from the weight function associated with the orthogonal basis. Finally, we demonstrate that when data are sampled from distributions aligned with the appropriate weight functions corresponding to the orthogonal basis, numerical conditioning improves, and orthogonal polynomial bases can yield improved model recovery accuracy across two baseline models.

Figures & tables

Figure 1: Numerical ill-conditioning of candidate function libraries for baseline biological dynamical systems. Panels (a)–(b) show representative time-series data generated from numerical simulations of the Lotka–Volterra predator–prey system and a chemical reaction network (CRN), respectively. Panels (c)–(d) report the condition numbers of full polynomial function libraries constructed using monomial, Legendre, and Chebyshev bases as functions of library degree for the two systems. Panels (e)–(f) quantify the prevalence of ill-conditioning by counting the number of ill-conditioned two-term and three-term combinations in the candidate libraries, measured as the base-10 logarithm of the number of combinations exceeding an $\mathrm{R}^{2}$ threshold. Results are shown separately for the Lotka–Volterra system (left column) and the CRN model (right column).

System	True Equation		Discovered Equation		Missing Terms	Wrong Terms	Con([Missing, Wrong])
LV (Eq1)	$\dot{x}_{1}$	$\begin{aligned} {}=&\,0.2x_{1}-0.118x_{1}^{2}\\ &\,-0.1x_{1}x_{2}-0.2x_{1}x_{3}\end{aligned}$	$\dot{x}_{1}$	${}=0.119x_{1}^{3}-0.273x_{1}^{2}x_{3}$	$x_{1},\,x_{1}x_{2},$ $x_{1}x_{3},\,x_{1}^{2}$	$x_{1}^{2}x_{3},\,x_{1}^{3}$	$8.13e5$
LV (Eq2)	$\dot{x}_{2}$	${}=-0.1x_{2}+0.1x_{1}x_{2}$	$\dot{x}_{2}$	$\begin{aligned} {}=&\,-0.023x_{3}+0.031x_{1}x_{3}\\ &\,+0.015x_{1}^{2}x_{3}-0.02x_{1}x_{2}x_{3}\end{aligned}$	$x_{2},\,x_{1}x_{2}$	$x_{3},\,x_{1}x_{3},$ $x_{1}x_{2}x_{3},\,x_{1}^{2}x_{3}$	$3.04e3$
LV (Eq3)	$\dot{x}_{3}$	${}=-0.1x_{3}+0.4x_{1}x_{3}$	$\dot{x}_{3}$	$\begin{aligned} {}=&\,-0.098x_{3}+6.136x_{1}^{2}\\ &\,-2.02x_{1}^{3}-1.441x_{1}^{2}x_{3}\\ &\,+0.109x_{1}x_{3}^{2}\end{aligned}$	$x_{1}x_{3}$	$x_{1}^{2},\,x_{1}x_{3}^{2},$ $x_{1}^{2}x_{3},\,x_{1}^{3}$	$1.23e5$
CRN (Eq1)	$\dot{x}_{1}$	${}=1.10x_{3}-2.12x_{1}x_{3}$	$\dot{x}_{1}$	$\begin{aligned} {}=&\,-1.773x_{1}+0.664x_{1}^{2}\\ &\,+0.518x_{3}^{2}\end{aligned}$	$x_{3},\,x_{1}x_{2}$	$x_{1},\,x_{1}^{2},\,x_{3}^{2}$	$1.36e17$
CRN (Eq2)	$\dot{x}_{2}$	${}=2.61x_{3}-2.12x_{1}x_{2}$	$\dot{x}_{2}$	$\begin{aligned} {}=&\,0.485x_{1}^{2}-1.538x_{1}x_{3}\\ &\,-0.636x_{2}x_{3}+0.672x_{3}^{2}\\ &\,+0.907x_{3}x_{4}\end{aligned}$	$x_{3},\,x_{1}x_{2}$	$x_{1}x_{3},\,x_{2}x_{3},$ $x_{3}x_{4},\,x_{1}^{2},\,x_{3}^{2}$	$2.00e18$
CRN (Eq3)	$\dot{x}_{3}$	${}=2.12x_{1}x_{2}-2.61x_{3}$	$\dot{x}_{3}$	$\begin{aligned} {}=&\,-0.485x_{1}^{2}+1.538x_{1}x_{3}\\ &\,+0.636x_{2}x_{3}-0.672x_{3}^{2}\\ &\,-0.907x_{3}x_{4}\end{aligned}$	$x_{3},\,x_{1}x_{2}$	$x_{1}x_{3},\,x_{2}x_{3},$ $x_{3}x_{4},\,x_{1}^{2},\,x_{3}^{2}$	$2.00e18$

Table 1: Conditioning of error-associated subspaces in sparse identification of baseline dynamical systems. For each equation of the Lotka–Volterra (L-V) and chemical reaction network (CRN) baseline models, the table lists the true governing equation and the equation recovered by SINDy using a monomial polynomial library (degree 3 for L-V and degree 2 for CRN), together with the sets of false negatives or missing and false positives or wrong terms. The reported condition number corresponds to the submatrix formed by the candidate functions associated with these misidentified terms.

Figure 2: Conditioning of candidate function libraries across benchmark biological models as a function of model complexity. Panel (a) shows the dependence of the full library condition number on model complexity for each model. Panel (b) focuses on the conditioning of sub-matrices formed by features corresponding to false negatives or missing and false positives or wrong terms in the mis-identified model after sparse regression, with each point representing the mean across equations within a model and error bars indicating one standard deviation.

Figure 3: Distribution–basis alignment restores orthogonality, improves conditioning, and enables accurate sparse model recovery. Panels (a)–(d) compare the empirical distributions induced by system dynamics with the theoretical distributions associated with the weight functions required to preserve orthogonality for Legendre and Chebyshev bases. Solid curves represent empirical probability density functions constructed from simulated state trajectories generated by numerical simulation of the L-V system (left column) and a CRN (right column), reflecting the data distributions induced by the underlying dynamics and serving as proxies for experimentally observed data. Histograms depict data resampled from these trajectories according to the corresponding idealized distributions implied by the orthogonality weight functions (uniform for Legendre and arcsine for Chebyshev). Panels (e)–(f) report the condition numbers of full candidate function libraries constructed from monomial and orthogonal polynomial bases when evaluated on original data versus distribution-aligned samples. Panels (g)–(h) show the resulting model identification errors quantified as the number of incorrectly recovered equations. Candidate function libraries in panels (e)–(h) are constructed using polynomial bases of degree 5.

Sparse Identification of Nonlinear Dynamics

Sparse identification of nonlinear dynamics recovers compact governing equations from observed time series using data-driven candidate functions.

See all 33 papers

Sparse Orthogonal Regression Technique: A Spectral Framework for Equation Discovery, Approximation, and Integration
Aug 13, 2026Sabin Roman, Ljupco Todorovski, Saso DzeroskiSparse Identification of Nonlinear Dynamics Nonlinear System Identification
Department of Knowledge Technologies, Jožef Stefan Institute · Faculty of Mathematics and Physics, University of Ljubljana
Discovery of Nonlinear Dynamics with Automated Basis Function Generation
May 10, 2026Mohammad Amin Basiri, Charles NicholsonSparse Identification of Nonlinear Dynamics Nonlinear System Identification
Data Science and Analytics Institute, University of Oklahoma, Norman, OK, USA · School of Industrial and Systems Engineering, University of Oklahoma, Norman, OK, USA
Learning Dynamical Systems from Multiple Sparse Datasets: A Hierarchical Bayesian Modeling Approach
Jun 23, 2026Cristian Brugnara, Lea Multerer, Marco Forgione +1Irregular Time-Series Modeling Parameter Estimation
SUPSI, Dalle Molle Institute for Artificial Intelligence (IDSIA USI-SUPSI), Lugano, Switzerland

Aug 13, 2026J/K move · Enter open · S save

Sparse Orthogonal Regression Technique: A Spectral Framework for Equation Discovery, Approximation, and Integration

Sabin Roman, Ljupco Todorovski, Saso Dzeroski

Department of Knowledge Technologies, Jožef Stefan Institute · Faculty of Mathematics and Physics, University of Ljubljana

Sparse Identification of Nonlinear Dynamics Nonlinear System Identification Orthogonal Representation Learning ODE Discovery Ordinary Differential Equations

Open paper AskPDF arXiv

We develop the Sparse Orthogonal Regression Technique (SORT), a sparse spectral framework for learning orthonormal-basis expansions from noisy and irregularly sampled data. SORT estimates expansion coefficients directly from observations using L1-regularized regression, avoiding explicit quadrature or analytic inner-product evaluation. The central application is data-driven discovery of ordinary differential equations: vector fields are represented in chosen orthogonal bases and learned as sparse coefficient expansions. This provides a complementary route to symbolic regression, grammar-based discovery, and SINDy-style sparse identification by first recovering a compact spectral representation, which can later guide searches for simpler analytic forms. Across the dynamical-system experiments, SORT matches or improves upon library-based sparse-regression baselines when the basis is well adapted to the problem, and shows more stable degradation under sparse sampling, noisy derivative estimates, and representation mismatch. Specific examples illustrate why this representation is useful: if a finite library misses the problem-specific nonlinearity, the resulting model can fail. SORT is not immune to mismatch, but it shifts the problem away from brittle selection among generic terms to basis design adapted to the problem domain. The experiments also show that dominant low-order coefficients persist as model order increases, supporting order-consistent model growth. Beyond equation discovery, the same learned expansion supports nonlinear approximation and estimation of complex, high-dimensional integrals by coefficient readout. Overall, SORT provides a reusable intermediate representation for system identification, approximation, and integration, while making basis design an explicit part of the scientific modeling problem.

Abstract

Sparse Identification of Nonlinear Dynamics

Explore similar work

Sparse Orthogonal Regression Technique: A Spectral Framework for Equation Discovery, Approximation, and Integration

Discovery of Nonlinear Dynamics with Automated Basis Function Generation

Learning Dynamical Systems from Multiple Sparse Datasets: A Hierarchical Bayesian Modeling Approach