LLM-ODE: Data-driven Discovery of Dynamical Systems with Large Language Models

Amirmohammad Ziaei Bideh; Jonathan Gryak

cs.LGMar 21, 2026

LLM-ODE: Data-driven Discovery of Dynamical Systems with Large Language Models

Authors: Amirmohammad Ziaei Bideh, Jonathan Gryak

Organizations: The Graduate Center, CUNY New York, New York, USA · Queens College and The Graduate Center, CUNY New York, New York, USA

Abstract

Discovering the governing equations of dynamical systems is a central problem across many scientific disciplines. As experimental data become increasingly available, automated equation discovery methods offer a promising data-driven approach to accelerate scientific discovery. Among these methods, genetic programming (GP) has been widely adopted due to its flexibility and interpretability. However, GP-based approaches often suffer from inefficient exploration of the symbolic search space, leading to slow convergence and suboptimal solutions. To address these limitations, we propose LLM-ODE, a large language model-aided model discovery framework that guides symbolic evolution using patterns extracted from elite candidate equations. By leveraging the generative prior of large language models, LLM-ODE produces more informed search trajectories while preserving the exploratory strengths of evolutionary algorithms. Empirical results on 91 dynamical systems show that LLM-ODE variants consistently outperform classical GP methods in terms of search efficiency and Pareto-front quality. Overall, our results demonstrate that LLM-ODE improves both efficiency and accuracy over traditional GP-based discovery and offers greater scalability to higher-dimensional systems compared to linear and Transformer-only model discovery methods.

Figures & tables

Figure 1. Schematic overview of LLM-ODE. (1) Given trajectory data from an unknown dynamical system, the observations are decomposed into state variables. (2) A large language model acts as an evolutionary operator, guiding the evolution of symbolic equation populations toward higher-fitness candidates. (3) The final system of equations is selected from the Cartesian product of equation-level Pareto fronts.

Figure 3. System discovery rate as a function of search iterations across various NMSE thresholds

\lambda

. Overall, LLM-ODE methods achieve higher discovery rates across all iteration limits. LLM-ODE methods enjoy a faster convergence rate than PySR.

Figure 4. The number of points along the system Pareto front of GP-based methods during training. The error bars represent 95% confidence intervals across systems. LLM-ODE methods are more efficient at obtaining a diverse Pareto-front as compared to stochastic GP.

Figure 5. System pool sizes from which the system Pareto-front is computed.

Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

Hyperparameter	Possible Values
Optimizer threshold	0.05, 0.1, 0.15
Optimizer alpha	0.025, 0.05, 0.075
Max iterations	20, 100
Polynomial degree	$1,2,\dots,10$
Basis functions	{ polynomials } { polynomials, sin, cos, exp} { polynomials, sin, cos, log, exp, sqrt, inverse }

Appendix

Table 2. Hyperparameter search space of SINDy.

Name	Train IV	Test IV	Equation
RC-circuit	$x_{0}=10.0$	$x_{0}=3.54$	$\dot{x}_{0}=0.303-0.361\,x_{0}$
Population growth (naive)	$x_{0}=4.78$	$x_{0}=0.87$	$\dot{x}_{0}=0.23\,x_{0}$
Population growth (carrying capacity)	$x_{0}=7.3$	$x_{0}=21.0$	$\dot{x}_{0}=0.79\,x_{0}\left(1-0.0135\,x_{0}\right)$
RC-circuit (nonlinear resistor)	$x_{0}=0.8$	$x_{0}=0.02$	$\dot{x}_{0}=-0.5+\dfrac{1}{1+1.65\,e^{-1.04\,x_{0}}}$
Velocity of a falling object	$x_{0}=0.5$	$x_{0}=73.0$	$\dot{x}_{0}=9.81-0.00212\,x_{0}^{2}$
Autocatalysis	$x_{0}=0.13$	$x_{0}=2.24$	$\dot{x}_{0}=-0.5\,x_{0}^{2}+2.1\,x_{0}$

Appendix

Table 3. The governing equations and the initial values of one-dimensional (

D=1

) dynamical systems.

Figure 6. The training trajectories of dynamical systems with

D=1

state variables.

Figure 7. The training trajectories of dynamical systems with

D=2

state variables.

Figure 8. The training trajectories of dynamical systems with

D=3

state variables.

Figure 9. The training trajectories of dynamical systems with

D=4

state variables.

Explore similar work

Jun 23, 2026

LLM-ACES: Closed-Loop Discovery of Dynamical Systems with LLM-Guided Adaptive Search

Recovering governing Ordinary Differential Equations (ODEs) from data is a central challenge in modeling dynamical systems across scientific domains. Existing approaches cast discovery as a static inference problem over fixed datasets, assuming that the observed trajectories are sufficiently informative. However, dynamical systems evolve over large state spaces, and limited data can make multiple equations observationally indistinguishable, leading to identifiability gaps and the recovery of incorrect governing equations. To address this, we introduce LLM-ACES, or LLM-guided Active Closed-loop Equation Search, a closed-loop framework that jointly optimizes symbolic hypothesis construction and adaptive data acquisition. In LLM-ACES, a large language model (LLM) proposes operator priors that partition the large search space into distinct regions, within which candidate equations are fit to the observed data. The disagreement among these candidates guides the acquisition of informative trajectories, creating a feedback loop that iteratively refines both the hypothesis space and the discovered dynamics. On 122 ODE systems spanning ODEBench and ODEBase, LLM-ACES achieves the lowest median NMSE, outperforming state-of-the-art baselines by several orders of magnitude while achieving a high symbolic accuracy of 46.2% and 52.4%, respectively. Our analysis further shows that LLM-ACES is sample-efficient, achieving better performance with one-tenth the data. Furthermore, LLM-ACES's feedback-driven data acquisition makes it robust to noise and recovers the correct symbolic structure, while baselines introduce spurious terms that fit the data locally but obscure the true governing relationships.

Nikhil Abhyankar, Sha Li, Sanchit Kabra +3

Virginia Tech

Nonlinear System Identification System Identification Autonomous Scientific Discovery Large Language Model-Guided Scientific Discovery ODE Discovery Symbolic Regression

Ask PDF

Jul 15, 2026

Automatic Ordinary Differential Equations Discovery For Biological Systems Using Large Language Model Powered Agentic System

Automatic scientific discovery has long been a goal of computational scholars - a machine that can discover nature's secrets on its own, moving computational systems beyond data-fitting tools toward the generation and refinement of mechanistic models of the universe. Recent advances in symbolic regression (SR) and large-language-model (LLM)-based agents suggest that such systems can recover equations from data, incorporate domain priors, and automate parts of the research workflow. However, most existing approaches either focus on narrow equation-discovery benchmarks or broad end-to-end automation pipelines, while biological systems remain comparatively underexplored. Here, we introduce the MEDA system, an LLM- and SR-powered agentic framework for discovering ordinary-differential-equation (ODE) models of biological and biologically inspired dynamical systems. MEDA retrieves background knowledge, defines admissible variables, generates mechanistic constraints, proposes candidate ODEs, and fits and evaluates them. We evaluate it across canonical model retrieval, reasoning-based extrapolation to unseen variants, and open-ended discovery, with and without experimental data. Across these settings, MEDA recovered the correct state variables, achieved strong structural recovery in retrieval and extrapolation tasks, and produced biologically plausible discovery-oriented models. Ablation and robustness analyses show that knowledge-guided formalization and mechanistic constraints are load-bearing components, whereas numerical fitting alone can preserve trajectory-compatible but biologically incorrect equations.

David Krongauz, Arad Zulti, Eran Segal +1

Department of Computer Science and Applied Mathematics, Weizmann Institute of Science, Rehovot, Israel · Department of Molecular Cell Biology, Weizmann Institute of Science, Rehovot, Israel · Mohamed bin Zayed University of Artificial Intelligence, Abu Dhabi, United Arab Emirates +2

Dynamical Systems AI Agents for Scientific Discovery Autonomous Scientific Discovery ODE Discovery Symbolic Regression

Ask PDF

Apr 17, 2026

Neuro-Symbolic ODE Discovery with Latent Grammar Flow

Understanding natural and engineered systems often relies on symbolic formulations, such as differential equations, which provide interpretability and transferability beyond black-box models. We introduce Latent Grammar Flow (LGF), a neuro-symbolic generative framework for discovering ordinary differential equations from data. LGF embeds equations as grammar-based representations into a discrete latent space and forces semantically similar equations to be positioned closer together with a behavioural loss. Then, a discrete flow model guides the sampling process to recursively generate candidate equations that best fit the observed data. Domain knowledge and constraints, such as stability, can be either embedded into the rules or used as conditional predictors.

Karin Yu, Eleni Chatzi, Georgios Kissas

Institute of Structural Engineering, ETH Zurich, Zurich, Switzerland · ETH AI Center, ETH Zurich, Zurich, Switzerland · Swiss Data Science Center, Zurich, Switzerland

Autonomous Scientific Discovery Neurosymbolic Learning ODE Discovery Symbolic Regression

Ask PDF