Equation discovery with Bayesian tree-adjoining grammars

Christopher A. Lindley; Nikolaos Dervilis; Keith Worden

stat.MLSep 25, 2026

Equation discovery with Bayesian tree-adjoining grammars

Authors: Christopher A. Lindley, Nikolaos Dervilis, Keith Worden

Organizations: Dynamics Research Group University of Sheffield Mappin St, Sheffield, UK

Abstract

Tree-Adjoining Grammars (TAGs) have recently been introduced to Nonlinear System Identification (NLSI) as a means of encoding an entire model class as a finite set of grammatical rules, from which candidate models are assembled as trees. Existing TAG-based identifiers rely on evolutionary optimisation and return point estimates of the model structure. This paper instead proposes the TAG framework within a Bayesian setting. A generative prior is defined over tree structures and their parameters, and a Reversible-Jump MCMC sampler with structure-preserving tree moves is used to infer the joint posterior over model structure, parameters and predictions. Two training objectives are considered; that is, a one-step-ahead objective with conjugate parameter proposals, and a simulation-based objective handled by likelihood-free inference. The approach is validated on a simulated polynomial NARX system, the Silverbox benchmark, and wave-loading data from the Christchurch Bay Tower, where embedding Morison's equation as a fixed initial tree yields a grey-box model that outperforms the physics-driven baseline. The results demonstrate that Bayesian TAGs are well suited to quantifying uncertainty in equation discovery for dynamical systems and to fitting physics-informed models.

Figures & tables

Panel of Figure 1 — Figure 1: (a) Derived tree structure of string: “Naturally, Keith likes cats”. (b) Tree-adjoining grammar used to parse the string. The “sub” in $\eta_{2}$ and $\eta_{3}$ means “subject” and not a substitution.

Figure 2: Initial tree

\eta_{1}

and auxiliary tree set

A

of TAG

G_{N}

. Symbol notation: “expr” - expression , “par” - parameter , “op” - operation , “

q^{-1}

” - backward time shift . Reproduced from [ 3 ] .

Figure 3: Illustration of the add/remove term move type.

Structure prior (both methods)
Hyperparameter	Notation	Value
Term rate (prior mean number of terms)	$\lambda_{t}$	$\{1,\,5,\,20\}$
Term degree rate	$\lambda_{d}$	$0.3$
Lag decay	$\lambda_{l}$	$0.6$
Pure-power mixture weight	$\lambda_{m}$	$0.5$
Pure-power degree rate	$\lambda_{p}$	$0.7$

Table 1: Hyperparameters of the chosen prior distributions.

Shared by both methods
Hyperparameter	Notation	Value
Number of samples (OSA / MPO)	$n_{s}$	$30{,}000$ / $60{,}000$
Burn-in samples (OSA / MPO)	$n_{\mathrm{burn}}$	$0.5n_{s}$ / $0.75n_{s}$
Thinning interval	$n_{\mathrm{thin}}$	$5$
Lag window	$\ell$	$10$
Stay rate	$p_{o}$	$0.30$

Table 2: Default hyperparameters defined for the RJ-MCMC samplers.

Method	$\lambda_{t}$	MAP	FWD	MAP	FWD	Time (h)	Rec.	Mass
		Selected		Pooled			Truth
OSA	1	1.154	1.166	1.154	1.160	0.04	3/4	0.747
	5	1.154	1.166	1.154	1.165	0.03	4/4	0.987
	20	1.155	1.168	1.155	1.174	0.05	3/4	0.678
MPO	1	3.218	2.667	4.046	2.165	0.64	0/4	0.000
	5	1.154	1.164	3.173	1.748	0.71	1/4	0.254

Table 3: Results from Case Study 1. All scores are provided by the NMSE on the test set.

Term	True	Mean	$95\%$ CI	Mean	$95\%$ CI
		OSA		MPO
$u_{i-1}$	$+0.500$	$+0.5024$	$[+0.4847,\,+0.5196]$	$+0.5070$	$[+0.4776,\,+0.5333]$
$u_{i-5}^{3}$	$-0.300$	$-0.3011$	$[-0.3062,\,-0.2958]$	$-0.3023$	$[-0.3111,\,-0.2930]$
$y_{i-1}$	$+0.300$	$+0.2961$	$[+0.2827,\,+0.3088]$	$+0.2924$	$[+0.2699,\,+0.3151]$
$\sigma_{\epsilon}$	$0.1561$	$0.1752$	$[0.1682,\,0.1829]$	$0.1663$	—

Table 4: Parameters identified on the validation-selected chain of each training objective, both at

\lambda_{t}=5

Figure 4: Comparison between response predictions and observations of the academic case study.

Method	$\lambda_{t}$	MAP	FWD	MAP	FWD	No. Terms	Time (h)	RMSE (mV)
		Selected		Pooled
OSA	1	0.0061	0.0092	0.0061	0.0087	19.5	0.27	0.419
	5	0.0055	0.0080	0.0200	0.0095	20.0	0.38	0.397
	20	0.0068	0.0073	0.0065	0.0097	19.5	0.29	0.442
MPO	1	7.2078	6.9726	8.6794	7.8068	3.0	7.50	14.358
	5	0.7404	0.3286	7.1003	4.1935	3.5	7.66	4.602

Table 5: Results from Case Study 2. All scores are provided by the NMSE on the test set. The number of terms here corresponds to the median across chains.

Figure 5: Comparison of the observations in the test set $\mathcal{D}_{\mathrm{test}}$ with predictions made by the validation-selected models.

Nonlinear identification method	Test RMS simulation (mV)
Proposed method	0.397
Genetic Programming NARMAX ( [ 3 ] )	0.360
GP-NARX ( [ 36 ] )	0.720
p-NARX ( [ 36 ] )	2.250
p-NLSS ( [ 37 ] )	0.260

Table 6: Performance of the identified model computed on the Silverbox test data and compared with other results in the literature.

Figure 6: Morison’s equation tree representation.

Figure 7: NMSE on the CBT test set and median number of terms for each method, training objective and $\lambda_{t}$ , compared against the Morison’s equation (white-box) baseline.

Figure 8: Comparison between observations of the CBT case study and response predictions.

Identification method	NMSE ( $\%$ )
Morison’s equation (white-box)	$19.463$
Proposed method $1$ (black-box MPO)	$16.730$
Proposed method $2$ (residual MPO)	$14.600$
Proposed method $3$ (grey-box MPO)	13.730
Polynomial NARMAX [ 2 ]	$21.430$
GP-NARX [ 43 ]	$19.520$

Table 7: Performance estimate comparison from identification methods used on the CBT test set. The proposed methods are reported for the MPO objective with

\lambda_{t}=20

Explore similar work

Bayesian Symbolic Regression for Missing Physics
Mar 16, 2026Arno StrouwenSymbolic Regression Uncertainty Quantification
Strouwen Statistics · Biosystems Department, KULeuven, Leuven, Belgium
Fast and principled equation discovery from chaos to climate
Apr 13, 2026Yuzheng Zhang, Weizhen Li, Rui CarvalhoSparse Identification of Nonlinear Dynamics Scientific Discovery
Department of Engineering, Durham University, Lower Mountjoy, South Road, Durham, DH1 3LE, United Kingdom. · College of Control Science and Engineering, Zhejiang University, 38 Zheda Rd.(Yuquan Campus), Hangzhou, 310027, P.R. China. · Institute for Data Science, Durham University, South Road, Durham, DH1 3LE, United Kingdom.
A Bayesian Filtering Approach for Learning Lagrangian Dynamics from Noisy Measurements
Jun 30, 2026Kundan Kumar, Shreya Das, Simo SärkkäBayesian Filtering Nonlinear System Identification
K. Kumar is with the School of Engineering and Applied Science, Ahmedabad University, India · K. Kumar and S. Särkkä are with the ELLIS Institute Finland and the Department of Electrical Engineering and Automation, Aalto University, Finland · S. Das is with the Department of Electrical Engineering, Indian Institute of Technology (BHU) Varanasi, India

Mar 16, 2026J/K move · Enter open · S save

Bayesian Symbolic Regression for Missing Physics

Arno Strouwen

Strouwen Statistics · Biosystems Department, KULeuven, Leuven, Belgium

Symbolic Regression Uncertainty Quantification

Open paper AskPDF arXiv

Model-based approaches for (bio)process systems often suffer from incomplete knowledge of the underlying physical, chemical, or biological laws. Universal differential equations, which embed neural networks within differential equations, have emerged as powerful tools to learn this missing physics from experimental data. However, neural networks are inherently opaque, motivating their post-processing via symbolic regression to obtain interpretable mathematical expressions. Genetic algorithm-based symbolic regression is a popular approach for this post-processing step, but provides only point estimates and cannot quantify the confidence we should place in a discovered equation. We address this limitation by applying Bayesian symbolic regression, which uses Reversible Jump Markov Chain Monte Carlo to sample from the posterior distribution over symbolic expression trees. This approach naturally quantifies uncertainty in the recovered model structure. We demonstrate the methodology on a Lotka-Volterra predator-prey system and then show how a well-designed experiment leads to lower uncertainty in a fed-batch bioreactor case study.

Abstract

Explore similar work

Bayesian Symbolic Regression for Missing Physics

Fast and principled equation discovery from chaos to climate

A Bayesian Filtering Approach for Learning Lagrangian Dynamics from Noisy Measurements