From Overparameterised Expressions to Parsimonious Models: Fisher-Geometric Pruning for Symbolic Regression

Veronica Guidetti; Luca La Rocca; Fabrício Olivetti de França; Gabriel Kronberger

cs.LGOct 17, 2026

From Overparameterised Expressions to Parsimonious Models: Fisher-Geometric Pruning for Symbolic Regression

Authors: Veronica Guidetti, Luca La Rocca, Fabrício Olivetti de França, Gabriel Kronberger

Organizations: Department of Physics, Informatics and Mathematics, University of Modena and Reggio Emilia, via G. Campi 213/b, Modena, 41126, Italy · Department of Education and Humanities, University of Modena and Reggio Emilia, viale Timavo 93, Reggio nell’Emilia, 42121, Italy · Center for Mathematics, Computation and Cognition (CMCC), Federal University of ABC, Avenida dos Estados, 5001, Santo Andre, 09280-560, Brazil · University of Applied Sciences Upper Austria, Digital Campus Hagenberg, Softwarepark 11, Hagenberg, 4232, Austria

Abstract

Genetic programming symbolic regression (GPSR) often returns expressions that are larger than necessary, with dormant branches and finely tuned algebraic configurations that fit sampling noise. Existing complexity-control strategies, such as multi-objective size-accuracy search or information criteria embedded in the fitness function, act on syntactic representations during model search. We introduce Fisher-Geometric pruning (FGP), a post-hoc model-reduction framework that acts on fitted models and can complement these techniques. FGP combines local information geometry at the maximum-likelihood solution with explicit compatibility tests for interpretable parametric boundaries. A taxonomy of parametric directions separates local sensitivity from proximity to reducible boundaries, enabling the treatment of unidentifiable directions and reductions through zero constraints, asymptotic limits, and signed parameter identifications. Candidate reductions are realised by symbolic contraction and refitting, then accepted only when their cross-validated predictive risk remains within a prescribed tolerance. We evaluate FGP on controlled diagnostic problems and Pareto fronts generated by GPSR. Across both settings, FGP substantially reduces structural complexity while preserving, and often improving, out-of-sample accuracy. Comparisons with competing pruning strategies and ablations show that, by combining geometric screening, boundary compatibility, and predictive validation, FGP yields the most consistent joint gains in parsimony and generalisation.

Figures & tables

Family	Constraint $c(\boldsymbol{\theta})$	Gradient $\nabla c(\boldsymbol{\theta})$	Statistic $d_{c}^{2}$
Zeroing	$\theta_{k}$	$e_{k}$	$\dfrac{\hat{\theta}_{k}^{2}}{\big(\widehat{\mathcal{I}}_{[n]}^{-1}\big)_{kk}}$
Infinite saturation	$\phi_{k}\mp\frac{\pi}{2}$ , $\phi_{k}=\arctan\theta_{k}$	$e_{k}$ (in $\phi$ )	$\dfrac{\big(\arctan\hat{\theta}_{k}\mp\frac{\pi}{2}\big)^{2}\,\big(1+\hat{\theta}_{k}^{2}\big)^{2}}{\big(\widehat{\mathcal{I}}_{[n]}^{-1}\big)_{kk}}$
Signed identification	$\theta_{k}\mp\theta_{h}$	$e_{k}\mp e_{h}$	$\dfrac{(\hat{\theta}_{k}\mp\hat{\theta}_{h})^{2}}{(e_{k}\mp e_{h})^{\top}\widehat{\mathcal{I}}_{[n]}^{-1}(e_{k}\mp e_{h})}$

Table 1: Boundary families for the identifiable directions, with their constraint functions and corresponding compatibility statistics. Here,

e_{k}

denotes the

k

-th canonical basis vector. All statistics are screened against the common threshold

\tau

of section 3.2 .

Variant	Candidate selection	Allowed moves	Acceptance
FGP (ours)	staged according to eq. 18 ; null-space nomination followed by boundary screens	$0,1,\pm\infty$ , signed identification	CV admissibility eq. 15 + BIC tie-break
sloppy_only	parameter with largest loading on the current smallest-eigenvalue direction	same boundary families as FGP for the selected parameter	CV admissibility + BIC tie-break
zero_only	all parameters satisfying $d_{0}^{2}<\tau$ null-space/eigenvalue gating disabled	$\theta_{k}\to 0$ only	CV admissibility + BIC tie-break
magnitude	smallest $\|\hat{\theta}_{k}\|$ below $\tau_{\mathrm{mag}}=f_{\mathrm{mag}}\max_{h}\|\hat{\theta}_{h}\|$	$\theta_{k}\to 0$ only	threshold only; no geometric or predictive test
bic_only	identical to FGP	identical to FGP	$\Delta\mathrm{BIC}<B_{\mathrm{acc}}$ ; CV not used

Table 2: Alternative reduction operators. All variants share the fitting, contraction, and refitting machinery of algorithm 1 .

Figure 2: Problem #5, Bessel approximation. Top (bottom) row represents high (low) noise regimes. Left: data and fitted models before and after reduction. Center: ordered

\ln\lambda_{i}

\widehat{\mathcal{I}}

after each accepted move. Right: CV and BIC along the reduction trajectory. A single representative run is shown. The trajectories illustrate how improvements in full-sample fit can fail to persist under fold-wise refitting, leading BIC and cross-validated predictive risk to favour different reductions.

Figure 3: Aggregate results on the controlled suite. Left: Mean recovery and sufficiency rates over the

14

M

-closed configurations. Filled markers denote recovery and open markers sufficiency; their difference measures residual redundancy. Right: net Pareto-dominance rate

\Delta_{\mathrm{dom}}

on (complexity,

\mathrm{MSE}_{\star}

) over all

20

configurations. Points denote individual configurations, bars the median, and bands the interquartile range. Positive values favour FGP .

	Recovery		Complexity		$\mathrm{MSE}_{\star}$		Pareto
Comparison	median $\Delta_{\mathrm{rec}}$	$p_{\mathrm{Holm}}$	$r$	$p_{\mathrm{Holm}}$	$r$	$p_{\mathrm{Holm}}$	median $\Delta_{\mathrm{dom}}$	$p_{\mathrm{Holm}}$
FGP vs. zero_only	$+0.33$	$0.016$	$+0.53$	$0.116$	$+0.30$	$0.298$	$+0.17$	$0.024$
FGP vs. sloppy_only	$+0.40$	$0.009$	$+1.00$	$4.8\!\cdot\!10^{-4}$	$+0.97$	$3.8\!\cdot\!10^{-5}$	$+0.72$	$3.5\!\cdot\!10^{-4}$
FGP vs. magnitude	$+0.47$	$0.009$	$+0.75$	$0.016$	$+0.77$	$0.004$	$+0.57$	$0.001$
FGP vs. bic_only	$+0.00$	$0.873$	$+0.75$	$0.116$	$-0.40$	$0.298$	$+0.02$	$0.519$

Table 3: Direct paired comparisons of FGP against the competing operators on the controlled suite. Recovery is evaluated over the

14

M

-closed configurations; all other metrics use the full set of

20

configurations. Positive

\Delta_{\mathrm{rec}}

r

, and

\Delta_{\mathrm{dom}}

favour FGP .

p

-values are Holm-corrected within each metric; boldface marks

p_{\mathrm{Holm}}<0.05

Figure 4: Effect of the FGP operator on the six GPSR populations. Left and center panel: relative percentage change in node count versus noiseless test MSE for expressions on which at least one reduction is accepted. Negative values indicate lower complexity and lower test error, respectively. Small markers denote individual expressions; large markers and bars show the coordinate-wise medians and interquartile ranges. Open triangles indicate values clipped for display only. Right panel: composition of each complete population into untouched, modified-improved, and modified-worsened expressions. Labels report

P(\mathrm{MSE}_{\star}\text{ improves}\mid\text{modified})

Population	$N$	zero_only	sloppy_only	magnitude	bic_only
Friedman, $\sigma=0.5$	$42\text{--}43$	$+0.07\,{\scriptscriptstyle(8/5)}$	$+0.67\,{\scriptscriptstyle(28/0)}$	$+0.29\,{\scriptscriptstyle(12/0)}$	$-0.02\,{\scriptscriptstyle(7/8)}$
Friedman, $\sigma=1.0$	$58$	$+0.22\,{\scriptscriptstyle(25/12)}$	$+0.60\,{\scriptscriptstyle(36/1)}$	$+0.14\,{\scriptscriptstyle(19/11)}$	$+0.21\,{\scriptscriptstyle(26/14)}$
Friedman, $\sigma=13.6$	$50\text{--}51$	$+0.27\,{\scriptscriptstyle(23/9)}$	$+0.34\,{\scriptscriptstyle(27/10)}$	$+0.54\,{\scriptscriptstyle(28/1)}$	$+0.33\,{\scriptscriptstyle(25/8)}$
Additive, $n=50$	$70$	$-0.16\,{\scriptscriptstyle(9/20)}$	$+0.17\,{\scriptscriptstyle(24/12)}$	$-0.20\,{\scriptscriptstyle(13/27)}$	$+0.23\,{\scriptscriptstyle(25/9)}$
Additive, $n=100$	$62$	$+0.16\,{\scriptscriptstyle(21/11)}$	$+0.23\,{\scriptscriptstyle(18/4)}$	$-0.08\,{\scriptscriptstyle(12/17)}$	$+0.35\,{\scriptscriptstyle(30/8)}$
Additive, $n=1000$	$63\text{--}67$	$+0.31\,{\scriptscriptstyle(32/11)}$	$+0.25\,{\scriptscriptstyle(21/4)}$	$+0.24\,{\scriptscriptstyle(28/12)}$	$+0.40\,{\scriptscriptstyle(29/4)}$

Table 4: Expression-wise Pareto-dominance comparison of each variant against FGP on the final SR populations, using

(\mathrm{nodes},\mathrm{MSE}_{\star})

as objective pairs.

\Delta_{\mathrm{dom}}

is the fraction of expressions dominated by FGP minus the fraction dominated by the variant; parentheses report the corresponding counts ( FGP dominates / variant dominates), with the remainder being incomparable. Positive values therefore favor FGP . Boldface indicates statistically significant net dominance in favour of FGP (

p_{\mathrm{Holm}}<0.05

Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

#	Name	Truth $f^{\star}(x)$	Candidate $\hat{f}(\boldsymbol{\theta},x)$	Domain	$p$	$\sigma_{\mathrm{lo}}/\sigma_{\mathrm{hi}}$	$M$ -reg.
1	Nested sine–quadratic	$0.7x^{2}+2\sin(1.5x)$	$\theta_{0}\sin(\theta_{1}x)+\theta_{2}x^{2}+\theta_{3}x^{3}+\dfrac{\theta_{4}}{1+e^{-\theta_{5}x}}+e^{\theta_{6}x+\theta_{7}}$	$x\in(0,5]$	8	$0.5/1.5$	closed ⋆
2	Expansion 1D	$e^{0.7x}$	$\displaystyle\sum_{h=0}^{5}\theta_{h}x^{h}+\theta_{6}\sin(\theta_{7}x)+\dfrac{\theta_{8}}{1+e^{-\theta_{9}x}}$	$x\in(0,1.5]$	10	$0.05/0.2$	open
3	Expansion 2D	$e^{0.7x_{1}}$	$\displaystyle\sum_{h\in\{0,1,2,3,5\}}\theta_{h}x_{1}^{h}+\theta_{4}x_{2}^{4}+\theta_{6}\sin(\theta_{7}x_{1})+\dfrac{\theta_{8}}{1+e^{-\theta_{9}x_{2}}}$	$x_{1}\!\in\!(0,1.5]$ , $x_{2}\!\in\!(0,2.5]$	10	$0.05/0.2$	open
4	Near-alias Fourier	$\sin(0.65x+0.5)$	$\theta_{0}\sin(\theta_{1}x+\theta_{2})+\theta_{3}\sin(\theta_{4}x)+\theta_{5}+\theta_{6}x$	$x\in[0,30]$	7	$0.05/0.2$	closed
5	Bessel approximation	$1.3\,J_{0}(1.6x)$	$\theta_{0}e^{\theta_{1}x}\cos(\theta_{2}x+\theta_{3})+\theta_{4}+\dfrac{\theta_{5}+\theta_{6}x+\theta_{7}x^{2}}{\theta_{8}x+\theta_{9}x^{2}}$	$x\in[0.01,8]$	10	$0.05/0.3$	open
6	Rational function	$\dfrac{1.5x}{1+0.8x}$	$\dfrac{\theta_{0}x+\theta_{1}+\theta_{2}x^{2}}{\theta_{3}+\theta_{4}x+\theta_{5}x^{2}}+\dfrac{\theta_{6}x}{\theta_{7}+\theta_{8}x}$	$x\in[0,3]$	9	$0.01/0.2$	closed

Appendix

Table C.1: Definitions of the ten controlled diagnostic problems.

p

denotes the number of free parameters in the candidate expression;

n=100

in all cases. The columns

\sigma_{\mathrm{lo}}/\sigma_{\mathrm{hi}}

report the two observational noise levels. Inputs are sampled on a regular grid. The

M

-regime column uses closed ⋆ to indicate that the data-generating function belongs only to the closure of the candidate family and is recovered through an asymptotic parameter limit.

Figure D.1: Sensitivity of FGP on the controlled problems. Top: median ratio to the reference configuration over units whose outcome changes; bands show the interquartile range and hollow markers denote settings affecting fewer than

5\%

of all units. Bottom: fractions of all units for which each metric increases or decreases. The two rows therefore show effect magnitude and frequency, respectively.

Configuration	$n$	median $\Delta$ ( $r$ )	worse/tie/better	$p_{\text{Holm}}$
complexity – node count
$\beta$ : CV acceptance tolerance (reference $\beta=0.1$ )
$\beta=0$	$51$	$+6$ $(+0.88)$	$36/13/2$	$\mathbf{1.2\!\cdot\!10^{-5}}$
$\beta=0.02$	$53$	$+4$ $(+0.78)$	$32/18/3$	$\mathbf{2.0\!\cdot\!10^{-4}}$
$\beta=0.05$	$53$	$+4$ $(+0.58)$	$16/32/5$	$\mathbf{0.019}$
$\beta=0.2$	$52$	$-10$ $(-0.83)$	$3/34/15$	$\mathbf{0.004}$

Appendix

Table D.1: Sensitivity analysis of the FGP operator on the Friedman

\sigma=1.0

SR Pareto front. Each hyperparameter setting is compared expression-wise with the reference configuration. Entries report the median paired difference

m_{\mathrm{grid}}-m_{\mathrm{ref}}

, computed over expressions for which the two configurations yield different outcomes. Expressions left unreduced by both configurations are excluded from the corresponding comparison.

Figure E.1: Computational cost on the six Operon Pareto front. Left: wall-clock pruning time per expression across methods. Blocks show medians and interquartile ranges. Center: runtime of FGP versus the number $p$ of parameters in the starting expression; black markers show binned medians and interquartile ranges. Right: expression-wise retained parameter count and runtime of each variant relative to FGP ; large markers show per-variant medians.

Symbolic Regression

Symbolic regression searches data for compact mathematical expressions that balance predictive accuracy with interpretability.

See all 144 papers

Apr 19, 2026

On the Generalization Bounds of Symbolic Regression with Genetic Programming

Symbolic regression (SR) with genetic programming (GP) aims to discover interpretable mathematical expressions directly from data. Despite its strong empirical success, the theoretical understanding of why GP-based SR generalizes beyond the training data remains limited. In this work, we provide a learning-theoretic analysis of SR models represented as expression trees. We derive a generalization bound for GP-style SR under constraints on tree size, depth, and learnable constants. Our result decomposes the generalization gap into two interpretable components: a structure-selection term, reflecting the combinatorial complexity of choosing an expression-tree structure, and a constant-fitting term, capturing the complexity of optimizing numerical constants within a fixed structure. This decomposition provides a theoretical perspective on several widely used practices in GP, including parsimony pressure, depth limits, numerically stable operators, and interval arithmetic. In particular, our analysis shows how structural restrictions reduce hypothesis-class growth while stability mechanisms control the sensitivity of predictions to parameter perturbations. By linking these practical design choices to explicit complexity terms in the generalization bound, our work offers a principled explanation for commonly observed empirical behaviors in GP-based SR and contributes towards a more rigorous understanding of its generalization properties.

Masahiro Nomura, Ryoki Hamano, Isao Ono

Institute of Science Tokyo, Yokohama, Japan · CyberAgent, Shibuya, Japan

Genetic Programming Statistical Learning Theory Generalization Bounds Symbolic Regression

Ask PDF

May 21, 2026

Guiding Multi-Objective Genetic Programming with Description Length Improves Symbolic Regression Solutions

Symbolic regression with genetic programming (GPSR) may suffer from overfitting and structural bloat, especially when noise is present. In this paper we evaluate description length (DL) and fractional Bayes factor (FBF) criteria as principled, data-efficient alternatives to heuristics for selecting compact expressions that generalise well. We implement DL using a Fisher-information-based parameter encoding and compare it to AIC and BIC across multiple datasets, including noisy synthetic benchmarks and real-world regression problems. We study three search/selection strategies: (i) multi-objective search for accuracy and program length followed by DL/FBF selection; (ii) multi-objective search using DL directly as an objective; and (iii) single-objective optimisation with DL/FBF as the fitness. Across datasets we find that DL/FBF post-selection improves test performance compared to AIC/BIC baseline and that BIC in combination with the same function complexity penalty from DL/FBF produces similar results. In contrast, using DL/FBF directly as a fitness function in single-objective GPSR frequently induces premature convergence to overly simple models. We conclude with practical guidance for using DL/FBF as robust model-selection tools in genetic programming workflows.

Gabriel Kronberger, Fabricio Olivetti de Franca, Deaglan J. Bartlett +2

Heuristic and Evolutionary Algorithms Laboratory University of Applied Sciences Upper Austria Softwarepark 11, 4232 Hagenberg, Austria · Center of Mathematics, Computing and Cognition Federal University of ABC Av. dos Estados 5001, Santo Andre, 09280-560, SP, Brazil · Astrophysics, University of Oxford Denys Wilkinson Building, Keble Road, Oxford, OX1 3RH, UK +1

Minimum Description Length Genetic Programming Symbolic Regression

Ask PDF

Jun 20, 2026

Evolutional Math: Cross-Validated Island-Model Genetic Programming for Interpretable Symbolic Regression on Small, Wide Datasets

Symbolic regression via genetic programming routinely fails on small, wide datasets - a regime common in clinical-trial monitoring, biostatistics, and engineering pilot studies - by converging on bloated, overfit expressions that exploit correlation rather than prediction. We present Evolutional Math, an open-source genetic programming system that combines four design choices to yield compact, interpretable formulas in this regime. First, fitness is measured by R-squared on held-out cross-validation folds rather than Pearson correlation on the training set, eliminating single-variable shortcuts that correlate but mis-scale. Second, a multi-island architecture runs independent populations seeded with distinct operator subsets (algebraic, logarithmic, trigonometric, and full) with ring-topology migration every M generations, preventing the search from collapsing into one region of formula space. Third, a structural deduplication scheme treats formulas differing only in constants as equivalent, so the elite archive contains structurally distinct candidates rather than near-duplicate variants. Fourth, top-k individuals undergo numerical constant refinement via scipy L-BFGS-B after each migration phase, decoupling structure search from parameter fitting. We evaluate the system on synthetic benchmarks of the form log(x_i) * x_j / (x_k * c), trigonometric mixtures, and an anonymized clinical site-monitoring dataset with 24 rows and approximately 290 candidate numeric features. The system consistently recovers compact ground-truth structures with R-squared at or above 0.99 within tens of thousands of unique formula evaluations. A reference implementation is released under a noncommercial source-available license.

Artem Andrianov

Cyntegrity Germany GmbH, Hofheim am Taunus, Germany · Independent Researcher, Muensterer Str. 49, 65719 Hofheim am Taunus, Germany

Genetic Programming Symbolic Regression

Ask PDF

Abstract

Symbolic Regression

Explore similar work

On the Generalization Bounds of Symbolic Regression with Genetic Programming

Guiding Multi-Objective Genetic Programming with Description Length Improves Symbolic Regression Solutions

Evolutional Math: Cross-Validated Island-Model Genetic Programming for Interpretable Symbolic Regression on Small, Wide Datasets