Coefficient Calibration as Selection Pressure in Symbolic Regression

Mattia Billa; Veronica Guidetti; Federica Mandreoli

cs.LGOct 7, 2026

Coefficient Calibration as Selection Pressure in Symbolic Regression

Authors: Mattia Billa, Veronica Guidetti, Federica Mandreoli

Organizations: Department of Physics, Informatics and Mathematics, University of Modena and Reggio Emilia, 41125 Modena, Italy

Abstract

In memetic symbolic regression, candidate structures are compared after coefficient calibration, so the calibration protocol itself contributes to evolutionary selection. Standard centralized calibration evaluates each structure at its pooled-sample optimum, ignoring how stable this calibration is under covariate shifts, and can thus favor structures whose fit relies on sample-specific coefficients. We propose Dirichlet-Sinkhorn Constant Averaging (DSCA), a calibration strategy that partitions the optimization data into equally sized subsets with different covariate distributions, calibrates each candidate independently on every partition, and evaluates it at the mean of the resulting parameters. We show that the excess loss of DSCA relative to centralized calibration vanishes at the population level for correctly specified, identifiable expressions, whereas under misspecification it persists when partition-specific calibrations do not aggregate to the pooled optimum. On synthetic benchmarks and ten real-world datasets, DSCA improves functional recovery and the accuracy-complexity trade-off over centralized Broyden-Fletcher-Goldfarb-Shanno and Levenberg-Marquardt calibration, under selection by negative log-likelihood and by the Akaike and Bayesian information criteria. Mechanism analyses associate the DSCA excess loss with the generalization gap and show that the effect is not reproduced by repeated centralized fitting. These results indicate that controlled heterogeneous calibration provides a complementary source of selection pressure in symbolic-regression search.

Figures & tables

Parameter	Value/Setting
Population size ( $\mu$ )	$100$
Offspring size ( $\lambda$ )	$100$
Number of generations	$100$
Initialization method	Random with decay
Parsimony	$0.8$
Parsimony decay	$0.85$

TABLE I: Evolutionary and numerical optimization hyperparameters.

Function	Generating Function	Distribution	$d$	Reference
$f_{1}(x)$	$8x_{1}^{2}+8x_{2}^{3}-15$	$x_{i}\sim\mathcal{U}(-3,3)$	2	Jin-2 [ 6 , 39 , 40 ]
$f_{2}(x)$	$2.5x_{1}^{4}-1.3x_{1}^{3}+0.5x_{2}^{2}-1.7x_{2}$	$x_{i}\sim\mathcal{U}(-3,3)$	2	Jin-1 [ 6 , 39 , 40 ]
$f_{3}(x)$	$1.35x_{1}x_{2}+5.5\sin((x_{1}-1)(x_{2}-1))$	$x_{i}\sim\mathcal{U}(-3,3)$	2	Jin-6 [ 6 , 39 , 40 ]
$f_{4}(x)$	$\frac{e^{-(x_{1}-1)^{2}}}{1.2+(x_{2}-2.5)^{2}}$	$x_{i}\sim\mathcal{U}(0.3,4)$	2	Vladislavleva-1 [ 41 ]
$f_{5}(x)$	$\frac{30(x_{1}-1)(x_{3}-1)}{(x_{1}-10)x_{2}^{2}}$	$x_{1},x_{3}\sim\mathcal{U}(0.05,2),x_{2}\sim\mathcal{U}(1,2)$	3	Vladislavleva-5 [ 41 ]

TABLE II: Generating functions and variable distributions for the synthetic benchmark track.

Dataset	$d$	$B$	$\mathbb{R}$	total	test
Fri_c1	25	0	25	1000	250
Fri_c2	5	0	5	1000	250
Pollen	4	0	4	3790	948
Puma8NH	8	0	8	8192	2048
Echo M.	9	3	6	17233	4309
Houses	8	0	8	20330	5083

TABLE III: Characteristics of the selected PMLB datasets.

		NLL		AIC		BIC
$\boldsymbol{N}$	Noise	BFGS	LM	BFGS	LM	BFGS	LM
100	Low	3/1/1	4/1/0	3/1/1	3/1/1	3/2/0	1/3/1
100	High	2/3/0	2/3/0	2/3/0	2/3/0	4/1/0	2/2/1
200	Low	3/2/0	3/2/0	2/3/0	3/2/0	2/3/0	2/2/1
200	High	3/2/0	3/1/1	3/2/0	3/2/0	3/2/0	3/1/1

TABLE IV: Win/tie/loss counts of DSCA against each baseline over the synthetic datasets, by sample size, noise level, and selection criterion.

	100 Samples			200 Samples
Algorithm	NLL	AIC	BIC	NLL	AIC	BIC
DSCA (our)	26	31	43	43	49	55
BFGS	14	14	30	28	31	48
LM	5	13	37	24	30	49

TABLE V: Aggregate success counts on synthetic datasets by sample size and selection criterion.

Fig. 1: Bayesian Wilcoxon signed-rank test results evaluating predictive accuracy (top panels) and model complexity (bottom panels) against the baselines.

	Algorithm	BFGS	DSCA (our)
Criterion	$N$ samples
NLL	100	0.26 $\pm$ 0.57	0.85 $\pm$ 0.74
	200	0.28 $\pm$ 0.60	0.40 $\pm$ 0.63
	500	0.02 $\pm$ 0.04	0.14 $\pm$ 0.40
AIC	100	0.30 $\pm$ 0.43	0.78 $\pm$ 0.66
	200	0.36 $\pm$ 0.69	0.54 $\pm$ 0.67

TABLE VI: Predictive accuracy (

R^{2}

) differences (

\Delta_{a}

) relative to the Levenberg-Marquardt baseline (Mean

\pm

Std).

	Algorithm	BFGS	DSCA (our)
Criterion	$N$ samples
NLL	100	-0.2 $\pm$ 1.1	-15.8 $\pm$ 10.8
	200	-0.6 $\pm$ 2.3	-7.5 $\pm$ 6.2
	500	-1.2 $\pm$ 2.9	-4.0 $\pm$ 2.8
AIC	100	-0.8 $\pm$ 2.8	-23.3 $\pm$ 11.4
	200	-2.1 $\pm$ 4.9	-15.1 $\pm$ 13.3

TABLE VII: Model complexity differences (

\Delta_{a}

) relative to the Levenberg-Marquardt baseline (Mean

\pm

Std).

Fig. 2: Algorithmic training time (in seconds) across different selection criteria and sample size regimes.

Fig. 3: Generalization gap of the pooled final populations, by the optimizer that produced each structure (200-sample regime, centralized re-fit).

Fig. 4: Excess loss $\Delta_{N^{\prime}}$ against the generalization gap for the pooled final populations, colored by the optimizer that produced the structure.

Fig. 5: Ablation study of the predictive accuracy ( $R^{2}$ ) regret gap under different partitioning strategies.

Fig. 6: Ablation study of the model complexity regret gap under different partitioning strategies.

Appendix figures & tables2 assets

Supplementary material from the paper’s appendix.

Appendix

Fig. 7: Success rates on synthetic datasets (

N=100

) under low and high noise regimes.

Fig. 8: Success rates on synthetic datasets (

N=200

) under low and high noise regimes.

Explore similar work

A Comparative Study of Model Selection Criteria for Symbolic Regression
May 11, 2026Ali Soltani, Gabriel Kronberger, Fabricio Olivetti de Franca +2Model Selection Symbolic Regression
Aarhus University Department of Mechanical Engineering Aarhus, Denmark · Heuristic and Evolutionary Algorithms Laboratory (HEAL) University of Applied Sciences Upper Austria Hagenberg, Austria · Federal University of ABC Center for Mathematics, Computing, and Cognition Brazil +1
SymbolicArena: A Unified Infrastructure for Benchmark Distillation and Dynamic Evaluation in Symbolic Regression
Sep 28, 2026Ziwen Zhang, Xiju Wu, Yuheng Jing +9Benchmark Design Automated Evaluation
School of Artiﬁcial Intelligence, University of Chinese Academy of Sciences · C2DL, Institute of Automation, Chinese Academy of Sciences · Nanjing University of Science and Technology +1
FunctionEvolve: Structure-Guided Symbolic Regression with LLMs
Jun 5, 2026Zeyu Xia, Jun Zhu, Dong YanGenetic Programming Large Language Model-Guided Scientific Discovery
Bosch Center for Artificial Intelligence · Department of Computer Science and Technology, Tsinghua University · Department of Computer Science and Technology +1

May 11, 2026, cs.LGJ/K move · Enter open · S save

A Comparative Study of Model Selection Criteria for Symbolic Regression

Ali Soltani, Gabriel Kronberger, Fabricio Olivetti de Franca, Mattia Billa+1

Aarhus University Department of Mechanical Engineering Aarhus, Denmark · Heuristic and Evolutionary Algorithms Laboratory (HEAL) University of Applied Sciences Upper Austria Hagenberg, Austria · Federal University of ABC Center for Mathematics, Computing, and Cognition Brazil+1

Model Selection Symbolic Regression

Open paper AskPDF arXiv

Effective model selection is critical in symbolic regression (SR) to identify mathematical expressions that balance accuracy and complexity, and have low expected error on unseen data. Many modern implementations of genetic programming (GP) for SR generate a set of Pareto optimal candidate solutions, but reliable automatic selection of solutions that generalize well remains an open issue. Current literature offers various information-theoretic and Bayesian approaches, yet comprehensive comparisons of their performance across different data regimes are limited. This study presents a systematic empirical comparison of widely used selection criteria: the Akaike information criterion (AIC), the corrected AIC (AICc), the Bayesian information criterion (BIC), minimum description length (MDL), as well as Efron's bootstrap estimate for the in-sample prediction error on seven synthetic datasets with Gaussian noise. We rank candidate expressions generated by perturbing ground-truth functions to assess generalization error and selection probability of the ground-truth expression. Our findings reveal that MDL consistently identifies models with the lowest test error and the shortest length across most datasets. While no single criterion dominates all results, MDL and BIC produced the highest probability of selecting the ground-truth expressions.

Abstract

Explore similar work

A Comparative Study of Model Selection Criteria for Symbolic Regression

SymbolicArena: A Unified Infrastructure for Benchmark Distillation and Dynamic Evaluation in Symbolic Regression

FunctionEvolve: Structure-Guided Symbolic Regression with LLMs