EvoMO-SR: Multiobjective LLM-based Evolution of Symbolic Expressions with substructure guidance

Cristina Rossetti; Anna V. Kononova; Thomas Bäck; Fei Liu; Niki Van Stein

cs.LGSep 28, 2026

EvoMO-SR: Multiobjective LLM-based Evolution of Symbolic Expressions with substructure guidance

Authors: Cristina Rossetti, Anna V. Kononova, Thomas Bäck, Fei Liu, Niki Van Stein

Organizations: Department of Automation and Computer Science Polytechnic Institute of Turin Corso Duca degli Abruzzi, 24, Torino, 10129, Italy · LIACS Leiden University Einsteinweg 55, 2333 CC, Leiden, The Netherlands · University of Zurich & ETH Zurich

Abstract

Symbolic Regression (SR) is a data-driven method for scientific discovery which searches for interpretable analytical relationships within data. Recently, Large Language Models (LLMs) have also had a significant impact on scientific discovery, enabling the automation of various stages of the process. For these reasons, the possibility of harnessing the embedded scientific knowledge and programming capabilities of LLMs to solve SR tasks has emerged, showing promising performance compared with traditional methods. We propose EvoMO-SR, a novel LLM-driven SR framework in which the LLM generates equation skeletons, with their coefficients fitted separately by an external optimizer. The framework includes a multi-objective survival selection which controls bloating by balancing accuracy and complexity, and a substructure guidance mechanism which mutates expressions with candidate reusable building blocks. EvoMO-SR achieves the best accuracy in seven of the eight in-domain and out-of-domain settings for LSR-Synth, using a small LLM model, i.e., Llama-3.1-8B-Instruct. We also evaluated structural recovery through two symbolic accuracy metrics based on canonicalized subtree overlap and term matching, showing that our method has a greater probability of recovering highly accurate symbolic structures.

Figures & tables

Figure 1: EvoMO-SR main components. 1) Prompt Construction : the first step includes the construction of the prompt by role specification, task description, additional context based on considered dataset, and output and example format. 2) Initialization : the initial population is created by prompting the LLM with the constructed prompt, and each candidate is fitted and evaluated. 3) Evolutionary Loop : in subsequent generations, the LLM is prompted with the initial prompt enriched with the description, fitness and equation of the selected candidate, plus a mutation prompt randomly selected from the pre-fixed set. To form the new population, the fitted and evaluated set of candidates faces survival selection, which is on multi-objective optimization. At the end of each generation, each individual is scanned to extract added substructures and the substructure archive is updated accordingly.

Panel of Figure 2 — Figure 2: Example of the feedback generated by the LLM.

Method	Physics		Chemistry		Biology		Material Science
Method	ID $\downarrow$	OOD $\downarrow$	ID $\downarrow$	OOD $\downarrow$	ID $\downarrow$	OOD $\downarrow$	ID $\downarrow$	OOD $\downarrow$
uDSR	1.15e-1	9.30e-2	1.07e-1	86.11	1.99e-1	1.29	6.23e-2	3.28
pySR	3.23e-1	2.30e-1	2.63e-1	180.94	8.47e-1	366.68	2.50e-1	25.93
GPGomea	4.11e-2	4.05e-2	1.97e-3	2.59	2.29e-1	23.52	1.22e-3	1.22e-1
Operon	1.58e-1	1.51e-1	1.04e-2	7.54	3.42e-1	49.43	1.82e-2	1.11
LLM-SR	4.10e-5	7.65e-4	1.14e-5	3.34e-1	7.23e-5	5.29e-1	2.20e-5	6.67e-3

Table 1: Comparison of Median NMSE ID and OOD values between EvoMO-SR against LLM-SR and other SR baselines over LSR-Synth datasets (44 for Physics, 36 for Chemistry, 24 for Biology, 25 for Material Science) averaged across 5 runs. EvoMO-SR, LLM-SR and LaSR have been run with Llama-3.1-8B-Instruct backbone. Lower NMSE ID/OOD (

\downarrow

) values are better.

Figure 3: Comparison of $SA_{AST}$ and $SA_{Term}$ between EvoMO-SR and the other SR baselines over LSR-Synth Physics dataset.

Panel of Figure 4 — Figure 4: Percentage of recovered expressions across all LSR-Synth datasets and runs (for a total of 645 cases) as a function of the threshold $\tau$ for $SA_{AST}$ and $SA_{Term}$ .

Method	oscillator1				oscillator2
Method	ID $\downarrow$	OOD $\downarrow$	$SA_{AST}$ $\uparrow$	$SA_{Term}$ $\uparrow$	ID $\downarrow$	OOD $\downarrow$	$SA_{AST}$ $\uparrow$	$SA_{Term}$ $\uparrow$
LLM-SR	1.55e-5	2.61e-2	0.19	0.28	2.85e-5	8.25e-3	0.23	0
LaSR	4.37e-3	5.02e-1	0.27	0	2.00e-1	1.15e-1	0.12	0
EvoMO-SR	1.53e-6	1.80e-2	0.17	0.35	1.69e-6	2.43e-4	0.16	0.33

Table 2: Direct comparison of median NMSE ID, NMSE OOD and SA values over 20 runs with LLM-SR and LaSR over oscillator1 and oscillator2 datasets.

Problem	Method	ID $\downarrow$	OOD $\downarrow$	$SA_{AST}$ $\uparrow$	$SA_{Term}$ $\uparrow$
P1	EvoMO-SR	2.56e-32	1.40e-32	0.26	1.00
P1	LLM-SR	2.49e-10	4.69e-8	0.19	0.14
P2	EvoMO-SR	7.03e-15	1.94e-15	0.22	0.75
P2	LLM-SR	5.89e-12	4.06e-11	0.22	0.37
P3	EvoMO-SR	2.20e-7	4.02e-5	0.11	0.18
P3	LLM-SR	2.06e-5	1.56e-3	0.20	0.18

Table 3: Median values of NMSE ID/OOD and SA metrics obtained by EvoMO-SR and LLM-SR across datasets generated with the following ground truth expressions:

1.88\sin{x_{1}}+1.61\cos(x_{0})

(P1),

1.10x_{0}^{3}+2.02x_{0}^{2}-1.69x_{1}

(P2),

-1.95x_{0}^{2}-0.35x_{1}\sin{x_{1}}+2.36x_{2}^{2}+2.88

(P3),

1.90x_{1}^{2}+2.38x_{1}x_{2}-2.16\sin{x_{0}}-1.08\cos{x_{0}}-2.23\cos{x_{2}}-2.10

(P4).

Model	oscillator1				oscillator2
	w/o sub		with sub		w/o sub		with sub
	ID $\downarrow$	OOD $\downarrow$	ID $\downarrow$	OOD $\downarrow$	ID $\downarrow$	OOD $\downarrow$	ID $\downarrow$	OOD $\downarrow$
Llama8B	1.39e-5	2.06e-2	4.79e-6	2.11e-2	1.98e-5	3.83e-3	8.45e-6	1.60e-3
DeepSeek-v4-Flash	2.26e-6	2.98e-2	9.01e-7	1.32e-2	4.43e-4	1.88e-1	1.62e-4	2.76e-3
GPT-5.4-nano	5.34e-6	3.15e-2	9.13e-7	1.09e-2	1.28e-3	1.94e-2	8.37e-7	2.06e-4

Table 4: Performance comparison across LLM backbones (Llama-3.1-8B-Instruct, DeepSeek-v4-Flash, GPT-5.4-nano) with substructure guidance ( with sub ) and without substructure guidance ( w/o sub ) over oscillator1 and oscillator2 datasets.

Appendix figures & tables21 assets

Supplementary material from the paper’s appendix.

Appendix

Panel of Figure 5 — Figure 5: The task prompt we use, which contains the specification of the role and the description of the equation discovery task.

Panel of Figure 6 — Figure 6: One example of scientific prompt. Here the context extracted from metadata of oscillator1 dataset.

Panel of Figure 7 — Figure 7: The format prompt.

Panel of Figure 8 — Figure 8: An example of the LLM response to the refinement option for one iteration of a run over the oscillator1 dataset.

Panel of Figure 9 — Figure 9: An example of the LLM response to the mutation prompt and substructure guidance for one iteration of a run over the oscillator1 dataset.

Figure 10: Comparison of $SA_{AST}$ and $SA_{Term}$ between EvoMO-SR and the other SR baselines over LSR-Synth Biology dataset.

Figure 11: Comparison of $SA_{AST}$ and $SA_{Term}$ between EvoMO-SR and the other SR baselines over LSR-Synth Chemistry dataset.

Figure 12: Comparison of $SA_{AST}$ and $SA_{Term}$ between EvoMO-SR and the other SR baselines over LSR-Synth Material Science dataset.

Method		Oscillator 1
Sub	MO	ID $\downarrow$	OOD $\downarrow$	$SA_{AST}$ $\uparrow$	$SA_{Term}$ $\uparrow$	Complexity $\downarrow$
$\times$	$\times$	4.29e-6	2.30e-2	0.14	0.27	121.6
$\times$	$\checkmark$	1.39e-5	2.06e-2	0.31	0.37	60.2
$\checkmark$	$\times$	3.81e-6	1.70e-2	0.15	0.32	112.4
$\checkmark$	$\checkmark$	4.79e-6	2.11e-2	0.28	0.41	86.8

Appendix

Table 5: Comprehensive ablation study removing substructure guidance (Sub) and multi-objective (MO) selection mechanism over oscillator1 dataset.

Method		Oscillator 2
Sub	MO	ID $\downarrow$	OOD $\downarrow$	$SA_{AST}$ $\uparrow$	$SA_{Term}$ $\uparrow$	Complexity $\downarrow$
$\times$	$\times$	1.38e-6	1.34e-2	0.14	0.27	171.0
$\times$	$\checkmark$	1.98e-5	3.83e-3	0.29	0.32	94.0
$\checkmark$	$\times$	5.03e-6	2.69e-3	0.11	0.16	214.8
$\checkmark$	$\checkmark$	8.45e-6	1.60e-3	0.29	0.35	105.6

Appendix

Table 6: Comprehensive ablation study removing substructure guidance (Sub) and multi-objective (MO) selection mechanism over oscillator2 dataset.

Figure 13: Comparison of NMSE ID and OOD values between EvoMO-SR with LLM-driven mutation (original), with GP mutation, and hybrid mutations (both LLM-driven and GP-based).

Domain	Split	p-value	EvoMO-SR (ours) wins	LLM-SR wins
Physics	ID	0.885	24	20
Physics	OOD	0.007	30	14
Chemistry	ID	0.001	25	11
Chemistry	OOD	3.33e-5	31	5
Biology	ID	3.00e-5	22	2
Biology	OOD	2.00e-5	22	2

Appendix

Table 7: Domain-specific pairwise Wilcoxon test with Holm’s correction between EvoMO-SR and LLM-SR

Domain	Split	p-value	EvoMO-SR (ours) wins	LLM-SR wins
oscillator1	ID	0.09	14	6
oscillator1	OOD	0.24	12	8
oscillator2	ID	0.09	13	7
oscillator2	OOD	0.09	13	7

Appendix

Table 8: Pairwise Wilcoxon test with Holm’s correction between EvoMO-SR and LLM-SR over oscillator1 and oscillator2 datasets

Domain	Metric	p-value	EvoMO-SR (ours) wins	LLM-SR wins	Ties
oscillator1	$SA_{AST}$	0.01	5	15	0
oscillator1	$SA_{Term}$	0.13	13	6	1
oscillator2	$SA_{AST}$	0.001	3	17	0
oscillator2	$SA_{Term}$	0.001	17	1	2

Appendix

Table 9: Pairwise Wilcoxon test with Holm’s correction over SA metrics between EvoMO-SR and LLM-SR over oscillator1 and oscillator2 datasets.

Figure 14: Obtained approximated Pareto fronts of distinct runs over 4 problems: PO18 for the Physics domain, CRK31 for the Chemistry domain, BPG0 for the Biology domain, and MatSci13 for the Material Science domain.

Figure 15: Average per-run Median NMSE ID and OOD values of the best expressions selected based on the lowest training NMSE, the lowest complexity and the knee point of the final pareto front.

Figure 16: Average per-run Mean complexity values of the best expressions selected based on the lowest training NMSE, the lowest complexity and the knee point of the final pareto front.

Figure 17: Average per-run Mean similarity values of the best expressions selected based on the lowest training NMSE, the lowest complexity and the knee point of the final pareto front. On the left results concerning $SA_{AST}$ values, while on the right results concerning $SA_{Term}$ values.

Component	LLM-SR	Our method
LLM-based structural generation	✓	✓
External coefficient optimization	✓	✓
Data-driven iterative feedback	✓	✓
Multiple maintained candidates	✓	✓
Island-based experience management	✓	–
Fitness-biased sampling of previous solutions	✓	✓

Appendix

Table 10: High-level comparison between LLM-SR and our framework. Both approaches employ iterative LLM-based equation generation guided by previous evaluations, but they differ in how evolutionary information is represented and used. † LLM-SR performs fitness-biased selection of programs from its experience buffer for in-context prompting; here, “explicit parent selection” refers specifically to selecting individuals as parents in a population-based evolutionary algorithm.

Component	LaSR	Our method
Explicit population-based evolutionary search	✓	✓
Data-driven iterative feedback	✓	✓
LLM-guided evolutionary mutation	✓	✓
Standard symbolic GP mutation/crossover	✓	–
LLM-based mutation	✓	✓
LLM-based crossover	✓	–

Appendix

Table 11: High-level comparison between LaSR and our framework. Both approaches combine explicit evolutionary symbolic regression with LLM-based guidance, but differ substantially in how knowledge acquired during evolution is represented and reused. LaSR abstracts high- and low-performing expressions into natural-language concepts, whereas our method explicitly extracts and archives symbolic substructures introduced through parent–offspring transformations.

	Runtime (s)	# API requests	# Input tokens	# Output tokens
LLM-SR	$3^{\prime}279.90\pm 82.55$	$250.00\pm 0.00$	$767^{\prime}525.33\pm 31^{\prime}712.75$	$506^{\prime}200.33\pm 1^{\prime}565.21$
LaSR	$\mathbf{490.79\pm 9.25}$	$230.67\pm 15.82$	$59^{\prime}708.67\pm 3^{\prime}734.57$	$20^{\prime}654.00\pm 432.85$
EvoMO-SR	$594.77\pm 63.53$	$1^{\prime}000.00\pm 0.00$	$783^{\prime}589.67\pm 25^{\prime}376.85$	$150^{\prime}259.00\pm 7^{\prime}743.26$

Appendix

Table 12: Mean per-run computation time (in seconds), number of API requests, number of input tokens and number of output tokens over the oscillator1 dataset. Values are reported as mean

\pm

standard deviation over three runs.

Explore similar work

Deliberate Evolution: Agentic Reasoning for Sample-Efficient Symbolic Regression with LLMs
Jun 3, 2026Xinyu Pang, Zhanke Zhou, Xuan Li +5Large Language Model-Guided Optimization Large Language Model-Guided Scientific Discovery
Beijing National Research Center for Information Science and Technology (BNRist), Department of Automation, Tsinghua University, Beijing, P.R. China · TMLR Group, Department of Computer Science, Hong Kong Baptist University · Lenovo Research
FunctionEvolve: Structure-Guided Symbolic Regression with LLMs
Jun 5, 2026Zeyu Xia, Jun Zhu, Dong YanGenetic Programming Large Language Model-Guided Scientific Discovery
Bosch Center for Artificial Intelligence · Department of Computer Science and Technology, Tsinghua University · Department of Computer Science and Technology +1
A-SR: Self-Evolving Agentic LLMs for Symbolic Regression via Hierarchical Coordination
Aug 5, 2026Wenxiao Zhao, Dong Liu, Kaiyi Xu +9Multi-Agent LLM Systems LLM Agent Training
Shanghai AI Laboratory · University of California at Los Angeles · Tongji University

Jun 3, 2026J/K move · Enter open · S save

Deliberate Evolution: Agentic Reasoning for Sample-Efficient Symbolic Regression with LLMs

Xinyu Pang, Zhanke Zhou, Xuan Li, Fangrui Lv+4

Beijing National Research Center for Information Science and Technology (BNRist), Department of Automation, Tsinghua University, Beijing, P.R. China · TMLR Group, Department of Computer Science, Hong Kong Baptist University · Lenovo Research

Large Language Model-Guided Optimization Large Language Model-Guided Scientific Discovery Symbolic Regression

Open paper AskPDF arXiv

Symbolic regression (SR) discovers compact mathematical expressions from data, yet recent LLM-based evolutionary methods remain sample-inefficient because they rely mainly on scalar feedback such as MSE. We identify a core limitation: existing methods conflate candidate proposal with search guidance, requiring the LLM to infer how to evolve an expression, diagnose its errors, and reuse past experience from a single score. To address this, we propose Deliberate Evolution (DE), an agentic framework that decouples symbolic generation from search control. DE guides LLM proposals with adaptive operators for search direction, analytical tools for structural diagnosis, and reflective memory for trajectory-level experience. Experiments on LLM-SRBench show that DE consistently outperforms representative LLM-based SR baselines across diverse scientific domains while using only 40% of the standard sample budget.

Abstract

Explore similar work

Deliberate Evolution: Agentic Reasoning for Sample-Efficient Symbolic Regression with LLMs

FunctionEvolve: Structure-Guided Symbolic Regression with LLMs

A-SR: Self-Evolving Agentic LLMs for Symbolic Regression via Hierarchical Coordination