DIGEST: Discovery-Guided LLM Tuning for Symbolic Regression

Kaifeng Jin; Ruizhong Qiu; Zihao Li; Xiao Lin; Dongqi Fu; Yinglong Xia; Hong Li; Hanghang Tong

cs.LGOct 24, 2026

DIGEST: Discovery-Guided LLM Tuning for Symbolic Regression

Authors: Kaifeng Jin, Ruizhong Qiu, Zihao Li, Xiao Lin, Dongqi Fu, Yinglong Xia, Hong Li, Hanghang Tong

Organizations: University of Illinois Urbana-Champaign · Meta

Abstract

Symbolic regression (SR) seeks to discover mathematical expressions directly from data, without requiring task-specific scientific descriptions. Despite tremendous progress in both conventional search-based methods and recent LLM-based methods, two critical limitations largely remain. First (Limited Generation Flexibility), conventional search-based methods rely on either fixed search rules or specialized learned generators, severely limiting adaptive candidate generation across diverse expression structures. Second (Underutilized Search Discoveries), most LLM-based methods cannot reliably recover ground-truth expressions because they fall short in effectively leveraging the LLM itself to thoroughly digest past expressions or adaptively guide future search. To address these limitations, we propose DIGEST. To render greater flexibility in generation, DIGEST repurposes a pretrained decoder block from a lightweight LLM into a symbolic policy for Genetic Programming (GP) population seeding, thereby leveraging the broader modeling capacity of the LLM. To more thoroughly utilize search discoveries, high-quality expressions discovered by GP are fed back to update the LLM-derived policy online, thereby enabling the LLM to progressively internalize past search discoveries and translate them into improved subsequent population generation. Experiments show that on two ground-truth benchmarks, DIGEST outperforms the strongest competitor by up to 2.59% in recovery. Among LLM-based baselines, it improves recovery by at least 54.91% while using at most 17.4% of the strongest competitor's mean search time. On 122 BlackBox datasets, DIGEST achieves the highest accuracy while maintaining a moderate model complexity.

Figures & tables

Figure 1: DIGEST achieves the highest

R^{2}

with moderate model size on BlackBox datasets.

Symbol	Meaning
$\mathcal{D},\mathcal{A}$	Observations and symbolic vocabulary.
$\phi,\phi_{t},L$	Expression sequence, its token at step $t$ , and length.
$p_{t},s_{t}$	Parent and sibling tokens before generation step $t$ .
$\pi_{\theta},F_{\theta}$	Symbolic policy and pretrained decoder block.
$\mathbf{E}_{p},\mathbf{E}_{s}$	Parent and sibling embedding tables.
$\mathbf{W}_{o},\mathbf{b}_{o}$	Symbolic output projection and bias.

Table 1: Main notation.

Figure 2: Overview of DIGEST. Expression generation: an LLM-derived symbolic policy samples a population from expression-tree context. Candidate refinement: GP searches for higher-reward expressions through evolution. Adaptation from search discoveries: policy-generated expressions and GP elites form a feedback batch, from which high-reward expressions provide training signals for online policy adaptation. The adapted policy seeds the next GP restart.

	Conventional methods					LLM-based baselines
Benchmark	DSR	GEGL	NGGP	PySR	iMCTS	ICSR	LLM-SR	LaSR	RESTART	DIGEST
Nguyen	83.58	86.00	92.33	74.41	93.25	14.00	3.33	29.67	4.00	93.33
Livermore	30.41	56.36	71.09	46.14	71.41	16.73	1.27	19.09	2.18	74.00

Table 3: Average recovery rates (%) on Nguyen and Livermore. The best result in each row is bold.

Figure 4: Further comparison of DIGEST with LLM-based baselines over 25 independent runs per equation on Nguyen and Livermore. Left: median test NMSE on a logarithmic scale. Right: mean search time in seconds.

Figure 5: Performance on 122 SRBench BlackBox datasets. Left: cross-dataset median test

R^{2}

. Right: cross-dataset median model size. Asterisks identify symbolic regression methods. DIGEST achieves the highest median predictive accuracy while maintaining moderate model complexity.

		Livermore		Nguyen
Configuration	Trainable params	Recovery $\uparrow$	Time (s) $\downarrow$	Recovery $\uparrow$	Time (s) $\downarrow$
Non-LLM population seeding
Random-seed GP	0	66.91	120.0	91.00	46.8
Single-block policy
Fully frozen	0	72.91	130.8	90.67	55.2
Interface-only updates	0.027M	72.18	130.2	90.67	54.6

Table 4: Ablations of online adaptation, decoder depth, and population seeding. Recovery (%) and mean search time (s) are reported. Bold indicates the highest recovery within each neural policy-depth group.

Appendix figures & tables9 assets

Supplementary material from the paper’s appendix.

Appendix

Symbol	Text label	Symbol	Text label
add	add	sub	subtract
mul	multiply	div	divide
sin	sin	cos	cos
tan	tan	exp	exp
log	log	sqrt	sqrt
n2	square	inv	inverse

Appendix

Table 5: Canonical labels used to initialize the symbolic interface.

Problem	Target expression	Sampling domain
Nguyen-1	$x^{3}+x^{2}+x$	$U(-1,1,20)$
Nguyen-2	$x^{4}+x^{3}+x^{2}+x$	$U(-1,1,20)$
Nguyen-3	$x^{5}+x^{4}+x^{3}+x^{2}+x$	$U(-1,1,20)$
Nguyen-4	$x^{6}+x^{5}+x^{4}+x^{3}+x^{2}+x$	$U(-1,1,20)$
Nguyen-5	$\sin(x^{2})\cos(x)-1$	$U(-1,1,20)$
Nguyen-6	$\sin(x)+\sin(x+x^{2})$	$U(-1,1,20)$

Appendix

Table 6: Target expressions and sampling domains for the Nguyen benchmark used in our experiments.

Problem	Target expression	Sampling domain
Livermore-1	$\frac{1}{3}+x+\sin(x^{2})$	$U(-10,10,1000)$
Livermore-2	$\sin(x^{2})\cos(x)-2$	$U(-1,1,20)$
Livermore-3	$\sin(x^{3})\cos(x^{2})-1$	$U(-1,1,20)$
Livermore-4	$\log(x+1)+\log(x^{2}+1)+\log(x)$	$U(0,2,20)$
Livermore-5	$x^{4}-x^{3}+x^{2}-y$	$U(0,1,20)$
Livermore-6	$4x^{4}+3x^{3}+2x^{2}+x$	$U(-1,1,20)$

Appendix

Table 7: Target expressions and sampling domains for the Livermore benchmark used in our experiments.

Equation	DSR	GEGL	NGGP	PySR	iMCTS	DIGEST
Nguyen-1	100	100	100	100	100	100.00
Nguyen-2	100	100	100	98	100	100.00
Nguyen-3	100	100	100	70	100	100.00
Nguyen-4	100	100	100	2	97	100.00
Nguyen-5	72	92	100	89	100	100.00
Nguyen-6	100	100	100	100	100	100.00

Appendix

Table 8: Per-equation recovery (%) on Nguyen: conventional baselines and DIGEST. The best value per row is bold.

Equation	ICSR	LLM-SR	LaSR	RESTART	Random GP	DIGEST	Full LoRA	Full dense
Nguyen-1	20.00	16.00	28.00	24.00	100.00	100.00	100.00	100.00
Nguyen-2	4.00	0.00	4.00	8.00	100.00	100.00	100.00	100.00
Nguyen-3	0.00	12.00	0.00	0.00	100.00	100.00	100.00	100.00
Nguyen-4	0.00	0.00	0.00	4.00	100.00	100.00	100.00	100.00
Nguyen-5	0.00	0.00	0.00	0.00	100.00	100.00	100.00	100.00
Nguyen-6	0.00	0.00	32.00	0.00	100.00	100.00	100.00	100.00

Appendix

Table 9: Per-equation recovery (%) on Nguyen: LLM-based baselines, Random-seed GP, DIGEST, and full-depth ablations. The best value per row is bold.

Equation	DSR	GEGL	NGGP	PySR	iMCTS	DIGEST
Livermore-1	3	100	100	100	100	100.00
Livermore-2	87	44	100	32	95	100.00
Livermore-3	66	100	100	97	100	100.00
Livermore-4	76	100	100	90	100	100.00
Livermore-5	0	0	4	30	78	16.00
Livermore-6	97	64	88	0	19	48.00

Appendix

Table 10: Per-equation recovery (%) on Livermore: conventional baselines and DIGEST. The best value per row is bold.

Equation	ICSR	LLM-SR	LaSR	RESTART	Random GP	DIGEST	Full LoRA	Full dense
Livermore-1	0.00	0.00	96.00	0.00	100.00	100.00	100.00	100.00
Livermore-2	0.00	0.00	0.00	0.00	100.00	100.00	100.00	100.00
Livermore-3	0.00	0.00	0.00	0.00	92.00	100.00	100.00	100.00
Livermore-4	0.00	0.00	0.00	0.00	100.00	100.00	100.00	100.00
Livermore-5	0.00	0.00	0.00	0.00	4.00	16.00	52.00	36.00
Livermore-6	0.00	28.00	0.00	36.00	4.00	48.00	76.00	68.00

Appendix

Table 11: Per-equation recovery (%) on Livermore: LLM-based baselines, Random-seed GP, DIGEST, and full-depth ablations. The best value per row is bold.

Figure 6: Pareto view of cross-dataset median test

R^{2}

and median model size on 122 SRBench BlackBox datasets. DIGEST lies on the Pareto frontier with the highest median test

R^{2}

and moderate model size. The model-size axis is logarithmic; asterisks identify symbolic regression methods.

Method	Median test $R^{2}$ $\uparrow$	Mean model size $\downarrow$
TPSR	0.945	95.71
DIGEST	0.956	53.44

Appendix

Table 12: Comparison under the TPSR BlackBox evaluation setting.

Symbolic Regression

Symbolic regression searches data for compact mathematical expressions that balance predictive accuracy with interpretability.

See all 148 papers

InsightSR: Refining Symbolic Regression Search Spaces via Parallel Semantic and Structural LLM Guidance
Aug 26, 2026Yating Ling, Wenjing Cun, Zhitang ChenSymbolic Regression Large Language Model-Guided Scientific Discovery
L2G-SR: Learning to Guide LLM-based Symbolic Regression
Oct 14, 2026Letian Huang, Xiangyu Wang, Haowei Lin +1Symbolic Regression Large Language Model-Guided Optimization
Peking University
Deliberate Evolution: Agentic Reasoning for Sample-Efficient Symbolic Regression with LLMs
Jun 3, 2026Xinyu Pang, Zhanke Zhou, Xuan Li +5Large Language Model-Guided Optimization Large Language Model-Guided Scientific Discovery
Beijing National Research Center for Information Science and Technology (BNRist), Department of Automation, Tsinghua University, Beijing, P.R. China · TMLR Group, Department of Computer Science, Hong Kong Baptist University · Lenovo Research

Aug 26, 2026J/K move · Enter open · S save

InsightSR: Refining Symbolic Regression Search Spaces via Parallel Semantic and Structural LLM Guidance

Yating Ling, Wenjing Cun, Zhitang Chen

Symbolic Regression Large Language Model-Guided Scientific Discovery Genetic Programming

Open paper AskPDF arXiv

Symbolic regression (SR) seeks to discover parsimonious mathematical laws from observational data, yet conventional approaches often struggle with the vast combinatorial search space of physically meaningful expressions. We present InsightSR, a framework that embeds Large Language Models (LLMs) as a guiding layer around the PySR genetic programming engine. Rather than relying on LLMs to generate expressions directly, InsightSR uses LLMs to progressively transform the search space itself through two complementary pathways: a Semantic Seed Pathway that proposes dimensionally consistent functional skeletons, and a Structural Feature Pathway that recommends nonlinear feature transformations. These transformations accumulate over iterations, broadening the input space and shifting the symbolic search from constructing deep expression trees over raw variables to assembling shallow trees over a rich, semantically informed feature set. A post-generation feedback loop evaluates candidates, categorizes features by their empirical utility, and refines the guidance for the next iteration, transforming the discovery process from open-ended generation into iterative, self-correcting refinement. Across three benchmarks, InsightSR achieves a 95% exact recovery rate on the Feynman benchmark and 80.18% accuracy on the LLM-SRBench LSR-Transform task, substantially outperforming state-of-the-art genetic programming and neural-symbolic methods while maintaining strong out-of-distribution generalization on real-world datasets.

Abstract

Symbolic Regression

Explore similar work

InsightSR: Refining Symbolic Regression Search Spaces via Parallel Semantic and Structural LLM Guidance

L2G-SR: Learning to Guide LLM-based Symbolic Regression

Deliberate Evolution: Agentic Reasoning for Sample-Efficient Symbolic Regression with LLMs