Symbolic regression (SR) seeks to discover mathematical expressions directly from data, without requiring task-specific scientific descriptions. Despite tremendous progress in both conventional search-based methods and recent LLM-based methods, two critical limitations largely remain. First (Limited Generation Flexibility), conventional search-based methods rely on either fixed search rules or specialized learned generators, severely limiting adaptive candidate generation across diverse expression structures. Second (Underutilized Search Discoveries), most LLM-based methods cannot reliably recover ground-truth expressions because they fall short in effectively leveraging the LLM itself to thoroughly digest past expressions or adaptively guide future search. To address these limitations, we propose DIGEST. To render greater flexibility in generation, DIGEST repurposes a pretrained decoder block from a lightweight LLM into a symbolic policy for Genetic Programming (GP) population seeding, thereby leveraging the broader modeling capacity of the LLM. To more thoroughly utilize search discoveries, high-quality expressions discovered by GP are fed back to update the LLM-derived policy online, thereby enabling the LLM to progressively internalize past search discoveries and translate them into improved subsequent population generation. Experiments show that on two ground-truth benchmarks, DIGEST outperforms the strongest competitor by up to 2.59% in recovery. Among LLM-based baselines, it improves recovery by at least 54.91% while using at most 17.4% of the strongest competitor's mean search time. On 122 BlackBox datasets, DIGEST achieves the highest accuracy while maintaining a moderate model complexity.
Figures & tables
Figure 1: DIGEST achieves the highest R2 with moderate model size on BlackBox datasets.
Symbol
Meaning
D,A
Observations and symbolic vocabulary.
ϕ,ϕt,L
Expression sequence, its token at step t , and length.
pt,st
Parent and sibling tokens before generation step t .
πθ,Fθ
Symbolic policy and pretrained decoder block.
Ep,Es
Parent and sibling embedding tables.
Wo,bo
Symbolic output projection and bias.
Table 1: Main notation.
Figure 2: Overview of DIGEST. Expression generation: an LLM-derived symbolic policy samples a population from expression-tree context. Candidate refinement: GP searches for higher-reward expressions through evolution. Adaptation from search discoveries: policy-generated expressions and GP elites form a feedback batch, from which high-reward expressions provide training signals for online policy adaptation. The adapted policy seeds the next GP restart.
Conventional methods
LLM-based baselines
Benchmark
DSR
GEGL
NGGP
PySR
iMCTS
ICSR
LLM-SR
LaSR
RESTART
DIGEST
Nguyen
83.58
86.00
92.33
74.41
93.25
14.00
3.33
29.67
4.00
93.33
Livermore
30.41
56.36
71.09
46.14
71.41
16.73
1.27
19.09
2.18
74.00
Table 3: Average recovery rates (%) on Nguyen and Livermore. The best result in each row is bold.
Figure 4: Further comparison of DIGEST with LLM-based baselines over 25 independent runs per equation on Nguyen and Livermore. Left: median test NMSE on a logarithmic scale. Right: mean search time in seconds.
Figure 5: Performance on 122 SRBench BlackBox datasets. Left: cross-dataset median test R2 . Right: cross-dataset median model size. Asterisks identify symbolic regression methods. DIGEST achieves the highest median predictive accuracy while maintaining moderate model complexity.
Livermore
Nguyen
Configuration
Trainable params
Recovery ↑
Time (s) ↓
Recovery ↑
Time (s) ↓
Non-LLM population seeding
Random-seed GP
0
66.91
120.0
91.00
46.8
Single-block policy
Fully frozen
0
72.91
130.8
90.67
55.2
Interface-only updates
0.027M
72.18
130.2
90.67
54.6
Table 4: Ablations of online adaptation, decoder depth, and population seeding. Recovery (%) and mean search time (s) are reported. Bold indicates the highest recovery within each neural policy-depth group.
Appendix figures & tables9 assets
Supplementary material from the paper’s appendix.
Appendix
Symbol
Text label
Symbol
Text label
add
add
sub
subtract
mul
multiply
div
divide
sin
sin
cos
cos
tan
tan
exp
exp
log
log
sqrt
sqrt
n2
square
inv
inverse
Appendix
Table 5: Canonical labels used to initialize the symbolic interface.
Problem
Target expression
Sampling domain
Nguyen-1
x3+x2+x
U(−1,1,20)
Nguyen-2
x4+x3+x2+x
U(−1,1,20)
Nguyen-3
x5+x4+x3+x2+x
U(−1,1,20)
Nguyen-4
x6+x5+x4+x3+x2+x
U(−1,1,20)
Nguyen-5
sin(x2)cos(x)−1
U(−1,1,20)
Nguyen-6
sin(x)+sin(x+x2)
U(−1,1,20)
Appendix
Table 6: Target expressions and sampling domains for the Nguyen benchmark used in our experiments.
Problem
Target expression
Sampling domain
Livermore-1
31+x+sin(x2)
U(−10,10,1000)
Livermore-2
sin(x2)cos(x)−2
U(−1,1,20)
Livermore-3
sin(x3)cos(x2)−1
U(−1,1,20)
Livermore-4
log(x+1)+log(x2+1)+log(x)
U(0,2,20)
Livermore-5
x4−x3+x2−y
U(0,1,20)
Livermore-6
4x4+3x3+2x2+x
U(−1,1,20)
Appendix
Table 7: Target expressions and sampling domains for the Livermore benchmark used in our experiments.
Equation
DSR
GEGL
NGGP
PySR
iMCTS
DIGEST
Nguyen-1
100
100
100
100
100
100.00
Nguyen-2
100
100
100
98
100
100.00
Nguyen-3
100
100
100
70
100
100.00
Nguyen-4
100
100
100
2
97
100.00
Nguyen-5
72
92
100
89
100
100.00
Nguyen-6
100
100
100
100
100
100.00
Appendix
Table 8: Per-equation recovery (%) on Nguyen: conventional baselines and DIGEST. The best value per row is bold.
Equation
ICSR
LLM-SR
LaSR
RESTART
Random GP
DIGEST
Full LoRA
Full dense
Nguyen-1
20.00
16.00
28.00
24.00
100.00
100.00
100.00
100.00
Nguyen-2
4.00
0.00
4.00
8.00
100.00
100.00
100.00
100.00
Nguyen-3
0.00
12.00
0.00
0.00
100.00
100.00
100.00
100.00
Nguyen-4
0.00
0.00
0.00
4.00
100.00
100.00
100.00
100.00
Nguyen-5
0.00
0.00
0.00
0.00
100.00
100.00
100.00
100.00
Nguyen-6
0.00
0.00
32.00
0.00
100.00
100.00
100.00
100.00
Appendix
Table 9: Per-equation recovery (%) on Nguyen: LLM-based baselines, Random-seed GP, DIGEST, and full-depth ablations. The best value per row is bold.
Equation
DSR
GEGL
NGGP
PySR
iMCTS
DIGEST
Livermore-1
3
100
100
100
100
100.00
Livermore-2
87
44
100
32
95
100.00
Livermore-3
66
100
100
97
100
100.00
Livermore-4
76
100
100
90
100
100.00
Livermore-5
0
0
4
30
78
16.00
Livermore-6
97
64
88
0
19
48.00
Appendix
Table 10: Per-equation recovery (%) on Livermore: conventional baselines and DIGEST. The best value per row is bold.
Equation
ICSR
LLM-SR
LaSR
RESTART
Random GP
DIGEST
Full LoRA
Full dense
Livermore-1
0.00
0.00
96.00
0.00
100.00
100.00
100.00
100.00
Livermore-2
0.00
0.00
0.00
0.00
100.00
100.00
100.00
100.00
Livermore-3
0.00
0.00
0.00
0.00
92.00
100.00
100.00
100.00
Livermore-4
0.00
0.00
0.00
0.00
100.00
100.00
100.00
100.00
Livermore-5
0.00
0.00
0.00
0.00
4.00
16.00
52.00
36.00
Livermore-6
0.00
28.00
0.00
36.00
4.00
48.00
76.00
68.00
Appendix
Table 11: Per-equation recovery (%) on Livermore: LLM-based baselines, Random-seed GP, DIGEST, and full-depth ablations. The best value per row is bold.
Figure 6: Pareto view of cross-dataset median test R2 and median model size on 122 SRBench BlackBox datasets. DIGEST lies on the Pareto frontier with the highest median test R2 and moderate model size. The model-size axis is logarithmic; asterisks identify symbolic regression methods.
Method
Median test R2↑
Mean model size ↓
TPSR
0.945
95.71
DIGEST
0.956
53.44
Appendix
Table 12: Comparison under the TPSR BlackBox evaluation setting.
Symbolic regression (SR) seeks to discover parsimonious mathematical laws from observational data, yet conventional approaches often struggle with the vast combinatorial search space of physically meaningful expressions. We present InsightSR, a framework that embeds Large Language Models (LLMs) as a guiding layer around the PySR genetic programming engine. Rather than relying on LLMs to generate expressions directly, InsightSR uses LLMs to progressively transform the search space itself through two complementary pathways: a Semantic Seed Pathway that proposes dimensionally consistent functional skeletons, and a Structural Feature Pathway that recommends nonlinear feature transformations. These transformations accumulate over iterations, broadening the input space and shifting the symbolic search from constructing deep expression trees over raw variables to assembling shallow trees over a rich, semantically informed feature set. A post-generation feedback loop evaluates candidates, categorizes features by their empirical utility, and refines the guidance for the next iteration, transforming the discovery process from open-ended generation into iterative, self-correcting refinement. Across three benchmarks, InsightSR achieves a 95% exact recovery rate on the Feynman benchmark and 80.18% accuracy on the LLM-SRBench LSR-Transform task, substantially outperforming state-of-the-art genetic programming and neural-symbolic methods while maintaining strong out-of-distribution generalization on real-world datasets.
Symbolic regression searches for an equation that explains observed data. Numerical error is commonly used to guide the search, but it alone can favor an accurate surrogate over a candidate with the right structure. We propose L2G-SR, a framework for LLM-based symbolic regression that guides search with a learned structural verifier. We construct structural supervision signal using tree edit distance (TED), accounting for commutative operators and alternative expression forms. A verifier first aligns numerical observations with the formulas, then learns to score a candidate's structural similarity to the target formula. At search time, the target formula is hidden and the verifier complements numerical error in parent selection. An analyst then uses verifier feedback and diagnostic tools to propose revisions, which a generator converts into refined expressions. On LLM-SRBench, L2G-SR achieves state-of-the-art numerical fit, equation level accuracy and TED across the evaluated LLM backbones and domains. L2G-SR demonstrates the value of learned structural guidance for LLM-based symbolic regression.
Symbolic regression (SR) discovers compact mathematical expressions from data, yet recent LLM-based evolutionary methods remain sample-inefficient because they rely mainly on scalar feedback such as MSE. We identify a core limitation: existing methods conflate candidate proposal with search guidance, requiring the LLM to infer how to evolve an expression, diagnose its errors, and reuse past experience from a single score. To address this, we propose Deliberate Evolution (DE), an agentic framework that decouples symbolic generation from search control. DE guides LLM proposals with adaptive operators for search direction, analytical tools for structural diagnosis, and reflective memory for trajectory-level experience. Experiments on LLM-SRBench show that DE consistently outperforms representative LLM-based SR baselines across diverse scientific domains while using only 40% of the standard sample budget.
Xinyu Pang, Zhanke Zhou, Xuan Li +5
Beijing National Research Center for Information Science and Technology (BNRist), Department of Automation, Tsinghua University, Beijing, P.R. China · TMLR Group, Department of Computer Science, Hong Kong Baptist University · Lenovo Research