Organizations: The Hong Kong Polytechnic University, Hong Kong, China · Eastern Institute of Technology, Ningbo, China · Shanghai Jiao Tong University, Shanghai, China
Symbolic regression (SR) aims to discover interpretable mathematical expressions from data and plays a key role in scientific discovery. However, existing methods face a common bottleneck: the enormous search space and severe combinatorial explosion make it difficult to achieve a favorable trade-off between accuracy and algorithmic efficiency. We argue that, in addition to designing increasingly complex solver-specific SR strategies, an equally promising direction is to equip them with a shareable auxiliary strategy that can provide reliable search guidance. To this end, we propose DASS, a solver-adaptable Dynamic Auxiliary Search Strategy that improves SR by constructing and refining a high-potential auxiliary search subspace. DASS represents the auxiliary search space as a functional subspace spanned by a set of high-potential basis function terms. It estimates term utility through multi-environment evaluation, filters unstable guidance, and updates term potential scores with a Gibbs posterior, thereby calibrating the subspace to provide more reliable guidance for downstream solvers. The high-quality solutions produced by the solver are used to update the set of basis terms. This forms a virtuous closed-loop optimization process: the subspace guides the downstream solver, and the solver feedback reshapes the subspace for subsequent exploration. DASS can be easily integrated with various types of SR solvers. Extensive experiments on LLM-SRBench show that DASS significantly improves the accuracy of original solvers with a manageable increase in runtime.
Figures & tables
Figure 1: Core idea of DASS. Instead of letting an SR solver search the full expression space without guidance, DASS provides a solver-adaptable auxiliary search subspace spanned by high-potential basis terms. The subspace is initialized from LLM prior knowledge, calibrated by multi-environment evaluation, and reshaped using feedback from high-quality solver results. In this way, DASS guides diverse SR solvers toward promising regions while continuously refining the guided subspace.
Figure 2: Framework of DASS. DASS initializes an auxiliary search subspace from prior knowledge, calibrates it through multi-environment evaluation, stability filtering, and Gibbs posterior updating, and uses the resulting high-potential terms to guide downstream SR solvers. High-quality solver outputs are then fed back to reshape the subspace, yielding a closed-loop optimization process. In addition to the three SR solvers shown in the figure, we include two additional solver types in the Appendix D : latent-space-based and policy-based solvers.
Category
Method
Phys
MatSci
Chem
Bio
Time ↓
NMSE ↓
Acc0.01↑
Time ↓
NMSE ↓
Acc0.01↑
Time ↓
NMSE ↓
Acc0.01↑
Time ↓
NMSE ↓
Acc0.01↑
LLM-based
Qwen3.5-9B
LLMSR
197.2
5.32E-03
9.1
185
2.38E-05
38.7
177.3
4.96E-04
7.4
187.9
2.14E-03
6.9
LLMSR+DASS
205.1
6.04E-04
29.5
257.2
2.91E-06
81.3
260.4
1.15E-04
23.1
284.2
4.52E-04
13.9
LASR
413.5
2.24E-02
6.8
502.7
6.76E-04
16.0
477.1
5.24E-03
13.9
429
5.64E-02
6.9
LASR+DASS
523.8
4.18E-03
18.9
566.9
8.15E-05
41.3
496.2
6.24E-04
18.5
491.3
7.64E-03
12.5
Table 1: Main results on the four domain-specific datasets in LLM-SRBench. We evaluate DASS by integrating it into five representative SR solvers and report the average search time (Time), normalized mean squared error (NMSE), and Acc0.01 . Shaded rows indicate DASS-augmented variants, and boldface denotes better value within each base solver / +DASS pair.
Figure 3: Low-budget DASS versus high-budget base solvers on LLMSR, PySR, and SPL. Each solver is evaluated under low and high search budgets with all other hyperparameters fixed; the high-budget setting increases LLMSR max LLM calls from 100 to 1000 , PySR outer iterations from 65 to 130 , and SPL MCTS transplants from 10 to 20 . Markers indicate low-budget base solver (light-blue circle), low-budget DASS-augmented solver (green star), and high-budget base solver (large brown circle). Results are averaged over four domains. The x -axis gives relative search time normalized by the low-budget base solver ( 1× ), and the y -axis gives Acc0.01 .
Setting
Variant
LLMSR
PySR
SPL
Phys
MatSci
Chem
Bio
Phys
MatSci
Chem
Bio
Phys
MatSci
Chem
Bio
Single-env. utility
Extrap
26.5
74.7
20.4
9.7
21.2
32.0
21.3
30.6
18.2
25.3
21.3
12.5
Sparse
22.0
66.7
15.7
8.3
20.5
29.3
19.4
26.4
13.6
22.7
19.4
11.1
Noisy
24.2
72.0
21.3
8.3
21.2
30.7
18.5
27.8
16.7
22.7
22.2
11.1
Multi-env. utility (DASS)
29.5
81.3
23.1
13.9
22.7
36.0
25.0
33.3
22.0
26.7
24.1
12.5
Table 2: Ablation study on multi-environment evaluation, measured by Acc0.01 . All other hyperparameters are kept fixed, and we vary only the choice of environments used to compute utility. (1) Single env: evaluates utility using only one environment in E={extrap,sparse,noisy} . (2) All three envs: the DASS setting, which alternates sampling across the three evaluation environments.
Appendix figures & tables24 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 4: An example DASS-augmented LLM-SR prompt from the Physics subset of LLM-SRBench, shown in a compact two-column layout. The Reference Terms (DASS) panel at the top of the second column is the only DASS-specific addition, listing the top- k% terms from B(t) ranked by ω(t) as additional structural guidance for the LLM.
Category
Method
Phys
MatSci
Chem
Bio
NMSE ↓
Acc0.01↑
NMSE ↓
Acc0.01↑
NMSE ↓
Acc0.01↑
NMSE ↓
Acc0.01↑
RL-based
DSR
4.82E-01
0
8.65E-03
0
9.31E-01
0
1.54E+00
0
DSR+DASS
8.41E-02
0
2.97E-03
0
2.04E-01
0
7.18E-01
0
Latent-space-based
GenSR
5.21E-03
4.5
8.32E-05
18.7
2.86E-01
8.3
7.85E-01
4.2
GenSR+DASS
1.34E-03
12.1
2.18E-05
33.3
4.21E-02
16.7
8.62E-02
9.7
Appendix
Table 3: Performance of DSR and GenSR before and after coupling with DASS through the action-prior (DSR) and latent-prior (GenSR) interfaces. DASS-augmented rows shaded gray.
Method
Budget
Acc0.01
Complexity
SA
Qwen3.5-9B
LLM-SR
100 calls
13.95
48.5
2.58
LLM-SR + DASS
100 calls
34.85
50.1
4.65
LLM-SR (higher budget)
1,000 calls
35.60
55.0
4.91
LaSR
20 iterations
10.58
24.2
3.10
LaSR + DASS
20 iterations
21.94
22.7
4.91
Appendix
Table 4: Numerical accuracy, expression complexity, and symbolic accuracy on LSR-Synth for LLM-based solvers. DASS variants are compared with their base solvers at the same downstream search budget. The higher-budget LLM-SR result is provided separately as a reference. Accuracy values are percentages; higher accuracy and lower complexity are preferred.
Method
Budget
Acc0.01
Complexity
SA
PySR
65 iterations
17.83
19.6
3.10
PySR + DASS
65 iterations
27.89
21.2
3.88
PySR (higher budget)
130 iterations
27.40
23.8
4.39
SPL
10 transplants
12.12
16.8
0.52
SPL + DASS
10 transplants
21.73
20.4
0.52
SPL (higher budget)
20 transplants
18.10
18.6
0.78
Appendix
Table 5: Additional evaluation of GP-based and MCTS-based solvers on LSR-Synth. Higher-budget base-solver configurations are reference points rather than matched-budget comparisons. Accuracy values are percentages.
Method
Acc0.01
Acc0.001
Complexity
SA
SR-Scientist
49.61
39.53
28.5
4.65
SR-Scientist + DASS
61.24
43.41
26.2
6.20
Appendix
Table 6: Integration with SR-Scientist using GPT-OSS-120B on LSR-Synth. Both configurations use a maximum budget of 10×25 downstream calls. Accuracy values are percentages.
Backbone
Method
Acc0.01
Complexity
SA
Qwen3.5-9B
SGA
9.30
40.4
1.81
Qwen3.5-9B
SGA + DASS
19.64
41.5
3.36
Gemma-4-31B
SGA
13.70
39.6
2.58
Gemma-4-31B
SGA + DASS
23.26
36.8
4.13
Appendix
Table 7: Integration with SGA on LSR-Synth using 100 downstream calls. Each comparison holds the backbone and downstream call budget fixed. Accuracy values are percentages.
Method
Llama-3.1-8B
GPT-4o
Gemma-4-31B
LLM-SR
13.95
13.95
13.95
LLM-SR + DASS
25.58
34.85
37.21
LLM-SR + DASS (randomized)
22.48
31.01
33.33
PySR
17.83
17.83
17.83
PySR + DASS
23.26
27.89
30.23
PySR + DASS (randomized)
20.16
25.58
28.68
Appendix
Table 8: Sensitivity to the initial basis library on LSR-Synth, measured by Acc0.01 (%). Baselines do not use a basis library; their results are repeated for comparison. “Randomized” changes both term selection and initial potential scores within an LLM-generated candidate pool.
Initial search space for the target task
Acc0.01
No auxiliary space (PySR without DASS)
18.2
Physics domain-level prior (standard DASS)
22.7
Final space from another Physics task
24.2
Final space from a Chemistry task
20.5
Appendix
Table 9: Transfer to Physics tasks with low-budget PySR. All DASS settings continue adapting the space on the target task. The reported Acc0.01 values are percentages.
Method
Acc0.01
Time (s)
LLM-SR
13.95
187.6
LLM-SR + Counting DASS
25.84
241.5
LLM-SR + Original DASS
34.85
245.3
LLM-SR + Marginal DASS
36.18
364.1
Appendix
Table 10: Term-level utility attribution with LLM-SR and a downstream budget of 100 calls. Accuracy is Acc0.01 (%); time is the reported search time in seconds. The marginal variant includes additional term-removal, coefficient-refitting, and fitness-evaluation work.
Category
Method
Phys
MatSci
Chem
Bio
NMSE ↓
Acc0.01↑
NMSE ↓
Acc0.01↑
NMSE ↓
Acc0.01↑
NMSE ↓
Acc0.01↑
LLM-based
Qwen3.5-9B
LLMSR
2.18E-02
8.3
1.10E-05
45.3
7.26E-03
5.6
4.46E-02
5.6
LLMSR+DASS
2.07E-03
28.0
2.32E-06
85.3
1.84E-03
22.2
7.33E-03
12.5
LASR
9.90E-02
5.3
5.36E-04
30.7
9.54E-02
10.2
1.10E-03
4.2
LASR+DASS
1.19E-02
19.7
6.78E-05
50.7
1.09E-02
16.7
8.21E-04
11.1
Appendix
Table 11: OOD performance across 4 LSR-Synth domains. Time is omitted (identical to Table 1 ); DASS-augmented rows are shaded gray.
Figure 5: Budget efficiency on the Physics domain. Each panel corresponds to one base solver (LLMSR, PySR, SPL). The horizontal axis shows relative search time (base-solver low budget set to 1× ); the vertical axis shows Acc0.01 . The three markers within each panel correspond to the base solver at low budget, the base solver augmented with DASS at low budget, and the base solver at high budget.
Figure 6: Budget efficiency on the Material Science domain; the setup is identical to Fig. 5 .
Figure 7: Budget efficiency on the Chemistry domain; the setup is identical to Fig. 5 .
Figure 8: Budget efficiency on the Biology domain; the setup is identical to Fig. 5 .
Update mechanisms enabled
LLMSR
PySR
SPL
Phys
MatSci
Chem
Bio
Phys
MatSci
Chem
Bio
Phys
MatSci
Chem
Bio
Frozen prior ( B,ω fixed after init)
17.4
64.0
13.9
9.7
20.5
29.3
18.5
26.4
14.4
22.7
19.4
11.1
+ Calibration (only ω updated)
25.0
73.3
20.4
12.5
22.7
33.3
23.1
30.6
19.7
24.0
22.2
11.1
+ Reshaping (DASS; B,ω both updated)
29.5
81.3
23.1
13.9
22.7
36.0
25.0
33.3
22.0
26.7
24.1
12.5
Appendix
Table 12: Ablation study on the subspace update mechanism, measured by Acc0.01 . The three settings share the same LLM-generated initial basis term set B(0) and initial scores ω(0) . They differ only in which dynamic update mechanisms are enabled after initialization: (1) Frozen prior , where both B and ω are frozen after initialization; (2) + Calibration , where only subspace calibration is enabled; and (3) + Reshaping , where both subspace calibration and subspace reshaping are enabled, so that B and ω are updated according to search feedback. The last setting corresponds to the full DASS.
Figure 9: Sensitivity to the sample count m . Left: four-domain average Acc0.01 plotted against m , with the three curves corresponding to LLM-SR, PySR, and SPL. Right: the corresponding average per-problem search time for each value of m , measured as a multiple of the search time at m=50 .
Figure 10: Sensitivity to the basis-set size n . 4-domain mean Acc0.01 for LLM-SR, PySR, and SPL with n∈{15,25,35,45,55} .
Figure 11: Sensitivity to the elite ratio k . 4-domain mean Acc0.01 for LLM-SR, PySR, and SPL with k∈{10%,20%,30%,40%,50%} .
Figure 12: Sensitivity to the insertion count T . Left: 4-domain mean Acc0.01 for LLM-SR, PySR, and SPL with T∈{1,3,6,9,12} . Right: the corresponding average per-problem search time for each value of T , measured as a multiple of the search time at T=1 .
Figure 13: Overview of the prompt used to initialize the basis term set B . Top: a unified prompt template with six placeholders, including five domain-related fields and {count} , which specifies the number of terms to generate. Bottom: domain-placeholder values for Physics, Materials Science, Chemistry, and Biology in LLM-SRBench, instantiated from benchmark-provided metadata available as task input, not from target equations or evaluation data.
Figure 14: Prompt used to generate the initial basis terms set B(0) for the Physics subset of LLM-SRBench, with the benchmark-revealed placeholder values substituted in (top), and the sample LLM output it elicits in a single call (bottom). Each output record carries three fields: term | family/mechanism | one-line description . Ten such records are requested via {count} = 10. LLM backbone: GPT-4o.
Figure 15: Prompt used to generate the initial basis terms set B(0) for the Material Science subset of LLM-SRBench, with the benchmark-revealed placeholder values substituted in (top), and the sample LLM output it elicits in a single call (bottom). Each output record carries three fields: term | family/mechanism | one-line description . Ten such records are requested via {count} = 10. LLM backbone: GPT-4o.
Figure 16: Prompt used to generate the initial basis terms set B(0) for the Chemistry subset of LLM-SRBench, with the benchmark-revealed placeholder values substituted in (top), and the sample LLM output it elicits in a single call (bottom). Each output record carries three fields: term | family/mechanism | one-line description . Ten such records are requested via {count} = 10. LLM backbone: GPT-4o.
Figure 17: Prompt used to generate the initial basis terms set B(0) for the Biology subset of LLM-SRBench, with the benchmark-revealed placeholder values substituted in (top), and the sample LLM output it elicits in a single call (bottom). Each output record carries three fields: term | family/mechanism | one-line description . Ten such records are requested via {count} = 10. LLM backbone: GPT-4o.
Symbolic regression (SR) seeks to discover mathematical expressions directly from data, without requiring task-specific scientific descriptions. Despite tremendous progress in both conventional search-based methods and recent LLM-based methods, two critical limitations largely remain. First (Limited Generation Flexibility), conventional search-based methods rely on either fixed search rules or specialized learned generators, severely limiting adaptive candidate generation across diverse expression structures. Second (Underutilized Search Discoveries), most LLM-based methods cannot reliably recover ground-truth expressions because they fall short in effectively leveraging the LLM itself to thoroughly digest past expressions or adaptively guide future search. To address these limitations, we propose DIGEST. To render greater flexibility in generation, DIGEST repurposes a pretrained decoder block from a lightweight LLM into a symbolic policy for Genetic Programming (GP) population seeding, thereby leveraging the broader modeling capacity of the LLM. To more thoroughly utilize search discoveries, high-quality expressions discovered by GP are fed back to update the LLM-derived policy online, thereby enabling the LLM to progressively internalize past search discoveries and translate them into improved subsequent population generation. Experiments show that on two ground-truth benchmarks, DIGEST outperforms the strongest competitor by up to 2.59% in recovery. Among LLM-based baselines, it improves recovery by at least 54.91% while using at most 17.4% of the strongest competitor's mean search time. On 122 BlackBox datasets, DIGEST achieves the highest accuracy while maintaining a moderate model complexity.
Symbolic regression (SR) seeks to discover parsimonious mathematical laws from observational data, yet conventional approaches often struggle with the vast combinatorial search space of physically meaningful expressions. We present InsightSR, a framework that embeds Large Language Models (LLMs) as a guiding layer around the PySR genetic programming engine. Rather than relying on LLMs to generate expressions directly, InsightSR uses LLMs to progressively transform the search space itself through two complementary pathways: a Semantic Seed Pathway that proposes dimensionally consistent functional skeletons, and a Structural Feature Pathway that recommends nonlinear feature transformations. These transformations accumulate over iterations, broadening the input space and shifting the symbolic search from constructing deep expression trees over raw variables to assembling shallow trees over a rich, semantically informed feature set. A post-generation feedback loop evaluates candidates, categorizes features by their empirical utility, and refines the guidance for the next iteration, transforming the discovery process from open-ended generation into iterative, self-correcting refinement. Across three benchmarks, InsightSR achieves a 95% exact recovery rate on the Feynman benchmark and 80.18% accuracy on the LLM-SRBench LSR-Transform task, substantially outperforming state-of-the-art genetic programming and neural-symbolic methods while maintaining strong out-of-distribution generalization on real-world datasets.
Symbolic regression (SR) seeks explicit equations that explain observed data, and recent methods increasingly employ large language models (LLMs) and agents for equation discovery. However, these systems typically rely on manually designed workflows and prompts. We argue that the SR agent harness should itself be an optimization target: its design shapes search priors and how numerical feedback guides subsequent equation refinement. We therefore propose HarnessSR, a framework for jointly optimizing prompts and workflows to construct reusable SR agents. During both harness optimization and agent execution, HarnessSR supports either PySR or an LLM-based equation generator through tool-specific interfaces, allowing the framework to optimize search guidance for different equation-discovery mechanisms. HarnessSR first optimizes role prompts through equation-discovery feedback in basic workflows, then jointly searches prompt assignments and workflows with brief adaptation to align the two. Finally, it refines role instructions through reflection on residual patterns, candidate equation structures, and search outcomes, improving how these signals guide subsequent equation search. On LSR-Synth in LLM-SRBench, HarnessSR achieves a mean ID Acc0.01 of 61.33% across three backbones, a 45.0% relative improvement over the next-best baseline.
Qian Li, Yuxiao Hu, Qihao Duan +2
Shanghai Jiao Tong University, Shanghai, China · Eastern Institute of Technology, Ningbo, China · The Hong Kong Polytechnic University, Hong Kong, China +1