DASS: A Solver-Adaptable Dynamic Auxiliary Search Strategy for Symbolic Regression

Yuxiao Hu; Qian Li; Ding Wang; Yekun Zheng; Jinyue Yan; Yuntian Chen

cs.LGOct 12, 2026

DASS: A Solver-Adaptable Dynamic Auxiliary Search Strategy for Symbolic Regression

Authors: Yuxiao Hu, Qian Li, Ding Wang, Yekun Zheng, Jinyue Yan, Yuntian Chen

Organizations: The Hong Kong Polytechnic University, Hong Kong, China · Eastern Institute of Technology, Ningbo, China · Shanghai Jiao Tong University, Shanghai, China

Abstract

Symbolic regression (SR) aims to discover interpretable mathematical expressions from data and plays a key role in scientific discovery. However, existing methods face a common bottleneck: the enormous search space and severe combinatorial explosion make it difficult to achieve a favorable trade-off between accuracy and algorithmic efficiency. We argue that, in addition to designing increasingly complex solver-specific SR strategies, an equally promising direction is to equip them with a shareable auxiliary strategy that can provide reliable search guidance. To this end, we propose DASS, a solver-adaptable Dynamic Auxiliary Search Strategy that improves SR by constructing and refining a high-potential auxiliary search subspace. DASS represents the auxiliary search space as a functional subspace spanned by a set of high-potential basis function terms. It estimates term utility through multi-environment evaluation, filters unstable guidance, and updates term potential scores with a Gibbs posterior, thereby calibrating the subspace to provide more reliable guidance for downstream solvers. The high-quality solutions produced by the solver are used to update the set of basis terms. This forms a virtuous closed-loop optimization process: the subspace guides the downstream solver, and the solver feedback reshapes the subspace for subsequent exploration. DASS can be easily integrated with various types of SR solvers. Extensive experiments on LLM-SRBench show that DASS significantly improves the accuracy of original solvers with a manageable increase in runtime.

Figures & tables

Figure 1: Core idea of DASS. Instead of letting an SR solver search the full expression space without guidance, DASS provides a solver-adaptable auxiliary search subspace spanned by high-potential basis terms. The subspace is initialized from LLM prior knowledge, calibrated by multi-environment evaluation, and reshaped using feedback from high-quality solver results. In this way, DASS guides diverse SR solvers toward promising regions while continuously refining the guided subspace.

Figure 2: Framework of DASS. DASS initializes an auxiliary search subspace from prior knowledge, calibrates it through multi-environment evaluation, stability filtering, and Gibbs posterior updating, and uses the resulting high-potential terms to guide downstream SR solvers. High-quality solver outputs are then fed back to reshape the subspace, yielding a closed-loop optimization process. In addition to the three SR solvers shown in the figure, we include two additional solver types in the Appendix D : latent-space-based and policy-based solvers.

LLM-based	Qwen3.5-9B
Category	Method	Phys			MatSci			Chem			Bio
Category	Method	Time $\downarrow$	NMSE $\downarrow$	$\mathrm{Acc}_{0.01}$ $\uparrow$	Time $\downarrow$	NMSE $\downarrow$	$\mathrm{Acc}_{0.01}$ $\uparrow$	Time $\downarrow$	NMSE $\downarrow$	$\mathrm{Acc}_{0.01}$ $\uparrow$	Time $\downarrow$	NMSE $\downarrow$	$\mathrm{Acc}_{0.01}$ $\uparrow$
	LLMSR	197.2	5.32E-03	9.1	185	2.38E-05	38.7	177.3	4.96E-04	7.4	187.9	2.14E-03	6.9
	LLMSR+DASS	205.1	6.04E-04	29.5	257.2	2.91E-06	81.3	260.4	1.15E-04	23.1	284.2	4.52E-04	13.9
	LASR	413.5	2.24E-02	6.8	502.7	6.76E-04	16.0	477.1	5.24E-03	13.9	429	5.64E-02	6.9
	LASR+DASS	523.8	4.18E-03	18.9	566.9	8.15E-05	41.3	496.2	6.24E-04	18.5	491.3	7.64E-03	12.5

Table 1: Main results on the four domain-specific datasets in LLM-SRBench. We evaluate DASS by integrating it into five representative SR solvers and report the average search time (Time), normalized mean squared error (NMSE), and

\mathrm{Acc}_{0.01}

. Shaded rows indicate DASS-augmented variants, and boldface denotes better value within each base solver / +DASS pair.

Figure 3: Low-budget DASS versus high-budget base solvers on LLMSR, PySR, and SPL. Each solver is evaluated under low and high search budgets with all other hyperparameters fixed; the high-budget setting increases LLMSR max LLM calls from $100$ to $1000$ , PySR outer iterations from $65$ to $130$ , and SPL MCTS transplants from $10$ to $20$ . Markers indicate low-budget base solver (light-blue circle), low-budget DASS-augmented solver (green star), and high-budget base solver (large brown circle). Results are averaged over four domains. The $x$ -axis gives relative search time normalized by the low-budget base solver ( $1{\times}$ ), and the $y$ -axis gives $\mathrm{Acc}_{0.01}$ .

Setting	Variant	LLMSR				PySR				SPL
Setting	Variant	Phys	MatSci	Chem	Bio	Phys	MatSci	Chem	Bio	Phys	MatSci	Chem	Bio
Single-env. utility	Extrap	26.5	74.7	20.4	9.7	21.2	32.0	21.3	30.6	18.2	25.3	21.3	12.5
	Sparse	22.0	66.7	15.7	8.3	20.5	29.3	19.4	26.4	13.6	22.7	19.4	11.1
	Noisy	24.2	72.0	21.3	8.3	21.2	30.7	18.5	27.8	16.7	22.7	22.2	11.1
Multi-env. utility (DASS)		29.5	81.3	23.1	13.9	22.7	36.0	25.0	33.3	22.0	26.7	24.1	12.5

Table 2: Ablation study on multi-environment evaluation, measured by

\mathrm{Acc}_{0.01}

. All other hyperparameters are kept fixed, and we vary only the choice of environments used to compute utility. (1) Single env: evaluates utility using only one environment in

\mathcal{E}=\{\rm extrap,sparse,noisy\}

. (2) All three envs: the DASS setting, which alternates sampling across the three evaluation environments.

Appendix figures & tables24 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 4: An example DASS-augmented LLM-SR prompt from the Physics subset of LLM-SRBench, shown in a compact two-column layout. The Reference Terms (DASS) panel at the top of the second column is the only DASS-specific addition, listing the top-

k\%

terms from

\mathcal{B}^{(t)}

ranked by

\bm{\omega}^{(t)}

as additional structural guidance for the LLM.

Category	Method	Phys		MatSci		Chem		Bio
Category	Method	NMSE $\downarrow$	$\mathrm{Acc}_{0.01}$ $\uparrow$	NMSE $\downarrow$	$\mathrm{Acc}_{0.01}$ $\uparrow$	NMSE $\downarrow$	$\mathrm{Acc}_{0.01}$ $\uparrow$	NMSE $\downarrow$	$\mathrm{Acc}_{0.01}$ $\uparrow$
RL-based	DSR	4.82E-01	0	8.65E-03	0	9.31E-01	0	1.54E+00	0
RL-based	DSR+DASS	8.41E-02	0	2.97E-03	0	2.04E-01	0	7.18E-01	0
Latent-space-based	GenSR	5.21E-03	4.5	8.32E-05	18.7	2.86E-01	8.3	7.85E-01	4.2
Latent-space-based	GenSR+DASS	1.34E-03	12.1	2.18E-05	33.3	4.21E-02	16.7	8.62E-02	9.7

Appendix

Table 3: Performance of DSR and GenSR before and after coupling with DASS through the action-prior (DSR) and latent-prior (GenSR) interfaces. DASS-augmented rows shaded gray.

Method	Budget	$\mathrm{Acc}_{0.01}$	Complexity	SA
Qwen3.5-9B
LLM-SR	100 calls	13.95	48.5	2.58
LLM-SR + DASS	100 calls	34.85	50.1	4.65
LLM-SR (higher budget)	1,000 calls	35.60	55.0	4.91
LaSR	20 iterations	10.58	24.2	3.10
LaSR + DASS	20 iterations	21.94	22.7	4.91

Appendix

Table 4: Numerical accuracy, expression complexity, and symbolic accuracy on LSR-Synth for LLM-based solvers. DASS variants are compared with their base solvers at the same downstream search budget. The higher-budget LLM-SR result is provided separately as a reference. Accuracy values are percentages; higher accuracy and lower complexity are preferred.

Method	Budget	$\mathrm{Acc}_{0.01}$	Complexity	SA
PySR	65 iterations	17.83	19.6	3.10
PySR + DASS	65 iterations	27.89	21.2	3.88
PySR (higher budget)	130 iterations	27.40	23.8	4.39
SPL	10 transplants	12.12	16.8	0.52
SPL + DASS	10 transplants	21.73	20.4	0.52
SPL (higher budget)	20 transplants	18.10	18.6	0.78

Appendix

Table 5: Additional evaluation of GP-based and MCTS-based solvers on LSR-Synth. Higher-budget base-solver configurations are reference points rather than matched-budget comparisons. Accuracy values are percentages.

Method	$\mathrm{Acc}_{0.01}$	$\mathrm{Acc}_{0.001}$	Complexity	SA
SR-Scientist	49.61	39.53	28.5	4.65
SR-Scientist + DASS	61.24	43.41	26.2	6.20

Appendix

Table 6: Integration with SR-Scientist using GPT-OSS-120B on LSR-Synth. Both configurations use a maximum budget of

10\times 25

downstream calls. Accuracy values are percentages.

Backbone	Method	$\mathrm{Acc}_{0.01}$	Complexity	SA
Qwen3.5-9B	SGA	9.30	40.4	1.81
Qwen3.5-9B	SGA + DASS	19.64	41.5	3.36
Gemma-4-31B	SGA	13.70	39.6	2.58
Gemma-4-31B	SGA + DASS	23.26	36.8	4.13

Appendix

Table 7: Integration with SGA on LSR-Synth using 100 downstream calls. Each comparison holds the backbone and downstream call budget fixed. Accuracy values are percentages.

Method	Llama-3.1-8B	GPT-4o	Gemma-4-31B
LLM-SR	13.95	13.95	13.95
LLM-SR + DASS	25.58	34.85	37.21
LLM-SR + DASS (randomized)	22.48	31.01	33.33
PySR	17.83	17.83	17.83
PySR + DASS	23.26	27.89	30.23
PySR + DASS (randomized)	20.16	25.58	28.68

Appendix

Table 8: Sensitivity to the initial basis library on LSR-Synth, measured by

\mathrm{Acc}_{0.01}

(%). Baselines do not use a basis library; their results are repeated for comparison. “Randomized” changes both term selection and initial potential scores within an LLM-generated candidate pool.

Initial search space for the target task	$\mathrm{Acc}_{0.01}$
No auxiliary space (PySR without DASS)	18.2
Physics domain-level prior (standard DASS)	22.7
Final space from another Physics task	24.2
Final space from a Chemistry task	20.5

Appendix

Table 9: Transfer to Physics tasks with low-budget PySR. All DASS settings continue adapting the space on the target task. The reported

\mathrm{Acc}_{0.01}

values are percentages.

Method	$\mathrm{Acc}_{0.01}$	Time (s)
LLM-SR	13.95	187.6
LLM-SR + Counting DASS	25.84	241.5
LLM-SR + Original DASS	34.85	245.3
LLM-SR + Marginal DASS	36.18	364.1

Appendix

Table 10: Term-level utility attribution with LLM-SR and a downstream budget of 100 calls. Accuracy is

\mathrm{Acc}_{0.01}

(%); time is the reported search time in seconds. The marginal variant includes additional term-removal, coefficient-refitting, and fitness-evaluation work.

LLM-based	Qwen3.5-9B
Category	Method	Phys		MatSci		Chem		Bio
Category	Method	NMSE $\downarrow$	$\mathrm{Acc}_{0.01}$ $\uparrow$	NMSE $\downarrow$	$\mathrm{Acc}_{0.01}$ $\uparrow$	NMSE $\downarrow$	$\mathrm{Acc}_{0.01}$ $\uparrow$	NMSE $\downarrow$	$\mathrm{Acc}_{0.01}$ $\uparrow$
	LLMSR	2.18E-02	8.3	1.10E-05	45.3	7.26E-03	5.6	4.46E-02	5.6
	LLMSR+DASS	2.07E-03	28.0	2.32E-06	85.3	1.84E-03	22.2	7.33E-03	12.5
	LASR	9.90E-02	5.3	5.36E-04	30.7	9.54E-02	10.2	1.10E-03	4.2
	LASR+DASS	1.19E-02	19.7	6.78E-05	50.7	1.09E-02	16.7	8.21E-04	11.1

Appendix

Table 11: OOD performance across 4 LSR-Synth domains. Time is omitted (identical to Table 1 ); DASS-augmented rows are shaded gray.

Figure 5: Budget efficiency on the Physics domain. Each panel corresponds to one base solver (LLMSR, PySR, SPL). The horizontal axis shows relative search time (base-solver low budget set to

1\times

); the vertical axis shows

\mathrm{Acc}_{0.01}

. The three markers within each panel correspond to the base solver at low budget, the base solver augmented with DASS at low budget, and the base solver at high budget.

Figure 6: Budget efficiency on the Material Science domain; the setup is identical to Fig. 5 .

Figure 7: Budget efficiency on the Chemistry domain; the setup is identical to Fig. 5 .

Figure 8: Budget efficiency on the Biology domain; the setup is identical to Fig. 5 .

Update mechanisms enabled	LLMSR				PySR				SPL
Update mechanisms enabled	Phys	MatSci	Chem	Bio	Phys	MatSci	Chem	Bio	Phys	MatSci	Chem	Bio
Frozen prior ( $\mathcal{B},\boldsymbol{\omega}$ fixed after init)	17.4	64.0	13.9	9.7	20.5	29.3	18.5	26.4	14.4	22.7	19.4	11.1
+ Calibration (only $\boldsymbol{\omega}$ updated)	25.0	73.3	20.4	12.5	22.7	33.3	23.1	30.6	19.7	24.0	22.2	11.1
+ Reshaping (DASS; $\mathcal{B},\boldsymbol{\omega}$ both updated)	29.5	81.3	23.1	13.9	22.7	36.0	25.0	33.3	22.0	26.7	24.1	12.5

Appendix

Table 12: Ablation study on the subspace update mechanism, measured by

\mathrm{Acc}_{0.01}

. The three settings share the same LLM-generated initial basis term set

\mathcal{B}^{(0)}

and initial scores

\bm{\omega}^{(0)}

. They differ only in which dynamic update mechanisms are enabled after initialization: (1) Frozen prior , where both

\mathcal{B}

and

\bm{\omega}

are frozen after initialization; (2) + Calibration , where only subspace calibration is enabled; and (3) + Reshaping , where both subspace calibration and subspace reshaping are enabled, so that

\mathcal{B}

and

\bm{\omega}

are updated according to search feedback. The last setting corresponds to the full DASS.

Figure 9: Sensitivity to the sample count

m

. Left: four-domain average

\mathrm{Acc}_{0.01}

plotted against

m

, with the three curves corresponding to LLM-SR, PySR, and SPL. Right: the corresponding average per-problem search time for each value of

m

, measured as a multiple of the search time at

m=50

Figure 10: Sensitivity to the basis-set size

n

. 4-domain mean

\mathrm{Acc}_{0.01}

for LLM-SR, PySR, and SPL with

n\in\{15,25,35,45,55\}

Figure 11: Sensitivity to the elite ratio

k

. 4-domain mean

\mathrm{Acc}_{0.01}

for LLM-SR, PySR, and SPL with

k\in\{10\%,20\%,30\%,40\%,50\%\}

Figure 12: Sensitivity to the insertion count

T

. Left: 4-domain mean

\mathrm{Acc}_{0.01}

for LLM-SR, PySR, and SPL with

T\in\{1,3,6,9,12\}

. Right: the corresponding average per-problem search time for each value of

T

, measured as a multiple of the search time at

T=1

Figure 13: Overview of the prompt used to initialize the basis term set

\mathcal{B}

. Top: a unified prompt template with six placeholders, including five domain-related fields and {count} , which specifies the number of terms to generate. Bottom: domain-placeholder values for Physics, Materials Science, Chemistry, and Biology in LLM-SRBench, instantiated from benchmark-provided metadata available as task input, not from target equations or evaluation data.

Panel of Figure 14 — Figure 14: Prompt used to generate the initial basis terms set $\mathcal{B}^{(0)}$ for the Physics subset of LLM-SRBench, with the benchmark-revealed placeholder values substituted in (top), and the sample LLM output it elicits in a single call (bottom). Each output record carries three fields: term | family/mechanism | one-line description . Ten such records are requested via {count} = 10. LLM backbone: GPT-4o.

Panel of Figure 15 — Figure 15: Prompt used to generate the initial basis terms set $\mathcal{B}^{(0)}$ for the Material Science subset of LLM-SRBench, with the benchmark-revealed placeholder values substituted in (top), and the sample LLM output it elicits in a single call (bottom). Each output record carries three fields: term | family/mechanism | one-line description . Ten such records are requested via {count} = 10. LLM backbone: GPT-4o.

Panel of Figure 16 — Figure 16: Prompt used to generate the initial basis terms set $\mathcal{B}^{(0)}$ for the Chemistry subset of LLM-SRBench, with the benchmark-revealed placeholder values substituted in (top), and the sample LLM output it elicits in a single call (bottom). Each output record carries three fields: term | family/mechanism | one-line description . Ten such records are requested via {count} = 10. LLM backbone: GPT-4o.

Panel of Figure 17 — Figure 17: Prompt used to generate the initial basis terms set $\mathcal{B}^{(0)}$ for the Biology subset of LLM-SRBench, with the benchmark-revealed placeholder values substituted in (top), and the sample LLM output it elicits in a single call (bottom). Each output record carries three fields: term | family/mechanism | one-line description . Ten such records are requested via {count} = 10. LLM backbone: GPT-4o.

Symbolic Regression

Symbolic regression searches data for compact mathematical expressions that balance predictive accuracy with interpretability.

See all 148 papers

DIGEST: Discovery-Guided LLM Tuning for Symbolic Regression
Oct 24, 2026Kaifeng Jin, Ruizhong Qiu, Zihao Li +5Symbolic Regression Large Language Model-Guided Optimization
University of Illinois Urbana-Champaign · Meta
InsightSR: Refining Symbolic Regression Search Spaces via Parallel Semantic and Structural LLM Guidance
Aug 26, 2026Yating Ling, Wenjing Cun, Zhitang ChenSymbolic Regression Large Language Model-Guided Scientific Discovery
Learning to Design SR Solvers: Harness Optimization for Symbolic Regression
Oct 12, 2026Qian Li, Yuxiao Hu, Qihao Duan +2Symbolic Regression Agent Harness Optimization
Shanghai Jiao Tong University, Shanghai, China · Eastern Institute of Technology, Ningbo, China · The Hong Kong Polytechnic University, Hong Kong, China +1

Oct 24, 2026J/K move · Enter open · S save

DIGEST: Discovery-Guided LLM Tuning for Symbolic Regression

Kaifeng Jin, Ruizhong Qiu, Zihao Li, Xiao Lin+4

University of Illinois Urbana-Champaign · Meta

Symbolic Regression Large Language Model-Guided Optimization Language Model-Guided Search Heuristics Genetic Programming

Open paper AskPDF arXiv

Symbolic regression (SR) seeks to discover mathematical expressions directly from data, without requiring task-specific scientific descriptions. Despite tremendous progress in both conventional search-based methods and recent LLM-based methods, two critical limitations largely remain. First (Limited Generation Flexibility), conventional search-based methods rely on either fixed search rules or specialized learned generators, severely limiting adaptive candidate generation across diverse expression structures. Second (Underutilized Search Discoveries), most LLM-based methods cannot reliably recover ground-truth expressions because they fall short in effectively leveraging the LLM itself to thoroughly digest past expressions or adaptively guide future search. To address these limitations, we propose DIGEST. To render greater flexibility in generation, DIGEST repurposes a pretrained decoder block from a lightweight LLM into a symbolic policy for Genetic Programming (GP) population seeding, thereby leveraging the broader modeling capacity of the LLM. To more thoroughly utilize search discoveries, high-quality expressions discovered by GP are fed back to update the LLM-derived policy online, thereby enabling the LLM to progressively internalize past search discoveries and translate them into improved subsequent population generation. Experiments show that on two ground-truth benchmarks, DIGEST outperforms the strongest competitor by up to 2.59% in recovery. Among LLM-based baselines, it improves recovery by at least 54.91% while using at most 17.4% of the strongest competitor's mean search time. On 122 BlackBox datasets, DIGEST achieves the highest accuracy while maintaining a moderate model complexity.

Abstract

Symbolic Regression

Explore similar work

DIGEST: Discovery-Guided LLM Tuning for Symbolic Regression

InsightSR: Refining Symbolic Regression Search Spaces via Parallel Semantic and Structural LLM Guidance

Learning to Design SR Solvers: Harness Optimization for Symbolic Regression