cs.SCOct 8, 2026

RISR: Residual-Informed Scientific Equation Discovery with Large Language Models

Authors: Haobo Li, Wenshuo Zhang, Wenxiao Zhao, Eunseo Jung, Rui Sheng, Yushi Sun, Peiqin Zhuang, Hao Chen, +1 more

Organizations: Shanghai AI Laboratory · The Hong Kong University of Science and Technology · University of California, Los Angeles

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Symbolic regression combines structural search with numerical fitting, but aggregate fit scores do not describe how the remaining error varies across inputs. We introduce RISR, a residual-informed method that uses these error patterns to guide formula discovery and learn which corrections are worth fitting. A residual encoder compresses aligned inputs, targets, current predictions, and residuals into continuous tokens that condition a language model to propose formulas. For subsequent refinement, a dual-view relational encoder uses additive and regularized multiplicative residuals to predict the post-fit utility of candidate corrections. We evaluate RISR on scientific tasks from the LLM-SRBench. RISR achieves 63.57% and 38.50% ID accuracy at the 1% and 0.1% pointwise relative-error tolerances, respectively. The corresponding OOD accuracies are 56.07% and 38.24%. RISR outperforms the reported baselines using the same backbone. The results show that our residual-informed approach can improve numerical equation recovery.

Figures & tables

Figure 1: Overview of RISR. (A) Residual-informed formula search conditions a frozen LLM on textual context and numerical tokens from the search residual encoder. LLM proposals and learned structural edits are fitted and evaluated to select an initial equation p0p_{0} . (B) Residual-guided refinement encodes additive and regularized multiplicative residual views and predicts the fitted utility of unary and product corrections.
Figure 2: ID and OOD accuracy at a 1%1\% relative-error tolerance. Colors denote methods. Filled and hatched bars represent ID and OOD results, respectively. All LLM-based methods use Qwen3-Coder-30B-A3B-Instruct.
ConfigurationID accuracy ↑\uparrowOOD accuracy ↑\uparrow
LLM-only formula search7.75 / 0.009.30 / 3.10
+ residual-informed formula search (Stage A)36.43 / 23.2640.31 / 30.23
+ coefficient fitting59.69 / 34.1153.49 / 38.76
+ random corrections60.47 / 34.8852.71 / 36.43
+ learned-scored corrections (RISR)63.57 / 37.2155.04 / 38.76
Table 2: The ablation results with one seed setting. Accuracy is reported at the 1%1\% / 0.1%0.1\% tolerances. For Stage B, all rows share the same Stage-A initial equations and numerical fitting protocol.
MethodLLM backboneSA ↑\uparrow
uDSR—0.77
PySR—4.65
LLM-SRGLM-4.5-Air5.43
LLM-SRGPT-OSS-120B4.65
LLM-SRQwen3-Coder-480B5.43
SR-ScientistGLM-4.5-Air7.75
Table 3: SA (%) on one seed setting.
Figure 3: Case study on PO30. RISR retains the target components, removes an extraneous term through structural editing, and calibrates the coefficients during refinement.
Appendix figures & tables10 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 4: Residual encoder architecture.
Figure 5: Numerical and program encoders within the structural edit generator.
Figure 6: Dual-view refinement encoder. The two views share the embedding network and relation-attention weights. The lower panel shows how pair features modify attention logits and values.
CollectionTrainValidationSupervision
Search residual-to-formula6,000768Target formula tokens
Editor residual pretraining50,0005,000Families, variables, gains
Structural edits20,0002,000Edits and fitted outcomes
Paired correction utility8,1921,024Eight actions per state
Appendix
Table 4: Synthetic training collections. Counts are examples except for the paired utility corpus, whose counts are pairs. The three utility test sets are additional to its training and validation splits.
SplitPairsStatesUtility labels
Training8,19216,384131,072
Validation1,0242,04816,384
New observation clouds1,0242,04816,384
Held-out product actions1,0242,04816,384
Eight-variable inputs1,0242,04816,384
Total12,28824,576196,608
Appendix
Table 5: Paired-state utility splits. Each pair has two states and eight actions per state. Only the training split contributes labels to gradient updates.
Result setID, 1%ID, 0.1%OOD, 1%OOD, 0.1%
Seed 065.89 (85)42.64 (55)57.36 (74)41.09 (53)
Seed 4261.24 (79)35.66 (46)55.81 (72)34.88 (45)
2026083163.57 (82)37.21 (48)55.04 (71)38.76 (50)
Mean ±\pm SD63.57±2.3363.57\pm 2.3338.50±3.6638.50\pm 3.6656.07±1.1856.07\pm 1.1838.24±3.1338.24\pm 3.13
Appendix
Table 6: Detailed results for the three seeds.
DomainPositiveTasksSA (%)
Materials1254.00
Chemistry53613.89
Biology32412.50
Physics84418.18
Overall1712913.18
Appendix
Table 7: Symbolic accuracy for the seed 20260831. All positive tasks receive ten Yes votes; all remaining tasks receive ten No votes.
AttemptModeCorrection familyOOD NMSE
0—Fitted parent2.09172.0917
1Additiverq⁡(t) tanh⁡(A)\operatorname{rq}(t)\,\tanh(A)4.0358×10−44.0358\times 10^{-4}
2Additiverq⁡(t) lsq⁡(A)\operatorname{rq}(t)\,\operatorname{lsq}(A)2.9398×10−42.9398\times 10^{-4}
3Additiverq⁡(t) sin⁡(A)\operatorname{rq}(t)\,\sin(A)1.7143×10−41.7143\times 10^{-4}
4Additivesin⁡(t) rq⁡(A)\sin(t)\,\operatorname{rq}(A)6.4102×10−66.4102\times 10^{-6}
Appendix
Table 8: CRK22 action sequence. All four actions are additive and accepted by the training objective. Define rq⁡(z)=(1+z2)−1\operatorname{rq}(z)=(1+z^{2})^{-1} and lsq⁡(z)=log⁡(1+z2)\operatorname{lsq}(z)=\log(1+z^{2}) .
AttemptModeCorrection familyOOD NMSE
0—Fitted parent5.478×10−75.478\times 10^{-7}
1Multiplicativecube⁡(ϵ)square⁡(T)\operatorname{cube}(\epsilon)\operatorname{square}(T)1.471×10−81.471\times 10^{-8}
2Multiplicativetanh⁡(ϵ)linear⁡(T)\tanh(\epsilon)\operatorname{linear}(T)3.322×10−93.322\times 10^{-9}
3Additivelsq⁡(ϵ)lsq⁡(T)\operatorname{lsq}(\epsilon)\operatorname{lsq}(T)2.878×10−92.878\times 10^{-9}
4Multiplicativecube⁡(T)\operatorname{cube}(T)7.566×10−97.566\times 10^{-9}
Appendix
Table 9: MatSci9 refinement trajectory. Action names omit fitted affine parameters, amplitudes, and offsets.
SelectorNew observation cloudsUnseen product actionsEight-variable inputs
Learned utility scorer0.462730.409940.45324
Residual-correlation ranking0.393360.378770.39710
Uniform random selection0.241010.229070.19557
Paired-state swap0.270990.256770.22226
Appendix
Table 10: Held-out utility of the selected action in fixed eight-action candidate pools. Each column contains 2,048 states. Utilities are clipped base-10 log error reductions measured after coefficient fitting. Higher values are better.

Explore similar work

CardsList

  1. Language models guide symbolic equation discovery by controlling search

    Jul 5, 2026Zikai Xie, Wenmei Li, Man Luo +2Large Language Model-Guided Scientific DiscoverySymbolic Regression

    State Key Laboratory of Precision and Intelligent Chemistry, Hefei National Research Center for Physical Sciences at the Microscale, School of Chemistry and Materials Science, University of Science and Technology of China, Hefei, China. · Center for Scientific Intelligence Innovation, Hefei, China · School of Chemistry, School of Computer Science, University of Birmingham, Birmingham, UK

  2. Influence-Guided Symbolic Regression: Scientific Discovery via LLM-Driven Equation Search with Granular Feedback

    May 27, 2026Evgeny S. Saveliev, Samuel Holt, Nabeel Seedat +3GenomicsLarge Language Model-Guided Scientific Discovery

    University of Cambridge · Thomson Reuters Foundational Research · U. Colorado, Anschutz Medical Campus +1

  3. ERBench: A Benchmark and Testsuite for Equation Discovery Algorithms

    Jun 8, 2026Paul Kahlmeyer, Henrik Voigt, Michael Habeck +1Benchmark DesignScientific Discovery

    Department of Computer Science University of Jena Inselplatz 5, 07743 Jena, Germany