cs.AIOct 2, 2026

Training Numerical Intelligence via Auto-Diagnosis and Skill Discovery

Authors: Peter Chen, Wotao Yin

Organizations: BAIR & EECS University of California, Berkeley Berkeley, CA, USA · Decision Intelligence Lab DAMO Academy, Alibaba Group U.S. Bellevue, WA, USA

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

AI agents are becoming increasingly capable of generating scientific code, but generating code is not the same as improving the algorithms behind it. For numerical solvers, execution feedback can expose poor performance, but rarely reveals its underlying cause and how to address it. We introduce Auto-Diagnosis and Skill Discovery (ADSD), a framework that links numerical diagnosis to reusable solver self-improvement. ADSD follows a diagnosis-first paradigm that first explains why a solver performs poorly, then uses this diagnosis to guide the discovery of appropriate numerical methods. The resulting knowledge is packaged into reusable solver skills, turning solver improvement from trial-and-error editing into a structured process of diagnosis, discovery, and implementation. Across four challenging numerical domains--power flow equation, AC optimal power flow control, stiff ordinary differential equations, and heterogeneous diffusion PDEs--ADSD consistently improves solver accuracy, robustness, and efficiency. On GOC-500 power flow, for example, ADSD reduces mean solver error by nearly 71×71\times, with improvements further transferring to unseen grid topologies and operating regimes.

Figures & tables

Figure 1: Overview of ADSD. Auto-diagnosis combines execution feedback from the frozen baseline solver N0N_{0} on public test cases with literature-derived diagnostic skills to identify numerical failure patterns. Skill discovery uses verified hypotheses to retrieve candidate methods and complete their implementations. The resulting reusable solver skills are supplied, together with N0N_{0} , to a fresh agent to generate the improved solver BB .
Base ModelConfigurationIEEE-118GOC-500
Run time (↓)(\downarrow)Error (↓)(\downarrow)Run time (↓)(\downarrow)Error (↓)(\downarrow)
GPT-5.5N0N_{0} solver0.73±0.210.73\pm 0.219.1×10−10±2.0×10−99.1{\times}10^{-10}\pm 2.0{\times}10^{-9}0.51±0.140.51\pm 0.149.3×10−10±2.2×10−99.3{\times}10^{-10}\pm 2.2{\times}10^{-9}
+ ADSD0.55±0.150.55\pm 0.152.7×10−10±9.9×10−102.7{\times}10^{-10}\pm 9.9{\times}10^{-10}0.50±0.140.50\pm 0.141.6×10−10±7.4×10−101.6{\times}10^{-10}\pm 7.4{\times}10^{-10}
Claude-Sonnet-5N0N_{0} solver0.16±0.040.16\pm 0.041.1×10−9±2.1×10−91.1{\times}10^{-9}\pm 2.1{\times}10^{-9}0.52±0.260.52\pm 0.267.9×10−10±1.9×10−97.9{\times}10^{-10}\pm 1.9{\times}10^{-9}
+ ADSD0.15±0.040.15\pm 0.047.4×10−10±1.8×10−97.4{\times}10^{-10}\pm 1.8{\times}10^{-9}0.52±0.250.52\pm 0.251.4×10−10±8.4×10−101.4{\times}10^{-10}\pm 8.4{\times}10^{-10}
Qwen3.8-2.4TN0N_{0} solver0.57±0.020.57\pm 0.029.9×10−10±2.1×10−99.9{\times}10^{-10}\pm 2.1{\times}10^{-9}0.18±0.090.18\pm 0.091.0×10−9±2.1×10−91.0{\times}10^{-9}\pm 2.1{\times}10^{-9}
Table 1: Performance on the IEEE-118 and GOC-500 validation sets. Values are mean ±\pm standard deviation. “ N0N_{0} solver” stands for the baseline performance using agent’s inherent knowledge, and “+ ADSD” denotes the agent’s performance after training with ADSD pipeline.
Figure 2: ADSD performance on IEEE-118 and GOC-500 under (a) topology generalization from N2 to unseen N1 / N grids and (b) feasibility generalization from feasible ( F ) to close-to-infeasible ( IF ) cases. Over all cases, learned skills can be well generalized to different grid setups and difficulties. Detailed results for figures (a) and (b) are presented in Tables 6 and 7 , respectively.
Base ModelConfigurationMean error (↓)(\downarrow)Runtime (↓)(\downarrow)Pass rate (↑)(\uparrow)
GPT-5.5N0N_{0} solver4.8×10−7±6.5×10−74.8\times 10^{-7}\pm 6.5\times 10^{-7}0.80±0.220.80\pm 0.2297.82%97.82\%
+ ADSD4.0×10−7±6.4×10−74.0\times 10^{-7}\pm 6.4\times 10^{-7}0.74±0.240.74\pm 0.2499.26%99.26\%
Claude-Sonnet-5N0N_{0} solver2.5×10−6±5.5×10−62.5\times 10^{-6}\pm 5.5\times 10^{-6}0.62±0.110.62\pm 0.1178.78%78.78\%
+ ADSD1.9×10−6±5.4×10−61.9\times 10^{-6}\pm 5.4\times 10^{-6}0.42±0.090.42\pm 0.0981.42%81.42\%
Qwen3.8-2.4TN0N_{0} solver2.7×10−6±8.6×10−72.7\times 10^{-6}\pm 8.6\times 10^{-7}0.26±0.040.26\pm 0.0414.42%14.42\%
+ ADSD1.2×10−6±8.2×10−71.2\times 10^{-6}\pm 8.2\times 10^{-7}0.18±0.050.18\pm 0.0598.36%98.36\%
Table 2: Performance on 5,000 AC-OPF hidden evaluation cases. N0N_{0} denotes the original solver and ADSD denotes its skill-augmented solver. Mean error is calculated as ∣f−fref∣/max⁡(∣fref∣,1)|f-f_{\mathrm{ref}}|/\max(|f_{\mathrm{ref}}|,1) , across all cases that N0N_{0} and ADSD solver passed in common.
Base ModelConfigurationPass rate (↑)(\uparrow)Median error (↓)(\downarrow)Time (↓)(\downarrow)
GPT-5.5N0N_{0} solver70.1%70.1\%9.39×10−69.39\times 10^{-6}4.713±9.5704.713\pm 9.570
+ ADSD90.6%90.6\%3.93×10−73.93\times 10^{-7}3.451±6.7433.451\pm 6.743
Claude-Sonnet-5N0N_{0} solver94.0%94.0\%4.43×10−74.43\times 10^{-7}0.618±1.4570.618\pm 1.457
+ ADSD100.0%100.0\%5.04×10−85.04\times 10^{-8}0.496±0.7820.496\pm 0.782
Qwen3.8-2.4TN0N_{0} solver98.3%98.3\%1.30×10−81.30\times 10^{-8}0.961±1.6880.961\pm 1.688
+ ADSD99.2%99.2\%5.95×10−95.95\times 10^{-9}0.646±1.0830.646\pm 1.083
Table 3: Performance on held-out stiff-ODE cases from 72 source families with 117 models. N0N_{0} denotes the solver generated from the agent’s inherent knowledge, and ADSD denotes the fresh solver generated with the constructed skill packet. Error is the median trajectory error. Wall time usages are per-case mean ±\pm standard deviation across all attempts.
Base ModelConfigurationℓ2\ell_{2} error (↓)(\downarrow)ℓ∞\ell_{\infty} error (↓)(\downarrow)Time (↓)(\downarrow)Operator calls (↓)(\downarrow)
GPT-5.5N0N_{0} solver1.69×10−9±0.86×10−91.69\times 10^{-9}\pm 0.86\times 10^{-9}1.46×10−9±0.91×10−91.46\times 10^{-9}\pm 0.91\times 10^{-9}7.29±1.327.29\pm 1.32215.49±40.35215.49\pm 40.35
+ ADSD5.27×10−10±2.04×10−105.27\times 10^{-10}\pm 2.04\times 10^{-10}5.93×10−10±3.33×10−105.93\times 10^{-10}\pm 3.33\times 10^{-10}2.78±0.582.78\pm 0.5871.22±17.5871.22\pm 17.58
Claude-Sonnet-5N0N_{0} solver1.12×10−11±0.54×10−111.12\times 10^{-11}\pm 0.54\times 10^{-11}9.96×10−12±6.12×10−129.96\times 10^{-12}\pm 6.12\times 10^{-12}0.86±0.030.86\pm 0.03148.16±4.82148.16\pm 4.82
+ ADSD8.15×10−12±5.41×10−128.15\times 10^{-12}\pm 5.41\times 10^{-12}5.49×10−12±4.97×10−125.49\times 10^{-12}\pm 4.97\times 10^{-12}0.79±0.020.79\pm 0.02125.71±13.40125.71\pm 13.40
Qwen3.8-2.4TN0N_{0} solver6.49×10−9±3.53×10−96.49\times 10^{-9}\pm 3.53\times 10^{-9}4.52×10−9±2.83×10−94.52\times 10^{-9}\pm 2.83\times 10^{-9}0.70±0.090.70\pm 0.0986.81±29.6886.81\pm 29.68
+ ADSD3.55×10−10±2.58×10−103.55\times 10^{-10}\pm 2.58\times 10^{-10}3.72×10−10±2.79×10−103.72\times 10^{-10}\pm 2.79\times 10^{-10}0.64±0.020.64\pm 0.0217.57±10.0317.57\pm 10.03
Table 4: Performance on 2,400 held-out ConDiff cases. Errors are relative to the benchmark’s discrete reference solutions. All error, wall-time, and operator-call statistics are mean ±\pm population std across 7,200 runs per solver (three repetitions per case).
Figure 3: Comparison of the N0N_{0} solver, self improvement, and ADSD on (a) IEEE-118, (b) GOC-500, and (c) stiff ODEs. Bars show mean error for grid tasks and median trajectory error for ODEs. Panel (c) uses a logarithmic scale; lower is better throughout.
Figure 4: Mean error on GOC-500 across recursive self-improvement iterations for (a) GPT-5.5, (b) Claude-Sonnet-5, and (c) Qwen3.8-2.4T. Dashed lines indicate the N0N_{0} and ADSD baselines. Iteration 0 denotes the original solver; lower is better.
Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

SolverMultistepMethod 9FactorizationsLinear solvesRuntime (s)PF residual
N0N_{0}––5.1165.1160.7279.10×10−109.10\times 10^{-10}
B1B_{1}✓\checkmark–2.7558.2550.4949.86×10−109.86\times 10^{-10}
B2B_{2}–✓\checkmark4.0038.0060.6547.46×10−137.46\times 10^{-13}
B3B_{3}✓\checkmark✓\checkmark3.1147.1140.5382.76×10−102.76\times 10^{-10}
Appendix
Table 5: Solver-skill ablation on IEEE-118 validation set. Each BB variant is separately generated with mandatory integration of the indicated skills. Factorizations, linear solves, runtime, and residual are per-case means. The residual is the independently evaluated ∥FPF(x;ξ)∥∞\|F_{\mathrm{PF}}(x;\xi)\|_{\infty} .
Base ModelDatasetConfigurationIn-domain N2Generalization N1Generalization N
Time (↓)(\downarrow)Error (↓)(\downarrow)Time (↓)(\downarrow)Error (↓)(\downarrow)Time (↓)(\downarrow)Error (↓)(\downarrow)
GPT-5.5IEEE-118N0N_{0} solver0.73±0.210.73\pm 0.219.1×10−10±2.0×10−99.1{\times}10^{-10}\pm 2.0{\times}10^{-9}0.74±0.210.74\pm 0.217.6×10−10±1.8×10−97.6{\times}10^{-10}\pm 1.8{\times}10^{-9}0.75±0.200.75\pm 0.208.9×10−10±2.0×10−98.9{\times}10^{-10}\pm 2.0{\times}10^{-9}
+ ADSD0.55±0.150.55\pm 0.152.7×10−10±9.9×10−102.7{\times}10^{-10}\pm 9.9{\times}10^{-10}0.54±0.150.54\pm 0.152.1×10−10±9.1×10−102.1{\times}10^{-10}\pm 9.1{\times}10^{-10}0.53±0.140.53\pm 0.142.3×10−10±8.3×10−102.3{\times}10^{-10}\pm 8.3{\times}10^{-10}
GOC-500N0N_{0} solver0.51±0.140.51\pm 0.149.3×10−10±2.2×10−99.3{\times}10^{-10}\pm 2.2{\times}10^{-9}0.38±0.090.38\pm 0.091.2×10−9±2.5×10−91.2{\times}10^{-9}\pm 2.5{\times}10^{-9}0.37±0.090.37\pm 0.098.6×10−10±2.1×10−98.6{\times}10^{-10}\pm 2.1{\times}10^{-9}
+ ADSD0.50±0.140.50\pm 0.141.6×10−10±7.4×10−101.6{\times}10^{-10}\pm 7.4{\times}10^{-10}0.62±0.150.62\pm 0.151.1×10−10±5.4×10−101.1{\times}10^{-10}\pm 5.4{\times}10^{-10}0.60±0.150.60\pm 0.151.9×10−10±7.9×10−101.9{\times}10^{-10}\pm 7.9{\times}10^{-10}
Claude-Sonnet-5IEEE-118N0N_{0} solver0.16±0.040.16\pm 0.041.1×10−9±2.1×10−91.1{\times}10^{-9}\pm 2.1{\times}10^{-9}0.18±0.040.18\pm 0.041.1×10−9±1.9×10−91.1{\times}10^{-9}\pm 1.9{\times}10^{-9}0.13±0.010.13\pm 0.011.1×10−9±1.9×10−91.1{\times}10^{-9}\pm 1.9{\times}10^{-9}
Appendix
Table 6: Topology generalization from the feasible N2 -topology grid to previously unseen feasible N1 and full-grid N cases. Values are mean ±\pm std over 10,000 cases per generalization topology.
Base ModelDatasetConfigurationIn-domain FGeneralization IF
Time (↓)(\downarrow)Error (↓)(\downarrow)Time (↓)(\downarrow)Error (↓)(\downarrow)
GPT-5.5IEEE-118N0N_{0} solver0.73±0.210.73\pm 0.219.1×10−10±2.0×10−99.1{\times}10^{-10}\pm 2.0{\times}10^{-9}1.68±0.241.68\pm 0.241.1×10−9±2.1×10−91.1{\times}10^{-9}\pm 2.1{\times}10^{-9}
+ ADSD0.55±0.150.55\pm 0.152.7×10−10±9.9×10−102.7{\times}10^{-10}\pm 9.9{\times}10^{-10}1.25±0.211.25\pm 0.213.3×10−10±1.3×10−93.3{\times}10^{-10}\pm 1.3{\times}10^{-9}
GOC-500N0N_{0} solver0.51±0.140.51\pm 0.149.3×10−10±2.2×10−99.3{\times}10^{-10}\pm 2.2{\times}10^{-9}0.52±0.140.52\pm 0.141.1×10−9±2.0×10−91.1{\times}10^{-9}\pm 2.0{\times}10^{-9}
+ ADSD0.50±0.140.50\pm 0.141.6×10−10±7.4×10−101.6{\times}10^{-10}\pm 7.4{\times}10^{-10}0.53±0.140.53\pm 0.148.5×10−10±1.9×10−98.5{\times}10^{-10}\pm 1.9{\times}10^{-9}
Claude-Sonnet-5IEEE-118N0N_{0} solver0.16±0.040.16\pm 0.041.1×10−9±2.1×10−91.1{\times}10^{-9}\pm 2.1{\times}10^{-9}0.73±0.110.73\pm 0.111.2×10−9±2.2×10−91.2{\times}10^{-9}\pm 2.2{\times}10^{-9}
Appendix
Table 7: Feasibility generalization from the feasible training set to previously unseen close-to-infeasible grids. Values are mean ±\pm std over 5,000 sampled from close-to-infeasible testcases.
More in this topic

Agent Skill Learning

Agent skill learning develops reusable procedures and behaviors that help AI agents solve new tasks and adapt from experience.

See all 174 papers

Explore similar work

CardsList

  1. AutoNumerics: An Autonomous, PDE-Agnostic Multi-Agent Pipeline for Scientific Computing

    Feb 19, 2026Jianda Du, Youran Sun, Haizhao YangPDE SolvingAutomated Algorithm Discovery

    Department of Mathematics, University of Maryland, College Park, MD, USA · Department of Computer Science, University of Maryland, College Park, MD, USA

  2. AutoPDE: Reliable Agentic PDE Solving via Explicitly Represented Solver Strategies

    Jun 9, 2026Huanshuo Dong, Keyao Zhang, Hong Wang +6AI Coding AgentsPDE Solving

    University of Science and Technology of China · 2Zhejiang University · University of the Chinese Academy of Sciences +2

  3. ATHENA: Agentic Team for Hierarchical Evolutionary Numerical Algorithms

    Dec 3, 2025Juan Diego Toscano, Daniel T. Chen, George Em KarniadakisScientific Code GenerationScientific ML

    Division of Applied Mathematics, Brown University, Providence, 02912, RI, USA