cs.AIOct 2, 2026

Training Numerical Intelligence via Auto-Diagnosis and Skill Discovery

Authors: Peter Chen, Wotao Yin

Organizations: BAIR & EECS University of California, Berkeley Berkeley, CA, USA · Decision Intelligence Lab DAMO Academy, Alibaba Group U.S. Bellevue, WA, USA

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

AI agents are becoming increasingly capable of generating scientific code, but generating code is not the same as improving the algorithms behind it. For numerical solvers, execution feedback can expose poor performance, but rarely reveals its underlying cause and how to address it. We introduce Auto-Diagnosis and Skill Discovery (ADSD), a framework that links numerical diagnosis to reusable solver self-improvement. ADSD follows a diagnosis-first paradigm that first explains why a solver performs poorly, then uses this diagnosis to guide the discovery of appropriate numerical methods. The resulting knowledge is packaged into reusable solver skills, turning solver improvement from trial-and-error editing into a structured process of diagnosis, discovery, and implementation. Across four challenging numerical domains--power flow equation, AC optimal power flow control, stiff ordinary differential equations, and heterogeneous diffusion PDEs--ADSD consistently improves solver accuracy, robustness, and efficiency. On GOC-500 power flow, for example, ADSD reduces mean solver error by nearly 71×71\times, with improvements further transferring to unseen grid topologies and operating regimes.

Figures & tables

Figure 1: Overview of ADSD. Auto-diagnosis combines execution feedback from the frozen baseline solver N0N_{0} on public test cases with literature-derived diagnostic skills to identify numerical failure patterns. Skill discovery uses verified hypotheses to retrieve candidate methods and complete their implementations. The resulting reusable solver skills are supplied, together with N0N_{0} , to a fresh agent to generate the improved solver BB .
Base ModelConfigurationIEEE-118GOC-500
Run time (↓)(\downarrow)Error (↓)(\downarrow)Run time (↓)(\downarrow)Error (↓)(\downarrow)
GPT-5.5N0N_{0} solver0.73±0.210.73\pm 0.219.1×10−10±2.0×10−99.1{\times}10^{-10}\pm 2.0{\times}10^{-9}0.51±0.140.51\pm 0.149.3×10−10±2.2×10−99.3{\times}10^{-10}\pm 2.2{\times}10^{-9}
+ ADSD0.55±0.150.55\pm 0.152.7×10−10±9.9×10−102.7{\times}10^{-10}\pm 9.9{\times}10^{-10}0.50±0.140.50\pm 0.141.6×10−10±7.4×10−101.6{\times}10^{-10}\pm 7.4{\times}10^{-10}
Claude-Sonnet-5N0N_{0} solver0.16±0.040.16\pm 0.041.1×10−9±2.1×10−91.1{\times}10^{-9}\pm 2.1{\times}10^{-9}0.52±0.260.52\pm 0.267.9×10−10±1.9×10−97.9{\times}10^{-10}\pm 1.9{\times}10^{-9}
+ ADSD0.15±0.040.15\pm 0.047.4×10−10±1.8×10−97.4{\times}10^{-10}\pm 1.8{\times}10^{-9}0.52±0.250.52\pm 0.251.4×10−10±8.4×10−101.4{\times}10^{-10}\pm 8.4{\times}10^{-10}
Qwen3.8-2.4TN0N_{0} solver0.57±0.020.57\pm 0.029.9×10−10±2.1×10−99.9{\times}10^{-10}\pm 2.1{\times}10^{-9}0.18±0.090.18\pm 0.091.0×10−9±2.1×10−91.0{\times}10^{-9}\pm 2.1{\times}10^{-9}
Table 1: Performance on the IEEE-118 and GOC-500 validation sets. Values are mean ±\pm standard deviation. “ N0N_{0} solver” stands for the baseline performance using agent’s inherent knowledge, and “+ ADSD” denotes the agent’s performance after training with ADSD pipeline.
Figure 2: ADSD performance on IEEE-118 and GOC-500 under (a) topology generalization from N2 to unseen N1 / N grids and (b) feasibility generalization from feasible ( F ) to close-to-infeasible ( IF ) cases. Over all cases, learned skills can be well generalized to different grid setups and difficulties. Detailed results for figures (a) and (b) are presented in Tables 6 and 7 , respectively.
Base ModelConfigurationMean error (↓)(\downarrow)Runtime (↓)(\downarrow)Pass rate (↑)(\uparrow)
GPT-5.5N0N_{0} solver4.8×10−7±6.5×10−74.8\times 10^{-7}\pm 6.5\times 10^{-7}0.80±0.220.80\pm 0.2297.82%97.82\%
+ ADSD4.0×10−7±6.4×10−74.0\times 10^{-7}\pm 6.4\times 10^{-7}0.74±0.240.74\pm 0.2499.26%99.26\%
Claude-Sonnet-5N0N_{0} solver2.5×10−6±5.5×10−62.5\times 10^{-6}\pm 5.5\times 10^{-6}0.62±0.110.62\pm 0.1178.78%78.78\%
+ ADSD1.9×10−6±5.4×10−61.9\times 10^{-6}\pm 5.4\times 10^{-6}0.42±0.090.42\pm 0.0981.42%81.42\%
Qwen3.8-2.4TN0N_{0} solver2.7×10−6±8.6×10−72.7\times 10^{-6}\pm 8.6\times 10^{-7}0.26±0.040.26\pm 0.0414.42%14.42\%
+ ADSD1.2×10−6±8.2×10−71.2\times 10^{-6}\pm 8.2\times 10^{-7}0.18±0.050.18\pm 0.0598.36%98.36\%
Table 2: Performance on 5,000 AC-OPF hidden evaluation cases. N0N_{0} denotes the original solver and ADSD denotes its skill-augmented solver. Mean error is calculated as ∣f−fref∣/max⁡(∣fref∣,1)|f-f_{\mathrm{ref}}|/\max(|f_{\mathrm{ref}}|,1) , across all cases that N0N_{0} and ADSD solver passed in common.
Base ModelConfigurationPass rate (↑)(\uparrow)Median error (↓)(\downarrow)Time (↓)(\downarrow)
GPT-5.5N0N_{0} solver70.1%70.1\%9.39×10−69.39\times 10^{-6}4.713±9.5704.713\pm 9.570
+ ADSD90.6%90.6\%3.93×10−73.93\times 10^{-7}3.451±6.7433.451\pm 6.743
Claude-Sonnet-5N0N_{0} solver94.0%94.0\%4.43×10−74.43\times 10^{-7}0.618±1.4570.618\pm 1.457
+ ADSD100.0%100.0\%5.04×10−85.04\times 10^{-8}0.496±0.7820.496\pm 0.782
Qwen3.8-2.4TN0N_{0} solver98.3%98.3\%1.30×10−81.30\times 10^{-8}0.961±1.6880.961\pm 1.688
+ ADSD99.2%99.2\%5.95×10−95.95\times 10^{-9}0.646±1.0830.646\pm 1.083
Table 3: Performance on held-out stiff-ODE cases from 72 source families with 117 models. N0N_{0} denotes the solver generated from the agent’s inherent knowledge, and ADSD denotes the fresh solver generated with the constructed skill packet. Error is the median trajectory error. Wall time usages are per-case mean ±\pm standard deviation across all attempts.
Base ModelConfigurationℓ2\ell_{2} error (↓)(\downarrow)ℓ∞\ell_{\infty} error (↓)(\downarrow)Time (↓)(\downarrow)Operator calls (↓)(\downarrow)
GPT-5.5N0N_{0} solver1.69×10−9±0.86×10−91.69\times 10^{-9}\pm 0.86\times 10^{-9}1.46×10−9±0.91×10−91.46\times 10^{-9}\pm 0.91\times 10^{-9}7.29±1.327.29\pm 1.32215.49±40.35215.49\pm 40.35
+ ADSD5.27×10−10±2.04×10−105.27\times 10^{-10}\pm 2.04\times 10^{-10}5.93×10−10±3.33×10−105.93\times 10^{-10}\pm 3.33\times 10^{-10}2.78±0.582.78\pm 0.5871.22±17.5871.22\pm 17.58
Claude-Sonnet-5N0N_{0} solver1.12×10−11±0.54×10−111.12\times 10^{-11}\pm 0.54\times 10^{-11}9.96×10−12±6.12×10−129.96\times 10^{-12}\pm 6.12\times 10^{-12}0.86±0.030.86\pm 0.03148.16±4.82148.16\pm 4.82
+ ADSD8.15×10−12±5.41×10−128.15\times 10^{-12}\pm 5.41\times 10^{-12}5.49×10−12±4.97×10−125.49\times 10^{-12}\pm 4.97\times 10^{-12}0.79±0.020.79\pm 0.02125.71±13.40125.71\pm 13.40
Qwen3.8-2.4TN0N_{0} solver6.49×10−9±3.53×10−96.49\times 10^{-9}\pm 3.53\times 10^{-9}4.52×10−9±2.83×10−94.52\times 10^{-9}\pm 2.83\times 10^{-9}0.70±0.090.70\pm 0.0986.81±29.6886.81\pm 29.68
+ ADSD3.55×10−10±2.58×10−103.55\times 10^{-10}\pm 2.58\times 10^{-10}3.72×10−10±2.79×10−103.72\times 10^{-10}\pm 2.79\times 10^{-10}0.64±0.020.64\pm 0.0217.57±10.0317.57\pm 10.03
Table 4: Performance on 2,400 held-out ConDiff cases. Errors are relative to the benchmark’s discrete reference solutions. All error, wall-time, and operator-call statistics are mean ±\pm population std across 7,200 runs per solver (three repetitions per case).
Figure 3: Comparison of the N0N_{0} solver, self improvement, and ADSD on (a) IEEE-118, (b) GOC-500, and (c) stiff ODEs. Bars show mean error for grid tasks and median trajectory error for ODEs. Panel (c) uses a logarithmic scale; lower is better throughout.
Figure 4: Mean error on GOC-500 across recursive self-improvement iterations for (a) GPT-5.5, (b) Claude-Sonnet-5, and (c) Qwen3.8-2.4T. Dashed lines indicate the N0N_{0} and ADSD baselines. Iteration 0 denotes the original solver; lower is better.
Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

SolverMultistepMethod 9FactorizationsLinear solvesRuntime (s)PF residual
N0N_{0}––5.1165.1160.7279.10×10−109.10\times 10^{-10}
B1B_{1}✓\checkmark–2.7558.2550.4949.86×10−109.86\times 10^{-10}
B2B_{2}–✓\checkmark4.0038.0060.6547.46×10−137.46\times 10^{-13}
B3B_{3}✓\checkmark✓\checkmark3.1147.1140.5382.76×10−102.76\times 10^{-10}
Appendix
Table 5: Solver-skill ablation on IEEE-118 validation set. Each BB variant is separately generated with mandatory integration of the indicated skills. Factorizations, linear solves, runtime, and residual are per-case means. The residual is the independently evaluated ∥FPF(x;ξ)∥∞\|F_{\mathrm{PF}}(x;\xi)\|_{\infty} .
Base ModelDatasetConfigurationIn-domain N2Generalization N1Generalization N
Time (↓)(\downarrow)Error (↓)(\downarrow)Time (↓)(\downarrow)Error (↓)(\downarrow)Time (↓)(\downarrow)Error (↓)(\downarrow)
GPT-5.5IEEE-118N0N_{0} solver0.73±0.210.73\pm 0.219.1×10−10±2.0×10−99.1{\times}10^{-10}\pm 2.0{\times}10^{-9}0.74±0.210.74\pm 0.217.6×10−10±1.8×10−97.6{\times}10^{-10}\pm 1.8{\times}10^{-9}0.75±0.200.75\pm 0.208.9×10−10±2.0×10−98.9{\times}10^{-10}\pm 2.0{\times}10^{-9}
+ ADSD0.55±0.150.55\pm 0.152.7×10−10±9.9×10−102.7{\times}10^{-10}\pm 9.9{\times}10^{-10}0.54±0.150.54\pm 0.152.1×10−10±9.1×10−102.1{\times}10^{-10}\pm 9.1{\times}10^{-10}0.53±0.140.53\pm 0.142.3×10−10±8.3×10−102.3{\times}10^{-10}\pm 8.3{\times}10^{-10}
GOC-500N0N_{0} solver0.51±0.140.51\pm 0.149.3×10−10±2.2×10−99.3{\times}10^{-10}\pm 2.2{\times}10^{-9}0.38±0.090.38\pm 0.091.2×10−9±2.5×10−91.2{\times}10^{-9}\pm 2.5{\times}10^{-9}0.37±0.090.37\pm 0.098.6×10−10±2.1×10−98.6{\times}10^{-10}\pm 2.1{\times}10^{-9}
+ ADSD0.50±0.140.50\pm 0.141.6×10−10±7.4×10−101.6{\times}10^{-10}\pm 7.4{\times}10^{-10}0.62±0.150.62\pm 0.151.1×10−10±5.4×10−101.1{\times}10^{-10}\pm 5.4{\times}10^{-10}0.60±0.150.60\pm 0.151.9×10−10±7.9×10−101.9{\times}10^{-10}\pm 7.9{\times}10^{-10}
Claude-Sonnet-5IEEE-118N0N_{0} solver0.16±0.040.16\pm 0.041.1×10−9±2.1×10−91.1{\times}10^{-9}\pm 2.1{\times}10^{-9}0.18±0.040.18\pm 0.041.1×10−9±1.9×10−91.1{\times}10^{-9}\pm 1.9{\times}10^{-9}0.13±0.010.13\pm 0.011.1×10−9±1.9×10−91.1{\times}10^{-9}\pm 1.9{\times}10^{-9}
Appendix
Table 6: Topology generalization from the feasible N2 -topology grid to previously unseen feasible N1 and full-grid N cases. Values are mean ±\pm std over 10,000 cases per generalization topology.
Base ModelDatasetConfigurationIn-domain FGeneralization IF
Time (↓)(\downarrow)Error (↓)(\downarrow)Time (↓)(\downarrow)Error (↓)(\downarrow)
GPT-5.5IEEE-118N0N_{0} solver0.73±0.210.73\pm 0.219.1×10−10±2.0×10−99.1{\times}10^{-10}\pm 2.0{\times}10^{-9}1.68±0.241.68\pm 0.241.1×10−9±2.1×10−91.1{\times}10^{-9}\pm 2.1{\times}10^{-9}
+ ADSD0.55±0.150.55\pm 0.152.7×10−10±9.9×10−102.7{\times}10^{-10}\pm 9.9{\times}10^{-10}1.25±0.211.25\pm 0.213.3×10−10±1.3×10−93.3{\times}10^{-10}\pm 1.3{\times}10^{-9}
GOC-500N0N_{0} solver0.51±0.140.51\pm 0.149.3×10−10±2.2×10−99.3{\times}10^{-10}\pm 2.2{\times}10^{-9}0.52±0.140.52\pm 0.141.1×10−9±2.0×10−91.1{\times}10^{-9}\pm 2.0{\times}10^{-9}
+ ADSD0.50±0.140.50\pm 0.141.6×10−10±7.4×10−101.6{\times}10^{-10}\pm 7.4{\times}10^{-10}0.53±0.140.53\pm 0.148.5×10−10±1.9×10−98.5{\times}10^{-10}\pm 1.9{\times}10^{-9}
Claude-Sonnet-5IEEE-118N0N_{0} solver0.16±0.040.16\pm 0.041.1×10−9±2.1×10−91.1{\times}10^{-9}\pm 2.1{\times}10^{-9}0.73±0.110.73\pm 0.111.2×10−9±2.2×10−91.2{\times}10^{-9}\pm 2.2{\times}10^{-9}
Appendix
Table 7: Feasibility generalization from the feasible training set to previously unseen close-to-infeasible grids. Values are mean ±\pm std over 5,000 sampled from close-to-infeasible testcases.
More in this topic

Agent Skill Learning

Agent skill learning develops reusable procedures and behaviors that help AI agents solve new tasks and adapt from experience.

See all 174 papers

Explore similar work

CardsList
Feb 19, 2026

AutoNumerics: An Autonomous, PDE-Agnostic Multi-Agent Pipeline for Scientific Computing

PDEs are central to scientific and engineering modeling, yet designing accurate numerical solvers typically requires substantial mathematical expertise and manual tuning. Recent neural network-based approaches improve flexibility but often demand high computational cost and suffer from limited interpretability. We introduce AutoNumerics, a multi-agent framework that autonomously designs, implements, debugs, and verifies numerical solvers for general PDEs directly from natural language descriptions. Unlike black-box neural solvers, our framework generates transparent solvers grounded in classical numerical analysis. We introduce a coarse-to-fine execution strategy and a residual-based self-verification mechanism. Experiments on 24 canonical and real-world PDE problems demonstrate that AutoNumerics achieves competitive or superior accuracy compared to existing neural and LLM-based baselines, and correctly selects numerical schemes based on PDE structural properties, suggesting its viability as an accessible paradigm for automated PDE solving.

Jianda Du, Youran Sun, Haizhao Yang

Department of Mathematics, University of Maryland, College Park, MD, USA · Department of Computer Science, University of Maryland, College Park, MD, USA

PDE SolvingAutomated Algorithm DiscoveryAI Agents for Scientific DiscoveryLLM-Based Program Synthesis
AskPDF
Jun 9, 2026

AutoPDE: Reliable Agentic PDE Solving via Explicitly Represented Solver Strategies

Numerical solvers for partial differential equations (PDEs) are core computational tools in science and engineering. Building reliable PDE solvers requires not only executable code, but a numerical solver strategy, a set of decisions about discretization, stabilization, solver configuration, and resolution control, that matches the PDE structure. Recent LLM-based coding agents have begun to reduce the programming burden by generating and debugging solver implementations. However, they typically move directly from a PDE problem to solver code, leaving the solver strategy implicit in implementation details. Feedback from a failed solve is therefore routed back to code edits rather than to the underlying strategy, so numerical decisions remain hard to check before code is generated and hard to revise using numerical evidence when it fails. To address this limitation, we propose AutoPDE, a code agent that maintains the solver strategy as an explicitly represented object throughout the solving process: an independent, inspectable object that is built before any code is written and can be revised, using numerical evidence, whenever a solve fails. AutoPDE builds and maintains this object in three stages, all drawing from a library of reusable PDE-solving skills: PDE analysis identifies the equation type and algebraic structure; numerical method selection chooses a numerical method that matches the analysis result and commits to a discretization, stabilization, and linear solver accordingly; and adaptive tuning runs low-cost pilot solves to calibrate resolution and tolerances under the prescribed accuracy and runtime budget. We evaluate AutoPDE on the PDE Agent Bench, where experimental results show that AutoPDE achieves a pass rate of 54.554.5%, improving over the strongest baseline by 14.214.2 percentage points.

Huanshuo Dong, Keyao Zhang, Hong Wang +6

University of Science and Technology of China · 2Zhejiang University · University of the Chinese Academy of Sciences +2

AI Coding AgentsPDE Solving
AskPDF
Dec 3, 2025

ATHENA: Agentic Team for Hierarchical Evolutionary Numerical Algorithms

Progress in computational science depends on complex numerical workflows that must faithfully encode physical laws, yet translating conceptual insight into reliable code remains a major bottleneck. Although large language models can generate isolated code fragments, they lack the structured reasoning required to design, verify, and iteratively refine complete scientific pipelines. Here we introduce ATHENA, an agentic framework explicitly designed to emulate scientific research modeled as a knowledge-driven contextual bandit process. Its core loop separates conceptual policy from numerical realization through expert-derived conceptual scaffolding, enabling principled diagnosis, reformulation, and repair of computational strategies. Across scientific computing and scientific machine learning tasks, ATHENA autonomously derives and correctly applies exact analytical solutions, constructs stable numerical solvers, diagnoses ill-posed formulations, and orchestrates hybrid symbolic-numeric workflows. Quantitatively, ATHENA matches and frequently surpasses the accuracy of expert-authored reference solutions reported in the literature on canonical benchmarks. By reframing computation as an object of agentic reasoning, our framework enables autonomous orchestration of heterogeneous algorithms across scientific domains.

Juan Diego Toscano, Daniel T. Chen, George Em Karniadakis

Division of Applied Mathematics, Brown University, Providence, 02912, RI, USA

Scientific Code GenerationScientific MLScientific Workflow Automation
AskPDF