Re4: Scientific Computing Agent with Rewriting, Resolution, Review and Revision

Ao Cheng; Lei Zhang; Guowei He

cs.AIAug 28, 2025

Re4: Scientific Computing Agent with Rewriting, Resolution, Review and Revision

Authors: Ao Cheng, Lei Zhang, Guowei He

Organizations: The State Key Laboratory of Nonlinear Mechanics, Institute of Mechanics, Chinese Academy of Sciences, Beijing 100190, China · School of Engineering Science, University of Chinese Academy of Sciences, Beijing 100049, China

Abstract

Large language models (LLMs) have demonstrated abilities to perform complex tasks in multiple domains, including mathematical and scientific reasoning. In this work, we construct a novel agent framework for solving representative problems in scientific computing. The proposed agent incorporates a "rewriting-resolution-review-revision" logical chain via three reasoning LLMs (the Consultant, Reviewer, and Programmer) in a collaborative and interactive manner. The Consultant module enables knowledge transfer to link problems to professional domain insights, thereby rewriting problem descriptions through text augmentation. The Programmer module generates and executes well-structured code to solve problems. The Reviewer module enables self-debugging and self-refinement through interactive feedback with code runtime outputs. Through the end-to-end review mechanism, the Programmer's executable code undergoes iterative revision. We comprehensively evaluate the framework on typical partial differential equations (PDEs), engineering-oriented flow past a cylinder, ill-conditioned linear systems, and data-driven physical analysis problems. Compared with a single-model approach, this collaborative framework significantly improves the bug-free code generation rate and reduces non-physical solutions, establishing a reliable framework for autonomous code generation from natural language descriptions. The review mechanism improved the average execution success (bug-free code and non-NaN solutions) rate of DeepSeek R1 from 59% to 82%, while ChatGPT 4.1-mini improved from 66% to 87% and Gemini-2.5 improved from 60% to 84%. Automatic code generation and review provide a promising workflow for scientific computing; analysis of solver-development trajectories further reveals recurring implementation difficulties in boundary treatment, array indexing, and the preservation of numerical constraints.

Figures & tables

	Application Scope	Reasoning	Expansion	Debugging	Refinement	Review	Single/Multiple LLMs
Non-reasoning LLMs	General NLP	$\times$	$\times$	$\times$	$\times$	$\times$	Single
Reasoning LLMs	Complex reasoning tasks	$\checkmark$	$\times$	$\times$	$\times$	$\times$	Single
CodePDE Agent	From-scratch code generation	$\checkmark$	$\times$	$\checkmark$	based on posterior error	$\times$	Single
PINNsAgent	NN architecture & hyperparameter tuning	$\checkmark$	$\times$	$\checkmark$	$\checkmark$	$\times$	Single
Madaan’s Agent	General NLP refinement	$\times$	$\times$	$\times$	$\checkmark$	$\times$	Single
CFDAgent	CFD simulation via external software	$\checkmark$	$\times$	$\times$	$\times$	$\times$	Multiple

Table 1: Comparison of application scopes and capabilities among existing LLM-based frameworks and the proposed Re4 agent.

Figure 1: Architecture and workflow of Re4. The three LLM-based modules are the Consultant (problem clarification and solution planning), Programmer (code generation and revision), and Reviewer (code and result assessment). Their interactions are organized into four stages: Rewriting, Resolution, Review, and Revision. The Programmer participates in both Resolution and Revision, with code execution handled by the Python execution tool.

Panel of Figure 2 — Figure 2: Summary of various equations in PDEbench.

Panel of Figure 3 — Figure 3: The overall average execution success rate of numerical algorithms employed by Programmers across all equations in the PDEbench.

Panel of Figure 4 — Figure 4: Boxplot of the average $L^{2}$ relative errors for all equations in the PDEbench. The annotated numbers in the figure denote the mean value of average relative errors.

PDE Systems

GPT-4.1-mini

Gemini-2.5-flash

Deepseek-R1

ans-0

rev-1

rev-2

Best

ans-0

rev-1

rev-2

Best

ans-0

rev-1

rev-2

Best

Burgers

5.8e-02

2.5e-02

2.3e-02

1.98e-02

4.1e-02

3.5e-02

3.1e-02

1.82e-02

7.0e-02

4.1e-02

3.0e-02

1.85e-02

Sod Shock

1.3e-01

6.0e-02

6.1e-02

3.60e-02

6.4e-02

7.0e-02

5.81e-02

1.7e-01

4.4e-02

4.6e-02

2.31e-02

Poisson

4.4e-02

2.6e-02

1.9e-02

9.90e-03

3.5e-02

2.4e-02

1.5e-02

1.18e-02

4.1e-02

2.6e-02

1.5e-02

7.15e-03

Helmholtz

4.9e-02

4.5e-02

3.7e-02

2.30e-02

3.4e-02

3.2e-02

2.6e-02

2.09e-02

4.5e-02

3.5e-02

2.8e-02

1.47e-02

Lid-Driven

4.2e-01

1.4e-01

5.7e-02

2.74e-03

2.7e-01

2.3e-01

9.9e-02

6.55e-03

3.0e-01

2.6e-01

1.8e-01

8.77e-03

Table 2: Summary of the relative

L^{2}

errors across all cases in PDEbench. For each Programmer, ans-0 , rev-1 , and rev-2 report the average errors across successful independent runs at each corresponding response stage. Underlined entries indicate a Programmer’s minimum average error, while bold entries indicate the overall minimum average error among all Programmers. The Best columns present the minimum error achieved from the best-of-

N

perspective at the final stage. The bold and italic entries denote the overall best accuracy achieved for each equation.

Panel of Figure 5 — Figure 5: Schematic of the Sod shock tube with initial conditions at $t=0$ .

Panel of Figure 6 — Figure 6: Proportional distribution of the spatial reconstruction schemes employed by Programmers. All Programmers selected the Harten-Lax-vanLeer-Contact (HLLC) Riemann solver during the sampling process.

Panel of Figure 7 — Figure 7: Proportional distribution of the time integration methods employed by Programmers.

Panel of Figure 8 — Figure 8: Schematic of the computational setup for the lid-driven cavity flow.

Panel of Figure 9 — Figure 9: Schematic of the computational setup for the unsteady NS equations with explicit internal forcing at $Re=100$ .

Figure 10: The runtime outputs of the executable code provided by Programmer Deepseek-R1 in the initial response ( ans-0 ), together with those in the Reviewer’s first ( rev-1 ) and second ( rev-2 ) interventions, respectively.

Panel of Figure 11 — Figure 11: Schematic of the computational domain and boundary conditions for the flow past a cylinder case at $Re=100$ .

Programmer	Stage / Configuration	Success Rate	Tokens (k)		Time (s)
Programmer	Stage / Configuration	Success Rate	in	out	Time (s)
GPT-5-mini	baseline (Programmer-Only)	0%	0.94	5.88	36.47
	ans-0 (with Consultant)	9.1% ( $\uparrow$ 9.1%)	4.02	6.82	51.51
	rev-1 (with Reviewer)	45.5% ( $\uparrow$ 36.4%)	8.15	7.27	48.62
	rev-2 (with Reviewer)	54.5% ( $\uparrow$ 45.4%)	8.68	7.93	48.87

Table 3: Ablation study and cost analysis on the flow past a cylinder task over multiple independent runs, detailing the execution success rates, average input and output token consumption (in thousands, k), and average Programmer response times (in seconds, s) at each reasoning stage. The arrows (

\uparrow

) denote the individual contributions of the Consultant and Reviewer modules to the execution success rate.

Figure 12: Stage-wise outcomes of two independent cylinder-flow simulations generated by Programmer GPT-5-mini. The panels show the velocity magnitude $|\vec{u}|$ at $t=20.0$ when execution succeeds, while “Failed” indicates that no numerical solution was produced. The columns correspond to the initial response ( ans-0 ) and two successive review–revision stages ( rev-1 and rev-2 ).

Figure 13: Visualization of the periodic vortex shedding process at various time instants, generated by the final refined code ( rev-2 ) from Programmer GPT-5-mini.

Figure 14: Schematic diagram of (a) variation of the 2-norm condition number with matrix dimension; (b)

L^{\infty}

error obtained by direct solution using the naive methods.

Panel of Figure 15 — Figure 15: Proportional distribution of executable code provided by Programmers across three different completion statuses. The total number of evaluated method implementations at each response stage ( ans-0 , rev-1 , rev-2 ) is as follows: GPT-4.1-mini (20, 30, 35), Gemini-2.5-flash (28, 34, 35), and Deepseek-R1 (27, 35, 46), respectively.

Figure 16: Schematic diagram of (a) vapor depression dynamics in laser–metal interaction; (b) turbulent Rayleigh–Bénard convection.

Panel of Figure 17 — Figure 17: The success rate of Programmers’ search algorithms in identifying dominant dimensionless quantities Ke.

Figure 18: Fitting curves of the keyhole aspect ratio

e^{*}

as a function of the discovered dimensionless number

\mathrm{Ke}^{-2}

derived by all Programmers in the final response ( rev-2 ). The red solid lines denote the power-law relationships obtained via linear regression between

\mathrm{Ke}^{-2}

and

e^{*}

in the log-log space.

Figure 19: The progressive refinement of the discovered dimensionless groups and their log-log linear fits with the Nusselt number (

Nu

) generated by Programmer GPT-4.1-mini across the ans-0 , rev-1 , and rev-2 stages. The red solid lines denote the power-law relationships obtained via linear regression. As the review–revision loop progresses, the selected candidate group changes and ultimately matches the classical Rayleigh number (

Ra

Appendix figures & tables11 assets

Supplementary material from the paper’s appendix.

Appendix

Panel of Figure 20 — Figure 20: Detailed results for the Sod shock tube problem. In the middle panel, (a) velocity $u$ , (b) density $\rho$ , and (c) pressure $p$ .

Panel of Figure 21 — Figure 21: Detailed results for the unsteady NS equation.

Panel of Figure 22 — Figure 22: Detailed results for the lid-driven cavity flow.

Panel of Figure 23 — Figure 23: Detailed results for Burgers’ equation.

Panel of Figure 24 — Figure 24: Detailed results for the Poisson equation.

Panel of Figure 25 — Figure 25: Detailed results for the Helmholtz equation.

Panel of Figure 26 — Figure 26: Proportional distribution of various algorithms employed by Programmers at different response stages. Percentages reflect the adoption frequency of each method across all independent runs. For each method, the bar is divided into three shaded segments based on completion status: the lightest shade ( $\blacksquare$ ) represents results containing NaNs, the medium shade ( $\blacksquare$ ) denotes results exceeding the $L^{\infty}$ threshold ( $10^{-2}$ ), and the darkest shade ( $\blacksquare$ ) indicates results below the $L^{\infty}$ threshold.

Panel of Figure 27 — Figure 27: Proportional distribution of specific numerical methods employed by Programmers across three completion statuses, serving as a detailed elaboration to the solution success rate. (a) Results Contain NaN; (b) Over $L^{\infty}$ threshold; (c) Below $L^{\infty}$ threshold.

Programmer	Methods	$L^{\infty}$ error w.r.t. matrix size $n$
Programmer	Methods	5	10	15	20	25
GPT-4.1-mini	Chol-Reg	3.08e-04	6.81e-04	9.66e-04	1.36e-03	1.36e-03
	LU-Reg	3.15e-04	8.77e-04	8.92e-04	1.38e-03	1.37e-03
	QR-Reg	3.20e-04	9.90e-04	1.18e-03	1.31e-03	1.62e-03
	Pre-CG	9.80e-12	8.79e-05	7.16e-05	4.32e-05	9.96e-05
	SVD	1.46e-11	7.32e-04	3.91e-03	7.81e-03	7.32e-04

Appendix

Table 5: The summary of

L^{\infty}

errors in solving linear systems with Hilbert matrices via various methods provided by each Programmer in the final response ( rev-2 ). The bold entries denote the minimum error achieved for each matrix size

n

Metric	CodePDE	Re4 (GPT)	Re4 (Gemini)	Re4 (R1)
Boundary Cond.	Periodic ( $x\in[0,1]$ )	Dirichlet ( $x\in[-1,1]$ )	Dirichlet ( $x\in[-1,1]$ )	Dirichlet ( $x\in[-1,1]$ )
Convection	2nd-order Central Diff. / Fourier Spectral	1st-order Upwind	1st-order Upwind	2nd-order MUSCL (Minmod)
Diffusion	2nd-order Central Diff. / Fourier Spectral	2nd-order Central Diff.	2nd-order Central Diff.	2nd-order Central Diff.
Time Integration	1st-order Explicit Euler / IMEX	Crank-Nicolson	Crank-Nicolson	IMEX SSP-RK3
Linear Solver	Standard tensor operations	Banded matrix solve ( solve_banded )	Sparse matrix solve ( spsolve )	LU pre-factorization ( splu )

Appendix

Table 6: Code-level algorithmic comparison of the generated 1-D Burgers’ solvers between CodePDE and different Programmer LLMs under Re4.

Task	Programmer	Stage	Success Rate	Tokens (k)		Time (s)
Task	Programmer	Stage	Success Rate	in	out	Time (s)
Burgers	GPT-4.1-mini	baseline	50.0%	0.51	1.59	18.85
		ans-0	67.0% ( $\uparrow$ 17.0%)	1.52	1.88	23.26
		rev-1	100% ( $\uparrow$ 33.0%)	2.60	1.97	25.01
		rev-2	100% ( $\uparrow$ 33.0%)	2.72	2.24	25.92
	Gemini-2.5-flash	baseline	50.0%	0.51	2.10	18.65

Appendix

Table 7: Comprehensive ablation study and cost analysis of the proposed Re4 framework across different reasoning stages for all nine evaluated tasks. The arrows denote the individual contributions of the Consultant (

\uparrow

) and Reviewer (

\uparrow

\downarrow

) modules to the success rate.

Scientific Code Generation

Scientific code generation develops systems that turn scientific questions, papers, and workflows into executable, scientifically valid programs.

See all 52 papers

Sep 24, 2026

SciWalker: Synthesizing Scientific Coding Problems with Operator Graphs and Execution Feedback

Improving the scientific coding capabilities of large language models (LLMs) requires high-quality training data. However, such data remain scarce because manually authoring realistic problems is costly and time-consuming, while systematically covering diverse scientific domains and algorithmic combinations remains challenging. To address this, we introduce SciWalker, a framework for synthesizing scientific coding problems through operator-chain sampling and execution feedback. The framework combines scientific library interfaces with operation modes to instantiate operators, organizes them into operator graphs, and samples operator chains as computational workflow cues. Guided by these cues, we adopt LLMs to generate scientifically grounded problem statements, reference solutions, and tests, with failed generations iteratively repaired using execution feedback. By combining structured workflow composition with verification and quality review, SciWalker enables scalable task generation while promoting scientific grounding, computational diversity, and executability. Using this framework, we construct 8,178 high-quality problems spanning 5 scientific domains and 32 subdomains. To evaluate their training utility, we conduct reinforcement learning on Qwen3.5-9B using the GSPO algorithm. This training improves SciCode subproblem accuracy by 9.9 percentage points, from 29.3% to 39.2%, with gains across scientific code generation, code repair, and reasoning benchmarks. The code for SciWalker is available at https://github.com/lichenx1/SciWalker.

Chenxi Li, Wenxuan Zeng, Yun Luo +4

The Hong Kong University of Science and Technology · Shanghai AI Laboratory · Tsinghua University +1

Execution-Guided Code Generation Scientific Code Generation Synthetic Benchmark Generation Synthetic Data Generation for Language Models

Ask PDF

Dec 3, 2025

ATHENA: Agentic Team for Hierarchical Evolutionary Numerical Algorithms

Progress in computational science depends on complex numerical workflows that must faithfully encode physical laws, yet translating conceptual insight into reliable code remains a major bottleneck. Although large language models can generate isolated code fragments, they lack the structured reasoning required to design, verify, and iteratively refine complete scientific pipelines. Here we introduce ATHENA, an agentic framework explicitly designed to emulate scientific research modeled as a knowledge-driven contextual bandit process. Its core loop separates conceptual policy from numerical realization through expert-derived conceptual scaffolding, enabling principled diagnosis, reformulation, and repair of computational strategies. Across scientific computing and scientific machine learning tasks, ATHENA autonomously derives and correctly applies exact analytical solutions, constructs stable numerical solvers, diagnoses ill-posed formulations, and orchestrates hybrid symbolic-numeric workflows. Quantitatively, ATHENA matches and frequently surpasses the accuracy of expert-authored reference solutions reported in the literature on canonical benchmarks. By reframing computation as an object of agentic reasoning, our framework enables autonomous orchestration of heterogeneous algorithms across scientific domains.

Juan Diego Toscano, Daniel T. Chen, George Em Karniadakis

Division of Applied Mathematics, Brown University, Providence, 02912, RI, USA

Scientific Code Generation Scientific ML Scientific Workflow Automation

Ask PDF

Apr 25, 2026

No Test Cases, No Problem: Distillation-Driven Code Generation for Scientific Workflows

Existing multi-agent Large Language Model (LLM) frameworks for code generation typically use execution feedback and improve iteratively using Input/Output (I/O) test cases. However, this does not work for scientific workflows, where I/O test cases do not exist, and generating them requires solving the very problem at hand. To address this, we introduce MOSAIC, a training-free multi-agent framework for scientific code generation without I/O supervision. Instead of execution feedback, MOSAIC employs a student-teacher knowledge distillation framework that grounds generation through domain-specific examples and structured problem decomposition. To further mitigate hallucinations across chained subproblems, we introduce a Consolidated Context Window (CCW) for maintaining consistent reasoning across agents. Experiments on the SciCode benchmark show that MOSAIC improves accuracy, executability, and numerical precision over existing approaches while relying on lightweight models.

Siddeshwar Raghavan, Tanwi Mallick

Purdue University · Electrical and Computer Engineering · West Lafayette, USA +3

Multi-Agent LLM Systems Code Generation Scientific Code Generation Scientific Workflow Automation

Ask PDF

Abstract

Scientific Code Generation

Explore similar work

SciWalker: Synthesizing Scientific Coding Problems with Operator Graphs and Execution Feedback

ATHENA: Agentic Team for Hierarchical Evolutionary Numerical Algorithms

No Test Cases, No Problem: Distillation-Driven Code Generation for Scientific Workflows