Re4: Scientific Computing Agent with Rewriting, Resolution, Review and Revision

Ao Cheng; Lei Zhang; Guowei He

cs.AIAug 28, 2025

Re4: Scientific Computing Agent with Rewriting, Resolution, Review and Revision

Authors: Ao Cheng, Lei Zhang, Guowei He

Organizations: The State Key Laboratory of Nonlinear Mechanics, Institute of Mechanics, Chinese Academy of Sciences, Beijing 100190, China · School of Engineering Science, University of Chinese Academy of Sciences, Beijing 100049, China

Abstract

Large language models (LLMs) have demonstrated abilities to perform complex tasks in multiple domains, including mathematical and scientific reasoning. In this work, we construct a novel agent framework for solving representative problems in scientific computing. The proposed agent incorporates a "rewriting-resolution-review-revision" logical chain via three reasoning LLMs (the Consultant, Reviewer, and Programmer) in a collaborative and interactive manner. The Consultant module enables knowledge transfer to link problems to professional domain insights, thereby rewriting problem descriptions through text augmentation. The Programmer module generates and executes well-structured code to solve problems. The Reviewer module enables self-debugging and self-refinement through interactive feedback with code runtime outputs. Through the end-to-end review mechanism, the Programmer's executable code undergoes iterative revision. We comprehensively evaluate the framework on typical partial differential equations (PDEs), engineering-oriented flow past a cylinder, ill-conditioned linear systems, and data-driven physical analysis problems. Compared with a single-model approach, this collaborative framework significantly improves the bug-free code generation rate and reduces non-physical solutions, establishing a reliable framework for autonomous code generation from natural language descriptions. The review mechanism improved the average execution success (bug-free code and non-NaN solutions) rate of DeepSeek R1 from 59% to 82%, while ChatGPT 4.1-mini improved from 66% to 87% and Gemini-2.5 improved from 60% to 84%. Automatic code generation and review provide a promising workflow for scientific computing; analysis of solver-development trajectories further reveals recurring implementation difficulties in boundary treatment, array indexing, and the preservation of numerical constraints.

Figures & tables

	Application Scope	Reasoning	Expansion	Debugging	Refinement	Review	Single/Multiple LLMs
Non-reasoning LLMs	General NLP	$\times$	$\times$	$\times$	$\times$	$\times$	Single
Reasoning LLMs	Complex reasoning tasks	$\checkmark$	$\times$	$\times$	$\times$	$\times$	Single
CodePDE Agent	From-scratch code generation	$\checkmark$	$\times$	$\checkmark$	based on posterior error	$\times$	Single
PINNsAgent	NN architecture & hyperparameter tuning	$\checkmark$	$\times$	$\checkmark$	$\checkmark$	$\times$	Single
Madaan’s Agent	General NLP refinement	$\times$	$\times$	$\times$	$\checkmark$	$\times$	Single
CFDAgent	CFD simulation via external software	$\checkmark$	$\times$	$\times$	$\times$	$\times$	Multiple

Table 1: Comparison of application scopes and capabilities among existing LLM-based frameworks and the proposed Re4 agent.

Figure 1: Architecture and workflow of Re4. The three LLM-based modules are the Consultant (problem clarification and solution planning), Programmer (code generation and revision), and Reviewer (code and result assessment). Their interactions are organized into four stages: Rewriting, Resolution, Review, and Revision. The Programmer participates in both Resolution and Revision, with code execution handled by the Python execution tool.

Panel of Figure 2 — Figure 2: Summary of various equations in PDEbench.

Panel of Figure 3 — Figure 3: The overall average execution success rate of numerical algorithms employed by Programmers across all equations in the PDEbench.

Panel of Figure 4 — Figure 4: Boxplot of the average $L^{2}$ relative errors for all equations in the PDEbench. The annotated numbers in the figure denote the mean value of average relative errors.

PDE Systems

GPT-4.1-mini

Gemini-2.5-flash

Deepseek-R1

ans-0

rev-1

rev-2

Best

ans-0

rev-1

rev-2

Best

ans-0

rev-1

rev-2

Best

Burgers

5.8e-02

2.5e-02

2.3e-02

1.98e-02

4.1e-02

3.5e-02

3.1e-02

1.82e-02

7.0e-02

4.1e-02

3.0e-02

1.85e-02

Sod Shock

1.3e-01

6.0e-02

6.1e-02

3.60e-02

6.4e-02

7.0e-02

5.81e-02

1.7e-01

4.4e-02

4.6e-02

2.31e-02

Poisson

4.4e-02

2.6e-02

1.9e-02

9.90e-03

3.5e-02

2.4e-02

1.5e-02

1.18e-02

4.1e-02

2.6e-02

1.5e-02

7.15e-03

Helmholtz

4.9e-02

4.5e-02

3.7e-02

2.30e-02

3.4e-02

3.2e-02

2.6e-02

2.09e-02

4.5e-02

3.5e-02

2.8e-02

1.47e-02

Lid-Driven

4.2e-01

1.4e-01

5.7e-02

2.74e-03

2.7e-01

2.3e-01

9.9e-02

6.55e-03

3.0e-01

2.6e-01

1.8e-01

8.77e-03

Table 2: Summary of the relative

L^{2}

errors across all cases in PDEbench. For each Programmer, ans-0 , rev-1 , and rev-2 report the average errors across successful independent runs at each corresponding response stage. Underlined entries indicate a Programmer’s minimum average error, while bold entries indicate the overall minimum average error among all Programmers. The Best columns present the minimum error achieved from the best-of-

N

perspective at the final stage. The bold and italic entries denote the overall best accuracy achieved for each equation.

Panel of Figure 5 — Figure 5: Schematic of the Sod shock tube with initial conditions at $t=0$ .

Panel of Figure 6 — Figure 6: Proportional distribution of the spatial reconstruction schemes employed by Programmers. All Programmers selected the Harten-Lax-vanLeer-Contact (HLLC) Riemann solver during the sampling process.

Panel of Figure 7 — Figure 7: Proportional distribution of the time integration methods employed by Programmers.

Panel of Figure 8 — Figure 8: Schematic of the computational setup for the lid-driven cavity flow.

Panel of Figure 9 — Figure 9: Schematic of the computational setup for the unsteady NS equations with explicit internal forcing at $Re=100$ .

Figure 10: The runtime outputs of the executable code provided by Programmer Deepseek-R1 in the initial response ( ans-0 ), together with those in the Reviewer’s first ( rev-1 ) and second ( rev-2 ) interventions, respectively.

Panel of Figure 11 — Figure 11: Schematic of the computational domain and boundary conditions for the flow past a cylinder case at $Re=100$ .

Programmer	Stage / Configuration	Success Rate	Tokens (k)		Time (s)
Programmer	Stage / Configuration	Success Rate	in	out	Time (s)
GPT-5-mini	baseline (Programmer-Only)	0%	0.94	5.88	36.47
	ans-0 (with Consultant)	9.1% ( $\uparrow$ 9.1%)	4.02	6.82	51.51
	rev-1 (with Reviewer)	45.5% ( $\uparrow$ 36.4%)	8.15	7.27	48.62
	rev-2 (with Reviewer)	54.5% ( $\uparrow$ 45.4%)	8.68	7.93	48.87

Table 3: Ablation study and cost analysis on the flow past a cylinder task over multiple independent runs, detailing the execution success rates, average input and output token consumption (in thousands, k), and average Programmer response times (in seconds, s) at each reasoning stage. The arrows (

\uparrow

) denote the individual contributions of the Consultant and Reviewer modules to the execution success rate.

Figure 12: Stage-wise outcomes of two independent cylinder-flow simulations generated by Programmer GPT-5-mini. The panels show the velocity magnitude $|\vec{u}|$ at $t=20.0$ when execution succeeds, while “Failed” indicates that no numerical solution was produced. The columns correspond to the initial response ( ans-0 ) and two successive review–revision stages ( rev-1 and rev-2 ).

Figure 13: Visualization of the periodic vortex shedding process at various time instants, generated by the final refined code ( rev-2 ) from Programmer GPT-5-mini.

Figure 14: Schematic diagram of (a) variation of the 2-norm condition number with matrix dimension; (b)

L^{\infty}

error obtained by direct solution using the naive methods.

Panel of Figure 15 — Figure 15: Proportional distribution of executable code provided by Programmers across three different completion statuses. The total number of evaluated method implementations at each response stage ( ans-0 , rev-1 , rev-2 ) is as follows: GPT-4.1-mini (20, 30, 35), Gemini-2.5-flash (28, 34, 35), and Deepseek-R1 (27, 35, 46), respectively.

Figure 16: Schematic diagram of (a) vapor depression dynamics in laser–metal interaction; (b) turbulent Rayleigh–Bénard convection.

Panel of Figure 17 — Figure 17: The success rate of Programmers’ search algorithms in identifying dominant dimensionless quantities Ke.

Figure 18: Fitting curves of the keyhole aspect ratio

e^{*}

as a function of the discovered dimensionless number

\mathrm{Ke}^{-2}

derived by all Programmers in the final response ( rev-2 ). The red solid lines denote the power-law relationships obtained via linear regression between

\mathrm{Ke}^{-2}

and

e^{*}

in the log-log space.

Figure 19: The progressive refinement of the discovered dimensionless groups and their log-log linear fits with the Nusselt number (

Nu

) generated by Programmer GPT-4.1-mini across the ans-0 , rev-1 , and rev-2 stages. The red solid lines denote the power-law relationships obtained via linear regression. As the review–revision loop progresses, the selected candidate group changes and ultimately matches the classical Rayleigh number (

Ra

Appendix figures & tables11 assets

Supplementary material from the paper’s appendix.

Appendix

Panel of Figure 20 — Figure 20: Detailed results for the Sod shock tube problem. In the middle panel, (a) velocity $u$ , (b) density $\rho$ , and (c) pressure $p$ .

Panel of Figure 21 — Figure 21: Detailed results for the unsteady NS equation.

Panel of Figure 22 — Figure 22: Detailed results for the lid-driven cavity flow.

Panel of Figure 23 — Figure 23: Detailed results for Burgers’ equation.

Panel of Figure 24 — Figure 24: Detailed results for the Poisson equation.

Panel of Figure 25 — Figure 25: Detailed results for the Helmholtz equation.

Panel of Figure 26 — Figure 26: Proportional distribution of various algorithms employed by Programmers at different response stages. Percentages reflect the adoption frequency of each method across all independent runs. For each method, the bar is divided into three shaded segments based on completion status: the lightest shade ( $\blacksquare$ ) represents results containing NaNs, the medium shade ( $\blacksquare$ ) denotes results exceeding the $L^{\infty}$ threshold ( $10^{-2}$ ), and the darkest shade ( $\blacksquare$ ) indicates results below the $L^{\infty}$ threshold.

Panel of Figure 27 — Figure 27: Proportional distribution of specific numerical methods employed by Programmers across three completion statuses, serving as a detailed elaboration to the solution success rate. (a) Results Contain NaN; (b) Over $L^{\infty}$ threshold; (c) Below $L^{\infty}$ threshold.

Programmer	Methods	$L^{\infty}$ error w.r.t. matrix size $n$
Programmer	Methods	5	10	15	20	25
GPT-4.1-mini	Chol-Reg	3.08e-04	6.81e-04	9.66e-04	1.36e-03	1.36e-03
	LU-Reg	3.15e-04	8.77e-04	8.92e-04	1.38e-03	1.37e-03
	QR-Reg	3.20e-04	9.90e-04	1.18e-03	1.31e-03	1.62e-03
	Pre-CG	9.80e-12	8.79e-05	7.16e-05	4.32e-05	9.96e-05
	SVD	1.46e-11	7.32e-04	3.91e-03	7.81e-03	7.32e-04

Appendix

Table 5: The summary of

L^{\infty}

errors in solving linear systems with Hilbert matrices via various methods provided by each Programmer in the final response ( rev-2 ). The bold entries denote the minimum error achieved for each matrix size

n

Metric	CodePDE	Re4 (GPT)	Re4 (Gemini)	Re4 (R1)
Boundary Cond.	Periodic ( $x\in[0,1]$ )	Dirichlet ( $x\in[-1,1]$ )	Dirichlet ( $x\in[-1,1]$ )	Dirichlet ( $x\in[-1,1]$ )
Convection	2nd-order Central Diff. / Fourier Spectral	1st-order Upwind	1st-order Upwind	2nd-order MUSCL (Minmod)
Diffusion	2nd-order Central Diff. / Fourier Spectral	2nd-order Central Diff.	2nd-order Central Diff.	2nd-order Central Diff.
Time Integration	1st-order Explicit Euler / IMEX	Crank-Nicolson	Crank-Nicolson	IMEX SSP-RK3
Linear Solver	Standard tensor operations	Banded matrix solve ( solve_banded )	Sparse matrix solve ( spsolve )	LU pre-factorization ( splu )

Appendix

Table 6: Code-level algorithmic comparison of the generated 1-D Burgers’ solvers between CodePDE and different Programmer LLMs under Re4.

Task	Programmer	Stage	Success Rate	Tokens (k)		Time (s)
Task	Programmer	Stage	Success Rate	in	out	Time (s)
Burgers	GPT-4.1-mini	baseline	50.0%	0.51	1.59	18.85
		ans-0	67.0% ( $\uparrow$ 17.0%)	1.52	1.88	23.26
		rev-1	100% ( $\uparrow$ 33.0%)	2.60	1.97	25.01
		rev-2	100% ( $\uparrow$ 33.0%)	2.72	2.24	25.92
	Gemini-2.5-flash	baseline	50.0%	0.51	2.10	18.65

Appendix

Table 7: Comprehensive ablation study and cost analysis of the proposed Re4 framework across different reasoning stages for all nine evaluated tasks. The arrows denote the individual contributions of the Consultant (

\uparrow

) and Reviewer (

\uparrow

\downarrow

) modules to the success rate.

Scientific Code Generation

Scientific code generation develops systems that turn scientific questions, papers, and workflows into executable, scientifically valid programs.

See all 52 papers

SciWalker: Synthesizing Scientific Coding Problems with Operator Graphs and Execution Feedback
Sep 24, 2026Chenxi Li, Wenxuan Zeng, Yun Luo +4Execution-Guided Code Generation Scientific Code Generation
The Hong Kong University of Science and Technology · Shanghai AI Laboratory · Tsinghua University +1
ATHENA: Agentic Team for Hierarchical Evolutionary Numerical Algorithms
Dec 3, 2025Juan Diego Toscano, Daniel T. Chen, George Em KarniadakisScientific Code Generation Scientific ML
Division of Applied Mathematics, Brown University, Providence, 02912, RI, USA
No Test Cases, No Problem: Distillation-Driven Code Generation for Scientific Workflows
Apr 25, 2026Siddeshwar Raghavan, Tanwi MallickMulti-Agent LLM Systems Code Generation
Purdue University · Electrical and Computer Engineering · West Lafayette, USA +3

Sep 24, 2026J/K move · Enter open · S save

SciWalker: Synthesizing Scientific Coding Problems with Operator Graphs and Execution Feedback

Chenxi Li, Wenxuan Zeng, Yun Luo, Fangchen Yu+3

The Hong Kong University of Science and Technology · Shanghai AI Laboratory · Tsinghua University+1

Execution-Guided Code Generation Scientific Code Generation Synthetic Benchmark Generation Synthetic Data Generation for Language Models

Open paper AskPDF arXiv

Improving the scientific coding capabilities of large language models (LLMs) requires high-quality training data. However, such data remain scarce because manually authoring realistic problems is costly and time-consuming, while systematically covering diverse scientific domains and algorithmic combinations remains challenging. To address this, we introduce SciWalker, a framework for synthesizing scientific coding problems through operator-chain sampling and execution feedback. The framework combines scientific library interfaces with operation modes to instantiate operators, organizes them into operator graphs, and samples operator chains as computational workflow cues. Guided by these cues, we adopt LLMs to generate scientifically grounded problem statements, reference solutions, and tests, with failed generations iteratively repaired using execution feedback. By combining structured workflow composition with verification and quality review, SciWalker enables scalable task generation while promoting scientific grounding, computational diversity, and executability. Using this framework, we construct 8,178 high-quality problems spanning 5 scientific domains and 32 subdomains. To evaluate their training utility, we conduct reinforcement learning on Qwen3.5-9B using the GSPO algorithm. This training improves SciCode subproblem accuracy by 9.9 percentage points, from 29.3% to 39.2%, with gains across scientific code generation, code repair, and reasoning benchmarks. The code for SciWalker is available at https://github.com/lichenx1/SciWalker.

Abstract

Scientific Code Generation

Explore similar work

SciWalker: Synthesizing Scientific Coding Problems with Operator Graphs and Execution Feedback

ATHENA: Agentic Team for Hierarchical Evolutionary Numerical Algorithms

No Test Cases, No Problem: Distillation-Driven Code Generation for Scientific Workflows