Agent-Guided Process Verification for Mathematical Self-Correction
Organizations: Gaoling School of Artificial Intelligence, Renmin University of China · National Engineering Research Center of Educational Big Data, Central China Normal University
Abstract
Large language models have made substantial progress in mathematical reasoning, yet in complex multi-step reasoning, an error at any step may propagate along an incorrect path, and the resulting solution may omit necessary branches. Existing natural-language reflection methods often struggle to locate critical errors precisely, while outcome verification typically checks only whether the available candidate answers satisfy the problem conditions and can therefore overlook necessary branches. To address these limitations, we propose Agent-Guided Process Verification (AGPV) for mathematical self-correction. AGPV decomposes the current solution into an ordered sequence of key claims, verifies them individually while invoking a shared mathematical toolbox when needed, and stops upon detecting the first error. A summarization agent converts the failure evidence into a diagnosis that guides a correction agent to re-derive the subsequent solution from that point; AGPV then adapts the next verification iteration to the revised solution. AGPV also verifies solution completeness. We further prove that AGPV guarantees solution correctness and completeness when key claims provide sufficient coverage and local verification is reliable, provided that all key claims pass verification. The implementation compresses long working contexts and reuses successful tool results within each problem. We evaluate AGPV on five challenging mathematical reasoning datasets with four models from DeepSeek, Qwen, and GPT. AGPV achieves an average accuracy of 63.8%, outperforming the strongest among the representative baselines evaluated (50.3%) by 13.5 percentage points, with higher repair coverage than the programmatic-verification baselines.
Figures & tables
| Method | DeepSeek V4.1-Flash | DeepSeek V4-Pro (0813) | Qwen3.6 35B-A3B-FP8 | GPT-5.6 Luna | Mean |
|---|---|---|---|---|---|
| Initial | 52.9 | 38.8 | 53.5 | 23.5 | 42.2 |
| Reflex | \text{60.0}_{\,\text{{\color[rgb]{0.1563,0.4766,0.3281}+7.1}}} | \text{37.6}_{\,\text{{\color[rgb]{0.7148,0.2578,0.2578}-1.2}}} | \text{54.7}_{\,\text{{\color[rgb]{0.1563,0.4766,0.3281}+1.2}}} | \text{31.2}_{\,\text{{\color[rgb]{0.1563,0.4766,0.3281}+7.6}}} | \text{45.9}_{\,\text{{\color[rgb]{0.1563,0.4766,0.3281}+3.7}}} |
| Self-Refine | \text{61.8}_{\,\text{{\color[rgb]{0.1563,0.4766,0.3281}+8.8}}} | \text{51.2}_{\,\text{{\color[rgb]{0.1563,0.4766,0.3281}+12.4}}} | \text{57.6}_{\,\text{{\color[rgb]{0.1563,0.4766,0.3281}+4.1}}} | \text{30.6}_{\,\text{{\color[rgb]{0.1563,0.4766,0.3281}+7.1}}} | \text{50.3}_{\,\text{{\color[rgb]{0.1563,0.4766,0.3281}+8.1}}} |
| Self-Reflection | \text{55.3}_{\,\text{{\color[rgb]{0.1563,0.4766,0.3281}+2.4}}} | \text{40.0}_{\,\text{{\color[rgb]{0.1563,0.4766,0.3281}+1.2}}} | \text{57.1}_{\,\text{{\color[rgb]{0.1563,0.4766,0.3281}+3.5}}} | \text{32.9}_{\,\text{{\color[rgb]{0.1563,0.4766,0.3281}+9.4}}} | \text{46.3}_{\,\text{{\color[rgb]{0.1563,0.4766,0.3281}+4.1}}} |
| CheckList | \text{55.3}_{\,\text{{\color[rgb]{0.1563,0.4766,0.3281}+2.4}}} | \text{41.2}_{\,\text{{\color[rgb]{0.1563,0.4766,0.3281}+2.4}}} | \text{57.1}_{\,\text{{\color[rgb]{0.1563,0.4766,0.3281}+3.5}}} | \text{30.0}_{\,\text{{\color[rgb]{0.1563,0.4766,0.3281}+6.5}}} | \text{45.9}_{\,\text{{\color[rgb]{0.1563,0.4766,0.3281}+3.7}}} |
| CoT-Tool | \text{54.1}_{\,\text{{\color[rgb]{0.1563,0.4766,0.3281}+1.2}}} | \text{40.0}_{\,\text{{\color[rgb]{0.1563,0.4766,0.3281}+1.2}}} | \text{41.8}_{\,\text{{\color[rgb]{0.7148,0.2578,0.2578}-11.8}}} | \text{22.4}_{\,\text{{\color[rgb]{0.7148,0.2578,0.2578}-1.2}}} | \text{39.6}_{\,\text{{\color[rgb]{0.7148,0.2578,0.2578}-2.6}}} |
Appendix figures & tables19 assets
Supplementary material from the paper’s appendix.
Appendix
| Agent | Core instruction (abridged) |
|---|---|
| Generate key claims and a verification plan | Extract the key claims from the current solution and plan the checks in reasoning order. Include the problem’s requirements for a complete answer. |
| Verify key claims | Check each claim according to the plan, using the mathematical toolbox as needed. Stop at the first failed claim and record the evidence. |
| Summarize evidence of failure | Use the verification record to identify the failed step, explain why it failed, and indicate where correction should begin. |
| Revise the solution | Use the feedback to retain the valid reasoning, redo the derivation from the error onward, and provide a complete solution. |
| Update the key claims and verification plan | When the answer stays the same after correction, recheck the original plan against the revised solution and verification record. Retain applicable checks, add missing ones, and adjust those that no longer match the reasoning. |
| Shared mathematical toolbox | Turn mathematical requests from the other agents into executable Python code, perform symbolic solving, enumeration, or constraint checks, and return the results. |
| Model setting | API identifier / checkpoint | Reasoning configuration |
|---|---|---|
| DeepSeek-V4.1-Flash | deepseek-flash | Thinking disabled |
| DeepSeek-V4-Pro (0813) | deepseek-v4-pro | Thinking disabled |
| Qwen3.6-35B-A3B-FP8 | Qwen/Qwen3.6-35B-A3B-FP8 | enable_thinking=False |
| GPT-5.6-Luna | gpt-5.6-luna | reasoning_effort=none |
| Method | Initial response | Main-experiment budget and execution mechanism |
|---|---|---|
| ProgCo | Shared | Up to three correction turns; program-driven verification and correction. |
| ProgCo-Py | Shared | Up to three correction turns; Python-tool feedback enabled within the ProgCo workflow. |
| Reflex | Shared | Up to three correction turns; one reflection-and-rewrite step per turn. |
| Self-Refine | Shared | Up to three correction turns; feedback is generated before each feedback-guided revision. |
| Self-Reflection | Shared | Up to three correction turns; one self-reflective correction per turn. |
| CheckList | Shared | One checklist is generated first, followed by up to three checklist-guided correction turns. |
| Dataset | Short | #Problems | Notes |
|---|---|---|---|
| AIME 2024 | A24 | 30 | competition math; integer answers |
| AIME 2025 | A25 | 30 | competition math; integer answers |
| BRUMO 2025 | B25 | 30 | hard contest-style reasoning |
| HMMT Feb. 2025 | HMM | 30 | contest math; mixed topics |
| AMO Bench | AMO | 50 | advanced math reasoning; mixed answer forms |
| DeepSeek-V4.1-Flash | DeepSeek-V4-Pro (0813) | |||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Method | A24 | A25 | B25 | HMM | AMO | Avg | A24 | A25 | B25 | HMM | AMO | Avg |
| Initial | 73.3 | 73.3 | 86.7 | 53.3 | 8.0 | 52.9 | 63.3 | 46.7 | 66.7 | 26.7 | 10.0 | 38.8 |
| Reflex | 83.3 | 80.0 | 90.0 | 60.0 | 16.0 | \text{60.0}_{\,\text{{\color[rgb]{0.1563,0.4766,0.3281}+7.1}}} | 63.3 | 43.3 | 63.3 | 30.0 | 8.0 | \text{37.6}_{\,\text{{\color[rgb]{0.7148,0.2578,0.2578}-1.2}}} |
| Self-Refine | 86.7 | 83.3 | 83.3 | 70.0 | 16.0 | \text{61.8}_{\,\text{{\color[rgb]{0.1563,0.4766,0.3281}+8.8}}} | 70.0 | 60.0 | 86.7 | 56.7 | 10.0 | \text{51.2}_{\,\text{{\color[rgb]{0.1563,0.4766,0.3281}+12.4}}} |
| Self-Reflection | 80.0 | 70.0 | 90.0 | 60.0 | 8.0 | \text{55.3}_{\,\text{{\color[rgb]{0.1563,0.4766,0.3281}+2.4}}} | 76.7 | 46.7 | 66.7 | 26.7 | 6.0 | \text{40.0}_{\,\text{{\color[rgb]{0.1563,0.4766,0.3281}+1.2}}} |
| CheckList | 76.7 | 76.7 | 86.7 | 53.3 | 12.0 | \text{55.3}_{\,\text{{\color[rgb]{0.1563,0.4766,0.3281}+2.4}}} | 66.7 | 46.7 | 66.7 | 33.3 | 12.0 | \text{41.2}_{\,\text{{\color[rgb]{0.1563,0.4766,0.3281}+2.4}}} |
| Qwen3.6-35B-A3B-FP8 | GPT-5.6-Luna | |||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Method | A24 | A25 | B25 | HMM | AMO | Avg | A24 | A25 | B25 | HMM | AMO | Avg |
| Initial | 73.3 | 60.0 | 73.3 | 56.7 | 24.0 | 53.5 | 40.0 | 36.7 | 40.0 | 16.7 | 0.0 | 23.5 |
| Reflex | 70.0 | 60.0 | 73.3 | 66.7 | 24.0 | \text{54.7}_{\,\text{{\color[rgb]{0.1563,0.4766,0.3281}+1.2}}} | 56.7 | 40.0 | 56.7 | 20.0 | 2.0 | \text{31.2}_{\,\text{{\color[rgb]{0.1563,0.4766,0.3281}+7.6}}} |
| Self-Refine | 80.0 | 66.7 | 73.3 | 63.3 | 26.0 | \text{57.6}_{\,\text{{\color[rgb]{0.1563,0.4766,0.3281}+4.1}}} | 53.3 | 43.3 | 46.7 | 23.3 | 4.0 | \text{30.6}_{\,\text{{\color[rgb]{0.1563,0.4766,0.3281}+7.1}}} |
| Self-Reflection | 73.3 | 60.0 | 76.7 | 63.3 | 30.0 | \text{57.1}_{\,\text{{\color[rgb]{0.1563,0.4766,0.3281}+3.5}}} | 56.7 | 40.0 | 53.3 | 30.0 | 4.0 | \text{32.9}_{\,\text{{\color[rgb]{0.1563,0.4766,0.3281}+9.4}}} |
| CheckList | 80.0 | 63.3 | 73.3 | 63.3 | 26.0 | \text{57.1}_{\,\text{{\color[rgb]{0.1563,0.4766,0.3281}+3.5}}} | 53.3 | 40.0 | 43.3 | 23.3 | 6.0 | \text{30.0}_{\,\text{{\color[rgb]{0.1563,0.4766,0.3281}+6.5}}} |
| Baseline | Mean gain | 95% CI | Minimum gain after excluding one dataset |
|---|---|---|---|
| Self-Refine | +13.5 | [9.9, 17.4] | +11.0 |
| ProgCo-Py | +14.4 | [10.9, 18.1] | +13.1 |
| ProgCo | +18.2 | [14.6, 22.1] | +16.9 |
| Return condition | Share of outputs | Accuracy within group |
|---|---|---|
| Original solution returned after passing the first check | 52.1% | 75.4% |
| Passed a later check after failing the first | 35.0% | 62.6% |
| Returned at the round limit after failing the last check | 12.9% | 20.5% |
| Model | AGPV | ProgCo | ProgCo-Py | Self-Refine | Self-Reflection |
|---|---|---|---|---|---|
| DeepSeek Flash | 42.5% | 11.3% | 20.0% | 26.3% | 11.3% |
| DeepSeek Pro | 48.1% | 6.7% | 17.3% | 26.0% | 5.8% |
| Qwen 35B | 22.8% | 7.6% | 11.4% | 8.9% | 10.1% |
| GPT-5.6 Luna | 38.5% | 3.8% | 6.9% | 10.0% | 13.1% |
| Model | Baseline | AGPV only | Baseline only | Both corrected it | Neither corrected it |
|---|---|---|---|---|---|
| DeepSeek Flash | ProgCo | 36.3% | 5.0% | 6.3% | 52.5% |
| ProgCo-Py | 27.5% | 5.0% | 15.0% | 52.5% | |
| Self-Refine | 23.8% | 7.5% | 18.8% | 50.0% | |
| Self-Reflection | 35.0% | 3.8% | 7.5% | 53.8% | |
| DeepSeek Pro | ProgCo | 43.3% | 1.9% | 4.8% | 50.0% |
| ProgCo-Py | 33.7% | 2.9% | 14.4% | 49.0% |
| Method | Accuracy | Model calls | Input tokens | Output tokens | Total tokens | Python runs | Time (min) |
|---|---|---|---|---|---|---|---|
| Shared initial answers | 52.9% | 1.86 | 2.97 | 3.54 | 6.51 | 0 | 0.57 |
| Reflex | 60.0% | 3.00 | 10.33 | 9.04 | 19.37 | 0 | 0.61 |
| Self-Refine | 61.8% | 6.00 | 23.77 | 12.18 | 35.95 | 0 | 0.92 |
| Self-Reflection | 55.3% | 3.00 | 11.55 | 10.66 | 22.21 | 0 | 0.77 |
| CheckList | 55.3% | 4.00 | 11.22 | 8.74 | 19.96 | 0 | 0.65 |
| CoT-Tool | 54.1% | 3.00 | 0.58 | 10.29 | 10.87 | 0 | 0.72 |
| Tool-code generation | Context compression | |||
|---|---|---|---|---|
| Backbone | Input | Output | Input | Output |
| DS-Flash | 8.08 | 0.96 | 0.00 | 0.00 |
| DS-Pro (0813) | 21.54 | 2.31 | 15.50 | 0.08 |
| Qwen-35B | 3.33 | 0.39 | 3.54 | 0.01 |
| GPT-Luna | 6.27 | 0.59 | 0.00 | 0.00 |
| Method | DS-Flash | DS-Pro (0813) | Qwen-35B | GPT-Luna |
|---|---|---|---|---|
| AGPV | 4.065 | 11.035 | 1.494 | 3.206 |
| ProgCo-Py | 39.788 | 10.471 | 3.124 | 0.006 |
| Model | Method | Initial | |||
|---|---|---|---|---|---|
| DS-Flash | AGPV | 52.9 | 68.8 | 70.0 | 72.4 |
| DS-Flash | ProgCo | 52.9 | 57.6 | 57.6 | 58.2 |
| DS-Flash | ProgCo-Py | 52.9 | 58.8 | 62.4 | 62.4 |
| DS-Flash | Self-Refine | 52.9 | 58.2 | 60.6 | 61.8 |
| DS-Pro (0813) | AGPV | 38.8 | 65.3 | 66.5 | 66.5 |
| DS-Pro (0813) | ProgCo | 38.8 | 40.0 | 41.2 | 41.2 |
| Additional token limit per problem | AGPV | Self-Refine | ProgCo-Py |
|---|---|---|---|
| 40,000 | 54.7% | 59.4% | 52.9% |
| 90,000 | 65.9% | 61.8% | 55.9% |
| 700,000 | 74.1% | 62.4% | 62.9% |
| Dataset | Problem-objective summary | Initial error | ProgCo / ProgCo-Py behavior | AGPV key evidence and correction | Characteristic |
|---|---|---|---|---|---|
| HMMT February 2025 | Count circular arrangements of with adjacent products at most ; rotations and reflections are distinct. | DeepSeek-Pro (0813), hmmt_feb_2025_10 : the initial answer is , not . After fixing , the counting argument wrongly excludes arrangements where is adjacent to . | Both methods retain . | Enumeration returns valid ordered arrangements and confirms that ’s neighbors are . Feedback identifies the incomplete count; the corrected argument includes the omitted placements and returns . | Counting-coverage diagnosis and correction. |
| AIME 2024 | Compute a product over the thirteenth roots of unity and return its remainder modulo . | GPT-Luna, aime_2024_19 : the initial answer is , not . The factorization is valid, but the sign of is wrong. | Both methods retain . | Verification identifies , rather than . Recomputing the conjugate product gives , hence remainder . | Local arithmetic-error diagnosis. |
| AIME 2025 | Find the expected number of regions formed by two perpendicular diameters and random cross-quadrant chords. | DeepSeek-Flash, aime_2025_12 : the initial answer is , not . It assumes random chords intersect with probability , ignoring the quadrant-conditioned sampling. | Both methods retain . | The saved feedback gives the conditioned probability ; the correction trace checks it numerically and recomputes the expected intersections and regions, returning . | Intermediate-probability correction. |
| AMO Bench | Find all possible remainders modulo of . | DeepSeek-Pro (0813), amo_bench_6 : the initial set is , omitting valid values. | ProgCo returns a set containing an extra ; ProgCo-Py returns an incomplete seven-element set. | The saved verification enumerates and finds missing even values. AGPV returns the reference-matching set , explicitly enumerated in the saved answer. | Missing-value detection from finite witnesses. |
| Method | Accuracy | Model calls per problem | Total tokens (k/problem) |
|---|---|---|---|
| Full AGPV | 77.1% | 19.44 | 154.37 |
| A. Continue past the first error | 74.1% | 17.96 | 226.95 |
| B. Remove the explicit prompt to check for omissions | 72.4% | 16.34 | 198.14 |
| C. Keep the first-round plan fixed | 76.5% | 16.03 | 128.38 |
| D. Disable mathematical tools | 67.1% | 6.71 | 54.57 |
| E. Verify directly without first listing key steps | 71.8% | 18.33 | 244.66 |
LLM Self-Correction
Large language model self-correction studies how language models detect and repair errors in their own answers, reasoning, and actions.
See all 171 papers