Authors: Mohsen Iranmanesh, Sina Moradi Sabet, Sina Marefat, Ali Javidi Ghasr, Maxime Newman Nereyabagabo, Kousha Amouzesh, Jaivir Malhi, Vinci Velasco, Alex Chung, Mohammad A. Tayebi
Organizations: Simon Fraser University, Canada · Amirkabir University of Technology, Iran · K. N. Toosi University of Technology, Iran · Ferdowsi University of Mashhad, Iran
Large language models are increasingly deployed as autonomous agents for automated program repair, patching software vulnerabilities through multi-step reasoning and tool use. Yet these systems typically validate a patch against the same evidence used to construct it -- a static alert, sanitizer trace, published exploit, or test suite. A patch that blocks that one piece of evidence can therefore look successful while remaining vulnerable to the same vulnerability with some variation. We present P2Patch, a generative adversarial repair system that treats each patch as a new attack target. Starting from a normalized vulnerability finding, an exploitation agent constructs a reproducing proof of vulnerability (PoV), a patching agent repairs the program, and the exploitation agent then attacks the patched program for a distinct same-vulnerability exploit. Every successful exploit becomes a permanent repair requirement, and the final patch must block all PoVs accumulated during the patching process, with construction-time PoVs kept strictly separate from held-out evaluation oracles. Across 101 C/C++ and Java CVE subjects, this adversarial loop surfaces a distinct confirmed exploit in 41% of runs after the patch has already blocked its original construction-time PoV. Against single-pass repair from the same finding using Claude Code, P2Patch raises held-out fix-closed coverage from 78.1% to 84.7% and beyond-upstream coverage from 42.1% to 62.5%. Under the same held-out oracles, it also certifies more attack paths as blocked than recent state-of-the-art agentic repair systems on overlapping subjects. These results show that adversarial testing and independent security oracles expose repair gaps that one-shot validation misses.
Figures & tables
Figure 1: Motivating plexus-utils example. The 2016 containment check is bypassed by a sibling path; the later fix closes the gap by canonicalizing paths and enforcing a separator boundary.
Figure 2: P2Patch workflow: bootstrap a live PoV, patch accumulated requirements, attack the patched program for bypasses, then score the final patch with held-out suites.
Figure 3: Construction and held-out evaluation are strictly separated. During repair, agents see only the normalized finding and accumulated live PoVs. After repair terminates, only the final patch crosses into held-out evaluation: fix-closed oracles measure whether it blocks paths closed by the official fix ( Cfix ), while residual oracles measure whether it blocks paths the official fix leaves open ( Cres ).
Java
C/C++
All
CVEs
60
41
101
CVEs with a fix-closed suite
60
41
101
CVEs with a residual suite
28
12
40
Fix-closed PoVs
213
111
324
Residual PoVs
51
15
66
Table 1: VulnRepairBench summary: 101 CVEs, 61 repositories, and 20 weakness classes. All subjects have fix-closed suites; 40 also have residual suites.
Oracle
n
Macro coverage
Paired comparison
SinglePass
P2Patch
k
Δ
+/=/−
pHolm
Cfix
101
0.781
0.847
20
+0.065
17/81/3
0.0286
Cres
40
0.421
0.625
13
+0.204
11/27/2
0.0254
Table 2: Primary held-out macro coverage under intention-to-treat scoring. Paired columns report non-tied pairs ( k ), wins/ties/losses, and Holm-adjusted exact Wilcoxon signed-rank p -values.
Step
Mean
Δ
+/=/−
p
Fix-closed coverage Cfix , n=101
SinglePass
0.781
—
—
—
+ live PoV
0.809
+0.027
10/88/3
0.2944
+ adversarial loop
0.847
+0.038
10/91/0
0.0020
Beyond-upstream coverage Cres , n=40
SinglePass
0.421
—
—
—
Table 3: Attribution of the coverage gain to the bootstrap PoV and adversarial loop. Each Δ is relative to the preceding row; p is an exact Wilcoxon signed-rank test.
Model
acc.
byp.
Cfix
Cres
$/subj.
min.
Claude Haiku 4.5
98
41%
0.847
0.625
2.09
25.8
GLM 5.2
100
47%
0.921
0.795
2.43
37.1
DeepSeek V4 Flash
94
35%
0.821
0.608
0.54
37.5
GPT-5.6-luna
66
31%
0.594
0.400
0.16
11.8
Table 4: Full P2Patch protocol across four model families under intention-to-treat scoring. “acc.” reports accepted repairs; “byp.” reports the share of runs with an execution-confirmed bypass. Cost is the mean per subject and wall time the median. Cfix and Cres use all 101 and all 40 subjects for every arm.
System
Oracle
n
Comp.
P2Patch
Δ
SinglePass
Cfix
101
0.781
0.847
+0.065
Cres
40
0.421
0.625
+0.204
San2Patch
Cfix
33
0.608
0.922
+0.314
Cres
10
0.100
0.600
+0.500
LoopRepair
Cfix
39
0.325
0.930
+0.605
Cres
11
0.182
0.636
+0.454
Table 5: Pairwise held-out coverage on shared subjects under identical certified suites and intention-to-treat scoring. n is the number of shared subjects with a measurable suite; “Comp.” denotes the comparison system.
Method
Runs
Total ($)
Mean/run ($)
SinglePass
101
45.90
0.45
San2Patch
38
18.22
0.48
LoopRepair
40
15.03
0.38
PatchAgent
15
4.79
0.32
P2Patch
101
210.76
2.09
Table 6: Operational repair cost over every actual run, including failed-run spend. All five methods were executed in our evaluation with Claude Haiku 4.5. PatchAgent cost is reconstructed from execution traces and is therefore a lower bound.
Appendix figures & tables10 assets
Supplementary material from the paper’s appendix.
Appendix
CVE
Project
CWE
Certified PoVs
∣Sfix∣
∣Sres∣
2014-7816
undertow
CWE-22
3
—
2016-5394
sling-xss
CWE-79
3
2
2016-6812
cxf
CWE-79
2
1
2016-9177
spark
CWE-22
2
2
2017-1000487
plexus-utils
CWE-78
8
—
Appendix
Table 7: All 101 VulnRepairBench subjects (60 Java and 41 C/C++) with CWE and certified PoV-suite sizes. ∣Sfix∣ and ∣Sres∣ are the fix-closed and residual suite sizes; — denotes no certified residual suite.
Oracle
Subset
n
SinglePass
P2Patch
Δ
Cfix
Java
60
0.730
0.801
+0.071
C/C++
41
0.857
0.913
+0.057
Cres
Java
28
0.494
0.643
+0.149
C/C++
12
0.250
0.583
+0.333
Appendix
Table 8: Exploratory language breakdown of Table 2 under the same intention-to-treat scoring. Coverage is macro-averaged per subject; these rows are descriptive diagnostics, not separate inferential results.
Budget
Cfix
Δ
Cres
Δ
R=0 (PoV only)
0.811
—
0.457
—
R=1
0.833
+0.022
0.556
+0.099
R=2
0.848
+0.014
0.590
+0.033
R=3
0.850
+0.002
0.615
+0.026
R=4 (default)
0.850
+0.000
0.615
+0.000
Appendix
Table 9: Round-budget sweep. Each row reports held-out coverage at the given maximum number of adversarial rounds. The analysis includes subjects measured at every budget: 98 for Cfix and 39 for Cres . Within this set, the intention-to-treat policy applies, so a run that shipped nothing is scored as zero. The different denominators are why R=4 here sits marginally under Table 2 ’s 0.847 and 0.625.
Figure 4: Hardening-round dynamics. Left: execution-confirmed bypass rate among runs entering each round; the pooled round-level rate is 42% (68/163), distinct from the 41% run-level rate in Section 6.1 . Right: held-out coverage versus round budget, with R=0 denoting PoV-only repair. Later-round cohorts are small; exact values are in Table 9 .
Figure 5: Operational cost versus strict fix-closed coverage on the shared C/C++ benchmark. Coverage is the fraction of subjects blocking every certified fix-closed path under intention-to-treat scoring. P2Patch model points use all 41 subjects; published systems use their measurable shared-subject sets. Cost is mean spend per actual run.
Claude Code
San2Patch
LoopRepair
PatchAgent
P2Patch
Project
CVE
Type
Cfix
Cres
Cfix
Cres
Cfix
Cres
Cfix
Cres
Cfix
Cres
libjpeg-turbo
2012-2806
OOB-W
✓
—
✓
—
✓
—
✓
—
✓
—
2017-15232
NPD
△
—
△
—
∅
∅
△
—
△
—
libxml2
2012-5134
BOF
×
—
✓
—
×
—
✓
—
✓
—
2016-1833
OOB-R
×
—
—
—
×
—
—
—
∅
—
2016-1838
OOB-R
✓
—
✓
—
×
—
✓
—
✓
—
Appendix
Table 10: Per-case held-out outcomes on the 41 C/C++ subjects under intention-to-treat scoring. ✓ , △ , and × denote full, partial, and zero blocking; ∅ denotes no produced patch, — no applicable suite or out-of-benchmark subject, and ? an unmeasured subject. A residual × matches upstream behavior and is not a repair failure. Shaded columns are P2Patch .
Figure 6: Residual gaps against upstream history. Each row is one gap the official CVE fix left open, and its length is how long that gap survived in upstream’s own code: up to the commit that later closed the same path ( ✓ , green), or up to August 2026 with the path still there ( × , red). A hollow marker is a commit that closed the path without recording that a defect existed. The PoVs column counts the certified residual PoVs driving that one gap, and the last column is the upstream commit that closed it. † the same gap is certified under CVE-2023-45278 as well; those two suites share one official fix and one closing commit. potrace has no public repository, so its row is dated release to release.
System
Loop ranges over
Feedback
Objective moves?
San2Patch
reasoning paths (ToT)
compiler, PoC result
no
PatchAgent
agent actions
test failure, rejected patches
no
LoopRepair
candidate patches
ranked failing patch
no
P2Patch
the objective
a new working exploit
yes
Appendix
Table 11: Recent systems all iterate. What differs is what the loop ranges over, what feedback it receives, and whether the repair target can move.
CVE
Project
∣Sfix∣
Cfix
Cres
b
base
ours
base
ours
2014-7816
undertow
3
1.00
1.00
—
—
0
2016-5394
sling-xss
3
1.00
1.00
0.00
0.00
0
2016-6812
cxf
2
0.50
0.50
1.00
1.00
4
2016-9177
spark
2
0.50
1.00
1.00
1.00
0
2017-1000487
plexus-utils
8
0.50
1.00
—
—
1
Appendix
Table 12: Per-subject held-out coverage for 60 Java subjects. ∣Sfix∣ is the fix-closed suite size and b=∣E∣−1 the number of appended execution-confirmed bypasses. “base” is SinglePass and “ours” is P2Patch ; bold marks improvement, ↓ regression, — no residual suite, and ∅ no accepted P2Patch patch. Residual suite sizes are in Table 7 .
Automated vulnerability repair has emerged as a promising direction to mitigate the growing number of software vulnerabilities. Recent advances in Large Language Models (LLMs) have further accelerated research in automated repair. However, existing frameworks remain largely restricted to memory-related vulnerabilities and locally repairable vulnerability settings, leaving generalization to unseen vulnerability types underexplored. Their evaluations are often limited to a single programming language, and largely rely on proprietary models. In this paper, we propose RAVEN, a scalable, efficient and autonomous framework that integrates an agentic retrieval-augmented generation (RAG) pipeline with controlled iterative repair in a unified framework. The framework utilizes open-source LLMs in a fully locally deployable setting with limited GPU requirements, while building a multi-faceted retrieval pipeline to retrieve historically relevant vulnerability fixes and guide the patch generation. In addition, RAVEN introduces a dedicated Curator Agent that retrieves cross-file dependencies from the target repository, to fix complex vulnerabilities that cannot be addressed using local vulnerable code alone. We evaluate RAVEN on 160 real-world CVE vulnerabilities across diverse vulnerability types, two programming languages, unseen CWE categories, and out-of-distribution settings. RAVEN achieves an overall repair success rate of 83.13%, outperforming all existing state-of-the-art repair frameworks, while also demonstrating strong generalization capabilities and maintaining the repair cost negligible.
Automated vulnerability repair aims to reduce the time and effort required to patch security flaws from a vulnerability triage report. Recent agentic AI approaches have shown promising results in automated program repair. However, vulnerability repair demands richer program context than general bug repair - context that security engineers routinely assemble in practice but that existing agentic approaches do not engineer. We identify three critical gaps: code-structure context capturing cross-file data flows and memory operation patterns, runtime-execution context revealing crash semantics and memory origins, and commit-history context recovering how fragile code patterns were introduced. We present AgenticRepair, an agentic vulnerability repair framework that addresses the gaps through multi-faceted program context engineering. AgenticRepair orchestrates three specialized LLM subagents to engineer the contexts, which are then embedded into the memory of a dedicated repair subagent for context-conditioned patch synthesis. Evaluated on SEC-Bench comprising 300 real-world instances with sanitizer-based patch verification, AgenticRepair achieves a 73% success rate, substantially outperforming the strongest baseline by 29%. Our ablation study confirms that the three context facets are mutually complementary, and that multi-agent scaffolding and base-model capacity each play an essential role. Collectively, these findings establish multi-faceted program context engineering as a promising design direction for agentic vulnerability repair.
Michael Fu, Qiyue Mei, Patanamon Thongtanunam +1
the School of Computing and Information Systems, The University of Melbourne, Melbourne, Australia. · Kla Tantithamthavorn is with the Faculty of Information Technology, Monash University, Melbourne, Australia.
As software systems grow increasingly complex, automated vulnerability repair (AVR) remains difficult because the materials available to a repair system are usually failure artifacts rather than repair guidance. Traditional analysis techniques can provide suspicious locations, reduced triggers, or constraints, but they are costly to configure across repositories and seldom directly actionable for patch generation. Recent LLM-based agents can edit and validate repository-level patches, and experience-based systems can reuse prior repair traces or demonstrations, but they still need current-instance evidence that turns a broad, symptom-level failure report into a concrete repair decision. We present ContraFix, an agentic AVR framework that constructs such evidence through contrastive runtime analysis. Starting from a failing witness, ContraFix generates nearby failing and non-failing variants, executes them through aligned probe sites, and compares their runtime states to infer the repair boundary and guide source-level patching. Each candidate patch is accepted only after build and validation. ContraFix also stores validated repair episodes in a dual-track skill base, reusing mutation skills to construct useful variants and correction skills to refine failed patches. On SEC-Bench, ContraFix with GPT-5-mini achieves resolution rate of 92.0% over three repeated runs and an average resolution rate of 91.8% +/- 0.8. On PatchEval, it resolves 73.8% of 225 Go, Python, and JavaScript instances. A semantic audit of benchmark-validated SEC-Bench patches shows that 58.2% of ContraFix's patches are semantically correct, compared with 31.3% for the strongest baseline, indicating that the proposed framework improves semantic correctness beyond benchmark validation.
Simiao Liu, Fang Liu, Peiding Wang +4
Beihang University Beijing, China · The University of Hong Kong Hong Kong, China