RepoRepair: Leveraging Code Documentation for Repository-Level Automated Program Repair
Authors: Zhongqiang Pan, Chuanyi Li, Wenkang Zhong, Yi Feng, Bin Luo, Vincent Ng
Organizations: State Key Laboratory for Novel Software Technology, Nanjing University, Nanjing, China · Human Language Technology Research Institute, University of Texas at Dallas, Richardson, Texas, USA
Automated program repair (APR) struggles to scale from isolated functions to full repositories, as it demands a global, task-aware understanding to locate necessary changes. Current methods, limited by context and reliant on shallow retrieval or costly agent iterations, falter on complex cross-file issues. To this end, we propose RepoRepair, a novel documentation-enhanced approach for repository-level fault localization and program repair. Our core insight is to leverage LLMs to generate hierarchical code documentation (from functions to files) for code repositories, creating structured semantic abstractions that enable LLMs to comprehend repository-level context and dependencies. Specifically, RepoRepair first employs a text-based LLM (e.g., DeepSeek-V3) to generate file/function-level code documentation for repositories, which serves as auxiliary knowledge to guide fault localization. Subsequently, based on the fault localization results and the issue description, a powerful LLM (e.g., Claude-4) attempts to repair the identified suspicious code snippets. Evaluated on SWE-bench Lite, RepoRepair achieves a 45.7% repair rate at a low cost of 0.44perfix.OnSWE−benchMultimodal,itdeliversstate−of−the−artperformancewitha37.10.56 per fix, demonstrating robust and cost-effective performance across diverse problem domains.
Figures & tables
Figure 1. Overview of RepoRepair.
Figure 2. Prompt Texts for Generating Code Documentation.
Figure 3. Multimodal Issue Description Processing for Retrieval.
Figure 4. Pruning a Code File.
Tool
LLM
%Resolved
Avg. $Cost
RAG
Claude-3 Opus
4.33%
-
SWE-Agent
Claude-3.5 Sonnet
23.00%
1.62
SWE-Agent
Claude-4 Sonnet
56.67%
-
AutoCodeRover
GPT-4o
30.67%
0.65
Moatless Tools
Claude-3.5 Sonnet
38.33%
0.17
Agentless
GPT-4o
27.33%
0.34
Table 1. Performance Comparison of Baseline Tools on SWE-bench Lite.
Tool
LLM
%Resolved
Avg. $Cost
RAG
GPT-4o
6.00%
0.17
SWE-Agent
Claude-3.5 Sonnet
12.19%
1.52
SWE-Agent JS
Claude-3.5 Sonnet
11.99%
3.11
SWE-agent M
GPT-4o
12.19%
2.94
Agentless
Claude-3.5 Sonnet
6.19%
0.42
Agentless Lite
Claude-3.5 Sonnet
25.34%
0.38
Table 2. Performance Comparison of Baseline Tools on SWE-bench Multimodal.
Figure 5. Unique bug resolutions of top-performing tools on (left) SWE-bench Lite and (right) SWE-bench Multimodal.
Repository
Agentless Lite
GUIRepair
RepoRepair
alibaba-fusion/next
6/39
10/39
10/39
bpmn-io/bpmn-js
27/54
37/54
51/54
carbon-design-system/carbon
9/134
24/134
24/134
eslint/eslint
2/11
6/11
8/11
GoogleChrome/lighthouse
4/54
8/54
6/54
grommet/grommet
2/21
5/21
5/21
Table 3. Repair Results (%Resolved) Comparison Between RepoRepair, Agentless Lite and GUIRepair Across Repositories in SWE-bench Multimodal.
Repository
File Retrieval
File Localization
AL
RR
AL
RR
astropy/astropy
6/6
6/6
5/6
6/6
django/django
109/114
110/114
97/114
100/114
matplotlib/matplotlib
21/23
22/23
10/23
16/23
mwaskom/seaborn
4/4
4/4
2/4
3/4
pallets/flask
3/3
3/3
3/3
3/3
Table 4. File Localization Results (%Correct Localization) Comparison Between RepoRepair and Agentless Lite Across Repositories in SWE-bench Lite. (AL = Agentless Lite; RR = RepoRepair)
Repository
File R
File L
AL
RR
AL
GR
RR
alibaba-fusion/next
23/39
21/39
14/39
13/39
17/39
bpmn-io/bpmn-js
49/54
51/54
15/54
22/54
31/54
carbon-design-system/carbon
119/134
116/134
32/134
67/134
97/134
eslint/eslint
9/11
10/11
5/11
6/11
10/11
GoogleChrome/lighthouse
37/54
39/54
21/54
14/54
27/54
Table 5. File Localization Results (%Correct Localization) Comparison Between RepoRepair, Agentless Lite, and GUIRepair Across Repositories in SWE-bench Multimodal. (File R = File Retrieval; File L = File Localization; AL = Agentless Lite; GR = GUIRepair; RR = RepoRepair)
Metric
SWE-bench Lite
SWE-bench Multimodal
Successful file localizations
237
309
Successful function localizations
205 (86.5%)
273 (88.35%)
Function localization in 1 round
222 (93.67%)
302 (97.73%)
Function localization requiring >1 round
15 (6.33%)
7 (2.27%)
Average function localization rounds
1.06
1.02
Table 6. Function-Level Localization Analysis of RepoRepair on two benchmarks.
SWE-bench Lite
SWE-bench Multimodal
Full
w/o FR
w/o Doc
w/o CP
Full
w/o FR
w/o Doc
w/o CP
File R
0
-
0
0
1.4M
-
1.4M
1.4M
File L
2.9M
44.6M
0.6M
2.9M
23.5M
132.6M
2.9M
23.5M
Func L
6.3M
13.5M
0.3M
6.3M
6.2M
16.5M
-
6.2M
Repair
2.2M
5.4M
1.9M
3.8M
8.1M
21.6M
9.0M
12.5M
Total
11.4M
63.5M
2.8M
13.0M
39.2M
170.7M
13.4M
43.6M
Table 7. Computational Cost (Tokens) of RepoRepair Variants in Localization and Repair on two benchmarks. (File R = File Retrieval; File L = File Localization; Func L = Function Localization; CP = Code Pruning)
SWE-bench Lite
SWE-bench Multimodal
Full
w/o FR
w/o Doc
w/o CP
Full
w/o FR
w/o Doc
w/o CP
File R
282/300
-
282/300
282/300
401/517
-
401/517
401/517
File L
237/300
229/300
225/300
237/300
309/517
282/517
180/517
309/517
Func L
205/300
190/300
159/300
205/300
273/517
249/517
-
273/517
Repair
137/300
124/300
111/300
128/300
192/517
183/517
136/517
174/517
Table 8. Performance of RepoRepair Variants in Localization (%Correct Localization) and Repair (%Resolved) on two benchmarks. (File R = File Retrieval; File L = File Localization; Func L = Function Localization; CP = Code Pruning)
Doc Generator
File R
File L
Func L
Repair
DeepSeek-V3
110/114 (96.5%)
100/114 (87.7%)
93/114 (81.6%)
67/114 (58.8%)
Qwen2.5-7B
98/114 (86.0%)
82/114 (71.9%)
68/114 (59.6%)
39/114 (34.2%)
Table 9. Impact of Documentation Quality on RepoRepair’s Performance (django/django). (File R = File Retrieval; File L = File Localization; Func L = Function Localization)
Document Granularity
Sample Pairs
Prefer DeepSeek-V3
Prefer Qwen2.5-7B
File-level
100
96 (96%)
4 (4%)
Function-level
627
615 (98.1%)
12 (1.9%)
Table 10. Human Comparative Evaluation of Generated Documentation Quality (django/django).
Figure 6. Case Comparison: Patches Generated by RepoRepair and Agentless Lite for Issue carbon-design-system/carbon-3610.
Automatic program repair can be a challenging task, especially when resolving complex issues at a repository-level, which often involves issue reproduction, fault localization, code repair, testing and validation. Issues of this scale can be commonly found in popular GitHub repositories or datasets that are derived from them. Some repository-level approaches separate localization and repair into distinct phases. Where this is the case, the fault localization approaches vary in terms of the granularity of localization. Where the impact of granularity is explored to some degree for smaller datasets, not all isolate this issue from the separate question of localization accuracy by testing code repair under the assumption of perfect fault localization. To the best of the authors' knowledge, no repository-scale studies have explicitly investigated granularity under this assumption, nor conducted a systematic empirical comparison of granularity levels in isolation. We propose a framework for performing such tests by modifying the localization phase of the Agentless framework to retrieve ground-truth localization data and include this as context in the prompt fed to the repair phase. We show that under this configuration and as a generalization over the SWE-Bench-Mini dataset, function-level granularity yields the highest repair rate against line-level and file-level. However, a deeper dive suggests that the ideal granularity may in fact be task dependent. This study is not intended to improve on the state-of-the-art, nor do we intend for results to be compared against any complete agentic frameworks. Rather, we present a proof of concept for investigating how fault localization may impact automatic code repair in repository-scale scenarios. We present preliminary findings to this end and encourage further research into this relationship between the two phases.
Joseph Townsend, Chandresh Pravin, Kwun Ho Ngan +1
Repository-grounded automated repair is often reported as a single end-to-end capability, which hides distinct failure modes such as poor file targeting, incorrect patch synthesis, and failed iterative debugging. We present Loc2Repair, a modular evaluation framework for controlled analysis of repository-grounded repair pipelines, and use it to isolate file-level issue localization as an upstream variable. Loc2Repair decouples localization and repair under a shared runtime, artifact schema, and evaluation harness, allowing researchers to combine different localization models and repair backbones under matched conditions. Using three repair backbones on SWE-bench Verified, we compare baseline repair without explicit localization, repair guided by predicted localization from two localizers, and repair guided by gold modified-file sets. Explicit localization consistently improves resolved rate across all backbones: pooled performance increases from 44.7% for baseline repair to 48.9% and 49.1% with predicted localization, and to 52.4% with gold localization. Localization also reduces mean elapsed time overall: in pooled paired analysis, mean elapsed time decreases by 100.94 s and 52.25 s for the two predicted-localization settings, and by 154.45 s with gold guidance, although token effects remain heterogeneous across models. Overall, Loc2Repair shows file-level localization is a consistent repair lever, improving effectiveness and mean latency in pooled analysis, while gold-guided failures expose headroom beyond localization.
Recent memory-augmented repository-level program repair methods reuse historical repair experiences to improve LLM-based issue resolution. However, our analysis reveals three limitations in existing repository-level memory retrieval. First, episodic memory is highly imbalanced across repositories, leaving low-resource repositories with little effective support. Second, more memory does not monotonically lead to higher repair success, suggesting that relevance, quality, and redundancy matter more than raw memory volume. Third, memory accumulation is phase-misaligned: repositories may contain many reproduction experiences but few patch or refinement experiences. To address these problems, we propose an adaptive experience retrieval framework for repository-level program repair. Our framework introduces coverage-aware retrieval, which falls back to cross-repository or repair-type-based memories when same-repository memory is insufficient; quality-aware selection, which ranks memories by relevance, historical utility, specificity, and redundancy; and stage-aware routing, which separates and retrieves memories for reproduction, localization, patch generation, patch refinement, and validation. Evaluated on SWE-Bench-Lite and SWE-Bench-Verified, the proposed framework improves repair performance on under-covered repositories, reduces noisy memory retrieval, and better supports failed-to-fixed patch refinement. Our results show that the key to memory-augmented repair is not simply accumulating more experiences, but retrieving the right experiences for the right repair context.