RepoRepair: Leveraging Code Documentation for Repository-Level Automated Program Repair
Authors: Zhongqiang Pan, Chuanyi Li, Wenkang Zhong, Yi Feng, Bin Luo, Vincent Ng
Organizations: State Key Laboratory for Novel Software Technology, Nanjing University, Nanjing, China · Human Language Technology Research Institute, University of Texas at Dallas, Richardson, Texas, USA
Automated program repair (APR) struggles to scale from isolated functions to full repositories, as it demands a global, task-aware understanding to locate necessary changes. Current methods, limited by context and reliant on shallow retrieval or costly agent iterations, falter on complex cross-file issues. To this end, we propose RepoRepair, a novel documentation-enhanced approach for repository-level fault localization and program repair. Our core insight is to leverage LLMs to generate hierarchical code documentation (from functions to files) for code repositories, creating structured semantic abstractions that enable LLMs to comprehend repository-level context and dependencies. Specifically, RepoRepair first employs a text-based LLM (e.g., DeepSeek-V3) to generate file/function-level code documentation for repositories, which serves as auxiliary knowledge to guide fault localization. Subsequently, based on the fault localization results and the issue description, a powerful LLM (e.g., Claude-4) attempts to repair the identified suspicious code snippets. Evaluated on SWE-bench Lite, RepoRepair achieves a 45.7% repair rate at a low cost of 0.44perfix.OnSWE−benchMultimodal,itdeliversstate−of−the−artperformancewitha37.10.56 per fix, demonstrating robust and cost-effective performance across diverse problem domains.
Figures & tables
Figure 1. Overview of RepoRepair.
Figure 2. Prompt Texts for Generating Code Documentation.
Figure 3. Multimodal Issue Description Processing for Retrieval.
Figure 4. Pruning a Code File.
Tool
LLM
%Resolved
Avg. $Cost
RAG
Claude-3 Opus
4.33%
-
SWE-Agent
Claude-3.5 Sonnet
23.00%
1.62
SWE-Agent
Claude-4 Sonnet
56.67%
-
AutoCodeRover
GPT-4o
30.67%
0.65
Moatless Tools
Claude-3.5 Sonnet
38.33%
0.17
Agentless
GPT-4o
27.33%
0.34
Table 1. Performance Comparison of Baseline Tools on SWE-bench Lite.
Tool
LLM
%Resolved
Avg. $Cost
RAG
GPT-4o
6.00%
0.17
SWE-Agent
Claude-3.5 Sonnet
12.19%
1.52
SWE-Agent JS
Claude-3.5 Sonnet
11.99%
3.11
SWE-agent M
GPT-4o
12.19%
2.94
Agentless
Claude-3.5 Sonnet
6.19%
0.42
Agentless Lite
Claude-3.5 Sonnet
25.34%
0.38
Table 2. Performance Comparison of Baseline Tools on SWE-bench Multimodal.
Figure 5. Unique bug resolutions of top-performing tools on (left) SWE-bench Lite and (right) SWE-bench Multimodal.
Repository
Agentless Lite
GUIRepair
RepoRepair
alibaba-fusion/next
6/39
10/39
10/39
bpmn-io/bpmn-js
27/54
37/54
51/54
carbon-design-system/carbon
9/134
24/134
24/134
eslint/eslint
2/11
6/11
8/11
GoogleChrome/lighthouse
4/54
8/54
6/54
grommet/grommet
2/21
5/21
5/21
Table 3. Repair Results (%Resolved) Comparison Between RepoRepair, Agentless Lite and GUIRepair Across Repositories in SWE-bench Multimodal.
Repository
File Retrieval
File Localization
AL
RR
AL
RR
astropy/astropy
6/6
6/6
5/6
6/6
django/django
109/114
110/114
97/114
100/114
matplotlib/matplotlib
21/23
22/23
10/23
16/23
mwaskom/seaborn
4/4
4/4
2/4
3/4
pallets/flask
3/3
3/3
3/3
3/3
Table 4. File Localization Results (%Correct Localization) Comparison Between RepoRepair and Agentless Lite Across Repositories in SWE-bench Lite. (AL = Agentless Lite; RR = RepoRepair)
Repository
File R
File L
AL
RR
AL
GR
RR
alibaba-fusion/next
23/39
21/39
14/39
13/39
17/39
bpmn-io/bpmn-js
49/54
51/54
15/54
22/54
31/54
carbon-design-system/carbon
119/134
116/134
32/134
67/134
97/134
eslint/eslint
9/11
10/11
5/11
6/11
10/11
GoogleChrome/lighthouse
37/54
39/54
21/54
14/54
27/54
Table 5. File Localization Results (%Correct Localization) Comparison Between RepoRepair, Agentless Lite, and GUIRepair Across Repositories in SWE-bench Multimodal. (File R = File Retrieval; File L = File Localization; AL = Agentless Lite; GR = GUIRepair; RR = RepoRepair)
Metric
SWE-bench Lite
SWE-bench Multimodal
Successful file localizations
237
309
Successful function localizations
205 (86.5%)
273 (88.35%)
Function localization in 1 round
222 (93.67%)
302 (97.73%)
Function localization requiring >1 round
15 (6.33%)
7 (2.27%)
Average function localization rounds
1.06
1.02
Table 6. Function-Level Localization Analysis of RepoRepair on two benchmarks.
SWE-bench Lite
SWE-bench Multimodal
Full
w/o FR
w/o Doc
w/o CP
Full
w/o FR
w/o Doc
w/o CP
File R
0
-
0
0
1.4M
-
1.4M
1.4M
File L
2.9M
44.6M
0.6M
2.9M
23.5M
132.6M
2.9M
23.5M
Func L
6.3M
13.5M
0.3M
6.3M
6.2M
16.5M
-
6.2M
Repair
2.2M
5.4M
1.9M
3.8M
8.1M
21.6M
9.0M
12.5M
Total
11.4M
63.5M
2.8M
13.0M
39.2M
170.7M
13.4M
43.6M
Table 7. Computational Cost (Tokens) of RepoRepair Variants in Localization and Repair on two benchmarks. (File R = File Retrieval; File L = File Localization; Func L = Function Localization; CP = Code Pruning)
SWE-bench Lite
SWE-bench Multimodal
Full
w/o FR
w/o Doc
w/o CP
Full
w/o FR
w/o Doc
w/o CP
File R
282/300
-
282/300
282/300
401/517
-
401/517
401/517
File L
237/300
229/300
225/300
237/300
309/517
282/517
180/517
309/517
Func L
205/300
190/300
159/300
205/300
273/517
249/517
-
273/517
Repair
137/300
124/300
111/300
128/300
192/517
183/517
136/517
174/517
Table 8. Performance of RepoRepair Variants in Localization (%Correct Localization) and Repair (%Resolved) on two benchmarks. (File R = File Retrieval; File L = File Localization; Func L = Function Localization; CP = Code Pruning)
Doc Generator
File R
File L
Func L
Repair
DeepSeek-V3
110/114 (96.5%)
100/114 (87.7%)
93/114 (81.6%)
67/114 (58.8%)
Qwen2.5-7B
98/114 (86.0%)
82/114 (71.9%)
68/114 (59.6%)
39/114 (34.2%)
Table 9. Impact of Documentation Quality on RepoRepair’s Performance (django/django). (File R = File Retrieval; File L = File Localization; Func L = Function Localization)
Document Granularity
Sample Pairs
Prefer DeepSeek-V3
Prefer Qwen2.5-7B
File-level
100
96 (96%)
4 (4%)
Function-level
627
615 (98.1%)
12 (1.9%)
Table 10. Human Comparative Evaluation of Generated Documentation Quality (django/django).
Figure 6. Case Comparison: Patches Generated by RepoRepair and Agentless Lite for Issue carbon-design-system/carbon-3610.