The rapid development of large language models (LLMs) for code, which closely aligns with the practical needs of real-world software development, has sparked growing attention in repository-level code completion. However, existing Code LLMs present challenges in focusing on suitable contexts from the repository and in deeply reasoning about cross-file dependencies. To address these challenges, we propose a novel reinforcement learning framework for repository-level code completion. To better understand both in-file and cross-file contexts, we employ identifier-driven intent recognition to capture completion intent, thereby improving the model's performance in real-world scenarios. To enhance cross-file reasoning, we propose a reward-driven completion learning to effectively reward in complex repository completion scenarios. To guide LLMs in completing code that aligns with intended functionality and repository context, we introduce a selective-exploration strategy that directs the model to focus on low-confidence, high-reward tokens, promoting the exploration of valuable, underexplored completion patterns. Experimental results show that our approach significantly improves the performance of Code LLMs. The code and the dataset are available at https://github.com/zhenlongDai/RepoCompletion.
Figures & tables
Figure 1: Situations of repo-level code completion.
Figure 2: Overview of IntentCoder. (a) Illustration of the Completion-Focused RL process. (b) Illustration of the Selective-Exploration Strategy Optimization.
Language
Sets
Num.
Completion position
IF
CF-F
CF-R
Java&Python
train
32,000
10,100
11,300
10,600
val
1,000
300
400
300
test
6,000
2,000
2,000
2,000
Table 1: Dataset Statistics of ICRepo.
Model
RepoBench-Java
RepoBench-Python
Code Match
Identifier Match
Code Match
Identifier Match
EM
ES
EM
Precision
Recall
F1
EM
ES
EM
Precision
Recall
F1
StarCoder2-7B
12.20
29.78
15.37
24.64
28.19
25.09
1.80
14.47
3.83
7.72
13.25
8.40
CodeLlama-7B
12.40
48.50
15.90
35.60
41.60
36.46
4.47
25.56
6.37
18.26
26.68
18.84
aixcoder-7B-v2
33.07
68.97
38.70
60.57
65.97
61.07
23.20
59.94
29.63
54.70
62.84
55.41
Starcoder2-15B
12.20
28.59
15.67
24.51
27.77
24.92
1.83
13.70
4.07
7.66
13.05
8.34
Table 2: Evaluation results on the RepoBench-Java/Python dataset. All results in the table are reported in percentage ( % ). The best method is in boldface, and the second best method is underlined for each metric.
Model
Code Match
Identifier Match
EM
ES
EM
Precision
Recall
F1
IntentCoder
35.93
71.58
42.70
65.57
63.80
63.37
w/o IDIR
34.60
69.98
41.27
63.74
63.10
62.05
w/o RDCL
34.30
68.97
40.20
62.24
60.87
60.25
w/o CFCL
34.47
67.73
40.70
62.06
59.61
59.45
w/o SESO
34.47
70.43
40.97
64.58
62.99
62.39
Table 3: Results of Ablation study on the RepoBench-Python dataset in Qwen2.5-Coder-7b.
Figure 3: Performance of IntentCoder in Various Completion Scenarios (CF-F,CF-R,IF).
Method
Java
Python
fail
pass
overall
fail
pass
overall
DeepSeek-Coder-6.7B
-131
+351
+220
-130
+189
+59
fine-tuning
-117
+599
+482
-61
+609
+548
GRPO
-78
+456
+378
-40
+691
+651
CoLT
-91
+462
+371
-156
+672
+516
IntentCoder deepseek
-83
+645
+562
-41
+709
+668
Table 4: Result of Exact Match (EM) count.
Appendix figures & tables4 assets
Supplementary material from the paper’s appendix.
Appendix
model
Java
Python
In-file
→
Both
In-file
→
Both
CodeLlama-7B
290
-143 | +227
374
134
-63 | +100
174
StarCoder2-7B
31
-143 | +192
80
55
-30 | +29
54
DeepSeek-Coder-6.7B
686
-131 |+ 351
906
390
-130 | +189
449
Qwen2.5-Coder-7B
631
-128 | +301
804
489
-138 | +225
576
DeepSeek-Coder-33B
541
-150 | +410
801
373
-166 | +214
421
Appendix
Table 5: Results of Exact Match (EM) Count Code Completion.
Model
BM25
OpenAI text-embedding-ada
Code
Identifier
Code
Identifier
EM
ES
EM
EM
ES
EM
Qwen2.5-Coder-7B
10.33
57.93
18.75
15.71
64.66
26.13
Qwen2.5-Coder-7B fine-tuning
11.87
58.09
19.35
20.35
67.80
27.21
Qwen2.5-Coder-7B GRPO
13.18
60.36
20.06
22.02
68.33
29.41
Qwen2.5-Coder-7B CoLT
12.90
59.72
19.54
23.18
69.72
30.67
Appendix
Table 6: Evaluation results on CrossCodeEval-Java.
Figure 5: The case study for IntentCodercompares it with other Code LLMs.
Model
RepoBench-Java
RepoBench-Python
Code Match
Identifier Match
Code Match
Identifier Match
EM
ES
EM
Precision
Recall
F1
EM
ES
EM
Precision
Recall
F1
IntentCoder Qwen
42.47
77.52
48.57
69.95
69.60
68.75
35.93
71.58
42.70
65.57
63.80
63.37
w/o Identifier-Driven Intent Recognition
40.27
76.77
46.47
68.73
68.62
67.67
34.60
69.98
41.27
63.74
63.10
62.05
w/o Reward-driven Completion Learning
40.80
76.94
47.67
69.56
68.49
68.03
34.30
68.97
40.20
62.24
60.87
60.25
w/o Completion-focused RL
40.90
76.13
47.07
68.49
67.62
66.95
34.47
67.73
40.70
62.06
59.61
59.45
Appendix
Table 7: Results of Ablation study on the RepoBench-java and RepoBench-Python dataset. All results in the table are reported in percentage ( % ).