Large language model (LLM) agents increasingly reuse prior experience, but most approaches assume that problems and solutions are already aligned. Software histories rarely provide this alignment: a pull request (PR) may contain multiple continuous integration (CI) problems, failed attempts, reverted edits, and unrelated changes, obscuring which changes resolve each problem. We present RETRACE, a framework for reconstructing problem-level repair experience from such histories. RETRACE combines an endpoint view that reasons backward from changes retained in the passing revision with a development view that traces repair evolution forward through commit history. CI execution evidence reconciles the two views, and the recovered experience is represented at three abstraction levels, from concrete fixes to transferable repair patterns. For new failures, RETRACE retrieves relevant problem-level experience to guide repair. On CI-REPAIR-BENCH, comprising 565 PR-level repairs from 101 repositories across 12 failure categories, RETRACE improves mini-SWE-agent Pass@1 from 19.6% to 31.9% with MiniMax-M2.5 and from 23.3% to 32.8% with DeepSeek-V4-Flash. On a matched subset, Codex improves from 15.5% to 27.5%. Combining both views consistently outperforms either alone, showing that recovering problem-change alignment enables historical CI repairs to serve as reusable repair experience.
Figures & tables
Metric (Avg.)
CI-Repair- Bench
SWE- bench
SWE-bench Verified
SWE- Gym
SWE- bench Live
SWE- bench Pro
SWE-bench Multimodal
FEA- Bench
# changed files
10.5
1.7
1.24
2.5
3.3
4.1
2.0 †
2.62
# changed lines
240.1
32.8
14.32
69.8
102.6
107.4
27 †
128.5
Table 1: Comparison of change size across repository-level software repair benchmarks.
Figure 1: Overview of RETRACE .
Eval.
Agent
Model
Method
Pass@1 (%)
Input (M)
Cost ($)
Full ( n=408 )
mini-SWE
MiniMax-M2.5
No-Experience
19.6 ∗
1.081
0.080
Historical-plan
18.4 ∗
1.144
0.050
RETRACE
31.9
1.903
0.139
mini-SWE
DeepSeek-V4-Flash
No-Experience
23.3 ∗
1.023
0.033
Historical-plan
17.9 ∗
1.282
0.022
RETRACE
32.8
1.800
0.052
Table 2: Main results on CI-Repair-Bench . ∗ indicates that RETRACE significantly outperforms the marked baseline (exact McNemar test with Holm correction; pHolm<0.05 ).
Agent ( n )
Model
Configuration
Pass@1 (%)
Input (M)
Cost ($)
mini-SWE (408)
MiniMax-M2.5
RETRACE w/oEndpoint
28.4*
1.262
0.108
RETRACE w/oDevelopment
27.2*
1.565
0.127
RETRACE Full
31.9
1.903
0.139
mini-SWE (408)
DeepSeek-V4-Flash
RETRACE w/oEndpoint
28.4*
1.194
0.040
RETRACE w/oDevelopment
27.7*
1.480
0.048
RETRACE Full
32.8
1.800
0.052
Table 3: Effect of reconstruction strategy on CI-Repair-Bench . All configurations use three-level experience abstractions. ∗ indicates that RETRACE Full significantly outperforms the marked single-view variant (exact McNemar test with Holm correction; pHolm<0.05 ).
Abstraction Level
Resolved
Pass@1 (%)
Δ (pp)
L1
52/200
26.0
–
L1+L2
56/200
28.0
+2.0
L1+L2+L3
61/200
30.5
+4.5
Table 4: Ablation of repair-experience abstraction levels on a fixed 200-instance subset using mini-SWE-agent with DeepSeek-V4-Flash. All other RETRACE components are held fixed; Δ denotes the absolute Pass@1 improvement over L1.
Appendix figures & tables5 assets
Supplementary material from the paper’s appendix.
Appendix
Dimension
Characteristic
Total
Mean
Median
Max
Scale
Repair instances
565
–
–
–
Repositories
101
–
–
–
Failure categories
12
–
–
–
CI validation
CI jobs
2,818
5.0
2
51
CI steps
15,561
27.5
14
557
Failed jobs
1,574
2.8
1
46
Appendix
Table 5: Overview of CI-Repair-Bench . Total denotes the aggregate quantity across all 565 repair instances. Mean, median, and maximum are computed at the instance level. Failure types are not mutually exclusive; the percentage denotes the fraction of instances containing each type.
Scope
Instances
Structural
Lexical
Within repository
507
0.436
0.811
Overall benchmark
565
0.445
0.796
Appendix
Table 6: Mean nearest-neighbor repair recurrence in CI-Repair-Bench . Structural and lexical similarity are measured independently.
Model
Release
Params.
Native Context
Context Limit
Temp.
Input
Cache Read
Output
MiniMax-M2.5
Feb. 2026
229B
200K
200K
0.0
$0.15/M
$0.030/M
$1.20/M
DeepSeek-V4-Flash
Apr. 2026
284B / 13B active
1M
163,840
0.0
$0.03/M
$0.016/M
$0.32/M
Appendix
Table 7: Backbone models and inference settings used in our evaluation. Context limit denotes the maximum context configured in our experiments. Prices are reported in USD per one million tokens.
Table 9
Figure 2: Overlap of resolved instances among the three RETRACE reconstruction configurations on the 408-instance evaluation set using mini-SWE-agent.