Code transformation is a foundational capability in the software development process, where its effectiveness relies on constructing a high-quality code representation to characterize the input code semantics and guide the transformation. Existing approaches treat code transformation as an end-to-end learning task, leaving the construction of the representation needed for semantic reasoning implicit in model weights or relying on rigid compiler-level abstractions. We present SemRep, a framework that improves code transformation through generative code representation learning. Our key insight is to employ the semantics-preserving transformations as the intermediate representation, which serves as both a generative mid-training task and the guidance for subsequent instruction-specific code transformations. Across general code editing and optimization tasks (e.g., GPU kernel optimization), SemRep outperforms the extensively finetuned baselines with strictly the same training budget by 6.9% in correctness, 1.1x in performance, 13.9% in generalization, and 6.7% in robustness. With the improved exploration of diverse code transformations, SemRep is particularly amenable to evolutionary search. Combined with an evolutionary coding agent, SemRep finds optimizations that 685B larger-weight baselines fail to discover while achieving the same performance with 25% less inference compute.
Figures & tables
Figure 1: A concrete example showing how SemRep optimizes CUDA kernels in Kernelbench ( Ouyang et al., 2025 ) . SemRep first generates a semantically equivalent code that explicitly exposes the computation of MSE loss, and then transforms it into a more efficient implementation that optimizes memory access by replacing separate passes over large arrays with a single pass, where each thread computes its global index, reads inputs into fast registers for local computation, and fuses multiple operators into a single kernel execution. The existing approach attempts to directly optimize MSE but fails on several core operations.
Figure 2: SemRep framework. The upper shows how SemRep trains a model to instill semantics understanding through the generative code representation learning. SemRep explicitly encourages diverse semantically equivalent implementations, e.g., both faster and slower code variants, even when editing objectives are not fully satisfied. The total training budget B is fixed to ensure the fair comparison to those finetuning-only baselines. α modulates the training budget allocated to Stage 1 and Stage 2. The lower figure shows how SemRep performs iterative inference.
Model
Model Size
Pass@1 ↑
Closed Weight Models
Claude Sonnet 4
-
66.67
GPT o4-mini
-
57.41
Gemini 2.5 Pro
-
54.63
Open Weight Models
SemRep QwQ-32B (Ours)
32B
57.41
Table 1: Resolve rate (pass@1, %) of SemRep and other representative models on EditBench ( Chi et al., 2025 ) leaderboard using Core set. SemRep uses QwQ-32B as a base model, and we report the single run for comparability.
Correctness ↑
Speedup ↑
fast 1∗↑
fast 1.5∗↑
best@16
avg@16
best@16
avg@16
best@16
avg@16
best@16
avg@16
QwQ-32B
33
4.13
1.25
1.02
14
2.06
4
0.69
GPT 4o-mini
47 +14
17.13 +13
1.27 +0.02
1.12 +0.1
22 +8
7.75 +5.69
8 +4
2.13 +1.44
Kevin-32B
65 +32
19.63 +15.5
1.36 +0.11
1.1 +0.08
21 +7
25 +22.94
9 +5
2.69 +2
SemRep (Ours)
93 +60
37.63 +33.5
2.87 +1.62
1.21 +0.19
72 +58
21.06 +19
12 +8
3.13 +2.44
Table 2: Main results for domain-specific code editing on KernelBench.
KernelBench
EditBench
Correctness ↑
Speedup ↑
fast 1↑
fast 1.5↑
Pass@1
Pass@16
best@16
avg@16
best@16
avg@16
best@16
avg@16
best@16
avg@16
Baseline
74
12.56
1.33
1
45
6.75
8
0.94
53.70
55.56
+ SemRep TTS
59
15.19
1.47
1.13
28
8.19
11
3.62
56.48
62.04
+ SemRep -trained
79
25.75
1.48
1.25
35
13.5
11
5.75
55.56
62.96
+ Both (Ours)
93
37.63
2.87
1.21
72
21.06
12
3.13
57.41
66.67
Table 3: Ablations on KernelBench and EditBench.
Correctness ↑
Speedup ↑
fast 1↑
fast 1.5↑
Model
best@16
avg@16
best@16
avg@16
best@16
avg@16
best@16
avg@16
Kevin-32B
56
10.81
1.51
1.14
10
1.88
6
1.5
Finetuned QwQ-32B
61 +5
12.63 +1.82
1.7 +0.19
1.13 +0.01
15 +5
3.94 +2.06
8 +2
2.08 +0.58
SemRep (Ours)
81 +25
20.38 +9.57
1.62 +0.11
1.25 +0.11
17 +7
4.13 +2.25
8 +2
2.25 +0.75
Table 4: Comparing SemRep to Baronio et al. (2025) against unseen hardware devices, i.e., H200, for generalization by measuring performance on KernelBench ( Ouyang et al., 2025 ) .
Model
Pass@1 ↑
Consistency ↑
QwQ-32B
47.22
76.85
Finetuned QwQ-32B
50.93 +3.71
75.00 -1.85
GPT o4-mini
53.70 +6.48
83.33 +6.48
SemRep (Ours)
53.70 +6.48
88.89 +12.04
Table 5: Comparing SemRep -trained model to state-of-the-art models against semantics-preserving code transformations.
Figure 3: Scaling the number of parallel samples on KernelBench.
Figure 4: An example in EditBench showing how (a) direct transformation and (b) SemRep generate with “run these in parallel”.
Figure 5: Best results over optimization steps with OpenEvolve for the minimum spanning tree and circle packing problem.
Appendix figures & tables1 asset
Supplementary material from the paper’s appendix.
Appendix
Figure 6: Visualized trajectories of circle packing optimization, illustrating the iterative refinement of configurations across different models.