Diffusion large language models (dLLMs) are appealing for code generation because they can update many tokens in parallel, yet many are optimized for sequence synthesis rather than edit prediction. This setting is poorly matched to modern AI-assisted IDEs, where edit prediction must infer the next code change from recent edits, the current file, and cursor-local context. Such changes are usually sparse and preserve most source tokens, so regenerating a full code segment wastes computation. We propose Diff Diffusion, a diffusion-inspired LLM framework for code editing that predicts edit paths from an original snippet to its revised form. Diff Diffusion represents denoising as structured edit operations, and combines this formulation with multi-stage training and efficient parallel decoding. By focusing computation on changed regions, Diff Diffusion supports accurate low-latency edits and multi-position parallel modification. Across public edit-completion benchmarks, edit-prediction evaluations, and online validation, Diff Diffusion matches or leads autoregressive baselines while delivering 2.78 times faster inference on public edit-completion benchmarks and a 39.0% P50 model-latency reduction in production. This work provides a new modeling perspective for AI applications that require high accuracy and low latency.
Figures & tables
Figure 1: Diff Diffusion models real-time IDE code edit prediction.
Figure 2: Diff Diffusion framework.
Model
Result Accuracy (%)
Steps per Sample
Inference Time per Sample (ms)
Speedup Relative to AR
AR
83.18
14.62
403
−
Diff Diffusion
81.33
3.90
145
2.78×
Table 1: Performance comparison on the C3PO edit-completion task.
Model
Act (%) ↑
Stay (%) ↑
Multi-line (%) ↑
Roll-back (%) ↓
Delete (%) ↓
AR
45.90
91.73
68.42
9.0
1.0
Mercury-Coder
31.97
54.32
36.84
52.5
19.5
Diff Diffusion + CT
41.80
89.21
68.42
7.0
0.5
Diff Diffusion + CT + SFT
49.18
89.21
73.68
9.5
0.5
Diff Diffusion + CT + SFT + DPO
49.18
92.45
68.42
7.5
0.5
Diff Diffusion + CT + SFT + AGRPO
50.00
96.40
73.68
10.0
0.5
Table 2: Results on the IDE edit-prediction evaluation set.
Method
Edit Range (tokens)
Output Length
Decoding Steps
Total Time (ms)
AR
140
126
126
3964
Diff Diffusion
140
110
4.90
415
Diff Diffusion + PIE
140
110
5.00
420
Diff Diffusion
1024
1008
7.56
784
Diff Diffusion + PIE
1024
1012
8.78
676
Table 3: Inference speed on the edit-prediction evaluation set.
Model
Acceptance rate
CPO
Accepted lines
P50 model latency
AR
23.8%
1.06
93.6
116 ms
Diff Diffusion
24.0%
1.05
102.6
70.8 ms
Table 4: Production A/B results for IDE edit prediction.
Figure 3: Latency by edit-distance bucket.
Appendix figures & tables2 assets
Supplementary material from the paper’s appendix.
Appendix
Method
Initial State
Edit Actions
Edit Task Focused
Main Goal
Production Grade
Seed Diffusion [ 38 ]
Masked sequence
Mask-to-token
Partial
Quality + latency
No
LLaDA2.1 [ 4 ]
Masked sequence
Mask-to-token, token replacement
No
Latency + quality
No
LLaDA2.2 [ 3 ]
Masked sequence
Keep, substitute, delete, insert
Partial
Self-correction
No
Edit Flows [ 15 ]
Random sequence
Insert, delete, substitute
No
Variable length
No
DID [ 11 ]
Masked sequence
Delete, insert
No
Variable length
No
DreamOn [ 44 ]
Masked span
Expand, delete, fill mask
Partial
Variable-length infilling
No
Appendix
Table 5: Representative dLLMs with edit operations.
Dataset / Benchmark
Role
Scope
Scale
CommitPack subset
Stage I edit-path pretraining
Git commit diffs in Python, C++, Java, JavaScript, Go, and Rust
87B tokens
Edit prediction training data
Stage II / III IDE post-training
High-quality next-edit examples collected from multiple scenarios
30M examples
C3PO Edit Completion
Public benchmark and SFT data
C# code edit examples from 53 GitHub projects
39K train / 5K eval examples
Edit prediction evaluation set
In-house IDE benchmark
Act, Stay, Multi-line, Roll-back, and Delete cases
Large Language Models (LLMs) are increasingly used for code editing, yet the prevalent full-code generation paradigm suffers from severe efficiency bottlenecks, posing challenges for interactive coding assistants that demand low latency and cost. Despite the predominant focus on scaling model capabilities, the edit format itself has been largely overlooked in model training. In this paper, we begin with a systematic study of conventional diff formats and reveal that fragile offsets and fragmented hunks make generation highly unnatural for LLMs. To address it, we introduce BlockDiff and FuncDiff, two structure-aware diff formats that represent changes as block-level rewrites of syntactically coherent units such as control structures and functions. Furthermore, we propose AdaEdit, a general adaptive edit strategy that trains LLMs to dynamically choose the most token-efficient format between a given diff format and full code. Extensive experiments demonstrate that AdaEdit paired with structure-aware diff formats consistently matches the accuracy of full-code generation, while reducing both latency and cost by over 30% on long-code editing tasks.
Wei Cheng, Yongchang Cao, Chen Shen +4
State Key Laboratory for Novel Software Technology, Nanjing University, China · Tongyi Lab, Alibaba Group, China
Code editing requires a model to decide where to make changes, generate the new content, and preserve everything else. We study how masked diffusion language models divide these responsibilities across four editing interfaces: whole-file rewriting, search-and-replace, locate-then-infill, and token-level editing. Experiments on CanItEdit reveal a composition gap: diffusion models can generate coordinated changes when the correct edit locations are supplied, but much of this capability is lost when those locations must be predicted. Access to the intact original code helps the model fill multiple edit regions, yet does not resolve the difficulty of selecting those regions. By varying the editable regions while holding the generation model and decoding procedure fixed, we identify two distinct requirements for successful editing: covering every required change and placing precise boundaries around it. Missing a required region prevents the corresponding change, while widening regions to ensure coverage can sharply reduce success by requiring unchanged code to be regenerated. A sentence-level Wiki editing probe shows the same qualitative gap between supplied and predicted locations beyond code. These findings show why strong infilling capability alone does not ensure reliable editing: the interface must expose all required changes while limiting regeneration of unchanged code.
Xijia Tao, Ziru Liu, Shansan Gong +7
The University of Hong Kong · Huawei Technologies Ltd. · City University of Hong Kong +1
Imperfections in AI-generated code require that software developers modify the generated code manually, or by re-prompting an AI programming assistant. Manual code edits provide more realistic and granular information on editing behavior than Git commits, which only contain final successful code snippets. Yet, due to a lack of high-quality, realistic code editing data, LLMs are mostly trained on publicly available Git data (e.g., commits). To address this gap, we introduce DECODE (Developer Edits of Code Dataset), a dataset of 53.6K real-world in-IDE code edits of AI-generated code in Python, TypeScript, and JavaScript, sourced from 1K+ developers. First, we demonstrate the utility of DECODE for data analysis, obtaining insights on when, why, and how AI-generated code is edited. We find that most edits occur within the first 15 minutes after accepting an AI completion, resulting in the removal of AI completions in 31% of edit trajectories. Second, we use DECODE to benchmark the ability of LLMs to predict code edits. We find that finetuning on DECODE enables open-source 3B models to perform code edit prediction tasks significantly better than frontier LLMs. We then discuss implications of this work, emphasizing the necessity of developer-centric machine learning approaches for future AI programming assistants.