Corrective Diffusion Language Models
Organizations: University of Wisconsin–Madison · Duke University
Abstract
While Diffusion Language Models (DLMs) are theoretically well-suited for iterative refinement due to their non-causal structure, off-the-shelf masked diffusion language models (MDLMs) often fail to reliably revise incorrect tokens in practice. The key challenge is that the model cannot tell correct from erroneous visible tokens: standard MDLM training supervises only masked positions, undermining confidence-guided refinement. We study corrective behavior in DLMs, the ability to assign lower confidence to incorrect tokens and iteratively refine them while preserving correct content. Under matched compute and data, we show that conventional masked-diffusion fine-tuning induces much less of this capability than a correction-oriented post-training principle that explicitly supervises visible corrupted tokens, which enables discriminative confidence and targeted refinement. We further find that also supervising clean visible tokens, as in BERT- or GIDD-style training, stops remask-based refinement from improving with additional steps. To evaluate it, we introduce the Code Revision Benchmark, a controllable, executable benchmark for error localization and in-place correction. The effect transfers from a 0.5B model to LLaDA-8B and persists, with a smaller margin, when corruptions are drawn from the base model's own most plausible predictions. The same error-aware confidence also makes parallel decoding more effective. In HumanEval(+) code completion at 0.5B, mathematical reasoning at 16B, and high-entropy parallel decoding with remasking (ParallelBench), models trained with our objective outperform standard masked-diffusion fine-tuning under matched compute, and outperform the base model on HumanEval(+), MATH, and the high-entropy ParallelBench average; at 8B, neither objective improves from-scratch generation over the base model. Code and data: https://github.com/zhangshuibai/CDLM
Figures & tables
| Model | gap | top-1 | top-5 | gap | top-1 | top-5 | gap | top-1 | top-5 | gap | top-1 | top-5 | gap | top-1 | top-5 |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Dream-7B-Base | 0.578 | 0.094 | 0.590 | 0.536 | 0.084 | 0.614 | 0.528 | 0.084 | 0.597 | 0.528 | 0.088 | 0.600 | 0.529 | 0.076 | 0.571 |
| LLaDA-8B-Base | 0.235 | 0.178 | 0.770 | 0.209 | 0.198 | 0.801 | 0.199 | 0.172 | 0.811 | 0.185 | 0.159 | 0.784 | 0.191 | 0.162 | 0.833 |
| Open-dCoder-0.5B | 0.106 | 0.147 | 0.509 | 0.100 | 0.222 | 0.655 | 0.085 | 0.243 | 0.701 | 0.089 | 0.325 | 0.759 | 0.093 | 0.348 | 0.795 |
| LLaDA-8B-Base | Gap | Hit@1 | Pass@1 | Pass@1 |
|---|---|---|---|---|
| Base (no fine-tuning) | 0.265 | 0.165 | 0.551 | 0.595 |
| + absorbing-only (MDLM) | 0.402 | 0.152 | 0.600 | 0.609 |
| + mixture (CDLM) | 0.777 | 0.608 | 0.725 | 0.623 |
| Code generation (HumanEval / HumanEval+, CDLM-0.5B) | ||||||||
|---|---|---|---|---|---|---|---|---|
| Vanilla decoding | ReMDM decoding | |||||||
| HumanEval | HumanEval+ | HumanEval | HumanEval+ | |||||
| Method | P@1 | P@10 | P@1 | P@10 | P@1 | P@10 | P@1 | P@10 |
| Base | 0.19 | 0.37 | 0.17 | 0.32 | 0.18 | 0.33 | 0.16 | 0.29 |
| MDLM | 0.19 | 0.38 | 0.17 | 0.34 | 0.20 | 0.38 | 0.17 | 0.34 |
| CDLM | 0.22 | 0.41 | 0.21 | 0.39 | 0.21 | 0.43 | 0.20 | 0.38 |
Appendix figures & tables40 assets
Supplementary material from the paper’s appendix.
Appendix
| Dimension | GIDD | CDLM |
|---|---|---|
| Framework | Generalized interpolating diffusion | Standard MDLM |
| Training | From scratch (as published) | Post-training of a pretrained MDLM |
| Loss | Unified ELBO (KL + Itakura–Saito) | MDLM loss + separable noise term ( ) |
| Noise schedule | Bell-shaped, varies with | Fixed mixture probability on visible tokens |
| Loss on clean tokens | Yes (all positions supervised) | No (only masked + corrupted) |
| Decoding | All tokens may change at every step | Remask-then-repredict, localized in-place |
| Dataset | MDLM | CDLM | Base |
|---|---|---|---|
| HumanEval | 0.157 | 0.257 | 0.168 |
| HumanEval+ | 0.182 | 0.242 | 0.196 |
| MBPP | 0.114 | 0.175 | 0.114 |
| MBPP+ | 0.137 | 0.226 | 0.120 |
| Average | 0.148 | 0.225 | 0.150 |
| Model | T=1 | T=2 | T=3 | T=4 |
|---|---|---|---|---|
| LLaDA-8B-Base | 0.550 | 0.571 | 0.577 | 0.580 |
| Dream-7B-Base | 0.528 | 0.548 | 0.402 | 0.427 |
| Open-dCoder-0.5B | 0.203 | 0.342 | 0.344 | 0.336 |
| Method | MBPP | MBPP+ | Avg | |||
|---|---|---|---|---|---|---|
| Pass@1 | Pass@10 | Pass@1 | Pass@10 | Pass@1 | Pass@10 | |
| MDLM (Vanilla) | 0.106 | 0.312 | 0.177 | 0.449 | 0.142 | 0.381 |
| CDLM (Vanilla) | 0.131 | 0.352 | 0.207 | 0.472 | 0.169 | 0.412 |
| Base (Vanilla) | 0.157 | 0.350 | 0.231 | 0.502 | 0.194 | 0.426 |
| MDLM Base | ||||||
| CDLM Base | ||||||
| Task | Base | MDLM | CDLM |
|---|---|---|---|
| Fixed / Remask | |||
| Low-Entropy (6 tasks) | |||
| Copy | 0.97 / 0.97 | 1.00 / 1.00 | 1.00 / 1.00 |
| Reverse | 0.60 / 0.63 | 1.00 / 1.00 | 1.00 / 1.00 |
| Sort | 0.15 / 0.19 | 0.61 / 0.61 | 0.60 / 0.64 |
| Insert Index | 0.23 / 0.24 | 1.00 / 1.00 | 0.99 / 0.99 |
| 0.5B code main comparison, seeds | 0.5B code sweep, CDLM ctw | LLaDA-8B LoRA | 16B MATH | 16B ParallelBench | |
| Training | |||||
| Base model | Open-dCoder-0.5B | Open-dCoder-0.5B | LLaDA-8B-Base | LLaDA2.0-mini-preview | LLaDA2.0-mini |
| Data | Nemotron-SFT-Code | Nemotron-SFT-Code | Nemotron-SFT-Code, 12,000 documents (3.55M tokens), each seen twice | Nemotron-Math-v2, 300K samples | 50,000 ParallelBench examples |
| Length | 2000 steps | 2000 steps | 2000 steps | 1 epoch (9,306 steps) | 6 epochs |
| Batch | , ; seeds 1234, 2025: | , | , | , | – |
| Optimizer | AdamW (0.9, 0.95), wd 0.01, clip 1.0 | same | AdamW (0.9, 0.95), wd 0.01 (not on 1-D), clip 1.0 | AdamW (0.9, 0.999), wd 0.1, clip 1.0 | – |
| Nemotron | HumanEval | HumanEval+ | MBPP | MBPP+ | |
|---|---|---|---|---|---|
| Sequences | 2000 | 164 | 163 | 499 | 378 |
| Mean tokens | 91.8 | 165.8 | 165.8 | 79.0 | 65.3 |
| Base | |||||
| MDLM | |||||
| CDLM | |||||
| MDLM Base |
| CDLM | Comparator | 95% CI | |||||
| CDLM vs. MDLM | |||||||
| 1 | 9,087 | 0.095 | 0.066 | 424/158 | |||
| 2 | 9,087 | 0.216 | 0.147 | 1032/399 | |||
| 3 | 9,087 | 0.225 | 0.163 | 1018/453 | |||
| 4 | 9,087 | 0.231 | 0.164 | 1052/442 | |||
| CDLM vs. Base | |||||||
| Comparator | ||||||
|---|---|---|---|---|---|---|
| MDLM | 1 | 3,180 | ||||
| 2 | 2,429 | |||||
| 3 | 1,478 | |||||
| 4 | 1,154 | |||||
| 5 | 846 | |||||
| Base | 1 | 3,174 |
| Dataset | |||||
|---|---|---|---|---|---|
| HumanEval | 1,473 | ||||
| ( ) | ( ) | ( ) | ( ) | ||
| HumanEval+ | 1,567 | ||||
| ( ) | ( ) | ( ) | ( ) | ||
| MBPP | 3,983 | ||||
| ( ) | ( ) | ( ) | ( ) |
| CDLM vs. MDLM ( ) | CDLM vs. Base ( ) | |||||
|---|---|---|---|---|---|---|
| 95% CI | 95% CI | |||||
| 1 | ||||||
| 2 | ||||||
| 3 | ||||||
| 4 | ||||||
| CDLM | CDLM ctw | 95% CI | |||||
|---|---|---|---|---|---|---|---|
| 1 | 3,213 | 0.185 | 0.196 | 206/244 | |||
| 2 | 3,213 | 0.214 | 0.198 | 334/283 | |||
| 3 | 3,213 | 0.211 | 0.190 | 335/266 | |||
| 4 | 3,213 | 0.205 | 0.190 | 323/272 |
| Pass@1 | |||||
| Arm | Seed (GPUs) | Gap | Hit@1 | ||
| CDLM | 42 (4) | 0.1888 | 0.2853 | 0.1636 | 0.2277 |
| 1234 (2) | 0.1829 | 0.2884 | 0.1497 | 0.2126 | |
| 2025 (2) | 0.1943 | 0.2836 | 0.1664 | 0.2295 | |
| mean SD | |||||
| MDLM | 42 (4) | 0.1386 | 0.2445 | 0.0976 | 0.1470 |
| Error type | Positions | Median rank | In top 5 | Geo. mean prob. | Ratio | Programs |
|---|---|---|---|---|---|---|
| Operator | 200 / 176 | 20.5 / 2 | 19.5% / 100% | / | 975.5 | 1,465 / 997 |
| Identifier | 131 / 98 | 39 / 2 | 14.5% / 100% | / | 494.5 | 934 / 728 |
| Literal | 155 / 200 | 3 / 2 | 81.3% / 100% | / | 13.2 | 814 / 1,036 |
| Random CRB | Plausible CRB | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| Metric | Base | MDLM | CDLM | Base | MDLM | CDLM | Retained | ||
| Gap | 0.099 | 0.094 | 0.158 | 0.064 | 0.054 | 0.051 | 0.084 | 0.033 | 52.1% |
| Hit@1 | 0.148 | 0.147 | 0.222 | 0.075 | 0.093 | 0.092 | 0.129 | 0.036 | 48.5% |
| Hit@3 | 0.337 | 0.343 | 0.452 | 0.109 | 0.242 | 0.240 | 0.309 | 0.069 | 63.2% |
| Hit@6 | 0.570 | 0.570 | 0.662 | 0.092 | 0.456 | 0.434 | 0.512 | 0.078 | 85.3% |
| Pass@1, | 0.140 | 0.140 | 0.185 | 0.044 | 0.099 | 0.087 | 0.116 | 0.029 | 66.2% |
| Gap | Hit@1 | Pass@1, | Pass@1, | ||||||
|---|---|---|---|---|---|---|---|---|---|
| Type | Arm | Rand. | Plaus. | Rand. | Plaus. | Rand. | Plaus. | Rand. | Plaus. |
| Operator | Base | 0.086 | 0.034 | 0.140 | 0.083 | 0.143 | 0.087 | 0.280 | 0.194 |
| MDLM | 0.080 | 0.029 | 0.137 | 0.076 | 0.144 | 0.075 | 0.264 | 0.168 | |
| CDLM | 0.146 | 0.049 | 0.221 | 0.101 | 0.181 | 0.085 | 0.307 | 0.187 | |
| retained | 31.1% | 30.0% | 26.7% | 45.4% | |||||
| Identifier | Base | 0.134 | 0.111 | 0.188 | 0.148 | 0.164 | 0.176 | 0.279 | 0.267 |
| Pass@1 | Modified | |||||
|---|---|---|---|---|---|---|
| Failing set | Arm | |||||
| Base-generated ( ) | Base | 0.000 (0) | 0.000 (0) | 0.000 (0) | 20.6% | 28.2% |
| MDLM | 0.001 (1) | 0.001 (1) | 0.001 (1) | 12.4% | 20.3% | |
| CDLM | 0.001 (1) | 0.001 (1) | 0.001 (1) | 16.0% | 19.8% | |
| CDLM-generated ( ) | Base | 0.001 (1) | 0.003 (3) | n/a | 23.6% | 30.1% |
| MDLM | 0.002 (2) | 0.001 (1) | n/a | 15.5% | 22.2% | |
| Set | Programs | Median | Quartiles | Within 4 | Mean length |
| Base-generated, HumanEval | 136 | 109 | 94–116 | 0 (0.0%) | 113 |
| Base-generated, HumanEval+ | 136 | 110 | 96–117 | 0 (0.0%) | 112 |
| Base-generated, MBPP | 404 | 41.5 | 26–97 | 0 (0.0%) | 61 |
| Base-generated, MBPP+ | 264 | 34 | 23–71 | 4 (1.5%) | 55 |
| Base-generated, all | 940 | 59 | 29–110 | 4 (0.4%) | 74 |
| CDLM-generated, all | 865 | 55 | 29–104 | 5 (0.6%) | 68 |
| Setting | Value |
|---|---|
| Base model | GSAI-ML/LLaDA-8B-Base , revision 0f2787f2 |
| Adapter | LoRA, rank 64, (scaling 2), dropout 0.05, on the 7 linear projections of each block |
| Frozen | all base weights, including embeddings and output layer |
| Trainable parameters | 167.77M (2.05%) |
| Corpus | Nemotron-SFT-Code: 12,000 documents, 3.55M tokens, each document seen twice |
| Sequence length | at most 4096 tokens, one document per sequence |
| Arm | HE | HE+ | MBPP | MBPP+ | HE | HE+ | MBPP | MBPP+ |
|---|---|---|---|---|---|---|---|---|
| Base | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 |
| MDLM | 5.6 | 78.7 | 0.4 | 0.5 | 9.9 | 88.5 | 1.5 | 1.6 |
| CDLM | 8.9 | 94.9 | 1.6 | 1.1 | 10.1 | 96.7 | 5.3 | 4.4 |
| HumanEval | HumanEval+ | MBPP | MBPP+ | |||||||||||
| Arm | op | id | lit | op | id | lit | op | id | lit | op | id | lit | Macro | |
| Raw | ||||||||||||||
| 1 | Base | 0.621 | 0.351 | 0.775 | 0.616 | 0.379 | 0.824 | 0.533 | 0.307 | 0.646 | 0.515 | 0.314 | 0.734 | 0.551 |
| 1 | MDLM | 0.638 | 0.527 | 0.697 | 0.129 | 0.117 | 0.131 | 0.606 | 0.442 | 0.655 | 0.640 | 0.473 | 0.650 | 0.475 |
| 1 | CDLM | 0.763 | 0.756 | 0.704 | 0.016 | 0.029 | 0.026 | 0.708 | 0.609 | 0.622 | 0.749 | 0.714 | 0.667 | 0.530 |
| 4 | Base | 0.674 | 0.412 | 0.817 | 0.682 | 0.437 | 0.843 | 0.558 | 0.333 | 0.673 | 0.596 | 0.368 | 0.746 | 0.595 |
| De-fenced | Raw | ||||
|---|---|---|---|---|---|
| Comparison | Difference | Difference | |||
| CDLM MDLM | 1 | ||||
| CDLM MDLM | 4 | ||||
| CDLM Base | 1 | ||||
| CDLM Base | 4 | ||||
| MDLM Base | 1 | ||||
| Arm | Steps | Clean | Corr. | Gap | Hit@1 | Hit@5 | Clean | Corr. | Gap | Hit@1 |
|---|---|---|---|---|---|---|---|---|---|---|
| Base | 0 | 0.970 | 0.705 | 0.265 | 0.165 | 0.771 | 0.988 | 0.791 | 0.197 | 0.325 |
| MDLM | 2000 | 0.970 | 0.567 | 0.402 | 0.152 | 0.874 | 0.990 | 0.771 | 0.219 | 0.460 |
| CDLM | 2000 | 0.969 | 0.192 | 0.777 | 0.608 | 0.980 | 0.983 | 0.866 | 0.117 | 0.153 |
| MDLM | 200 | 0.963 | 0.705 | 0.258 | 0.105 | 0.806 | 0.986 | 0.804 | 0.181 | 0.287 |
| CDLM | 200 | 0.962 | 0.173 | 0.790 | 0.530 | 0.970 | 0.989 | 0.889 | 0.099 | 0.196 |
| Operator ( ) | Identifier ( ) | Literal ( ) | ||||
|---|---|---|---|---|---|---|
| Arm | Gap | Hit@1 | Gap | Hit@1 | Gap | Hit@1 |
| Base | 0.269 | 0.220 | 0.133 | 0.096 | 0.376 | 0.135 |
| MDLM | 0.426 | 0.148 | 0.303 | 0.147 | 0.452 | 0.161 |
| CDLM | 0.794 | 0.660 | 0.857 | 0.801 | 0.681 | 0.355 |
| Hit@1 | Pooled | ||||||
|---|---|---|---|---|---|---|---|
| Arm | HE | HE+ | MBPP | MBPP+ | Hit@1 | Hit@5 | Gap |
| Base | 0.165 | 0.209 | 0.147 | 0.200 | 0.173 | 0.773 | 0.227 |
| MDLM | 0.152 | 0.345 | 0.443 | 0.480 | 0.393 | 0.892 | 0.379 |
| CDLM | 0.608 | 0.465 | 0.747 | 0.778 | 0.691 | 0.962 | 0.758 |
| Pass@1 | All 5 passes | Leading fence | |||||
|---|---|---|---|---|---|---|---|
| Arm | Dataset | (%), | |||||
| Base | HumanEval | 497 | 0.582 | 0.634 | 66.6 | 0 | 0 |
| Base | HumanEval+ | 511 | 0.607 | 0.655 | 56.4 | 0 | 0 |
| Base | MBPP | 1373 | 0.495 | 0.521 | 63.5 | 0 | 0 |
| Base | MBPP+ | 803 | 0.521 | 0.570 | 53.2 | 0 | 0 |
| MDLM | HumanEval | 497 | 0.647 | 0.645 | 44.5 | 0 | 4 |
| MBPP+ | |||||
|---|---|---|---|---|---|
| Model | Corpus | HumanEval+ | Primary | EvalPlus | Task filter |
| LLaDA-8B-Base | none | 0.280 | 0.529 | 0.447 | 0.381 |
| + MDLM | Nemotron-SFT-Code | 0.073 | 0.270 | 0.196 | 0.206 |
| + CDLM | Nemotron-SFT-Code | 0.073 | 0.246 | 0.196 | 0.217 |
| + MDLM | OpenCodeInstruct | 0.128 | 0.352 | 0.294 | 0.294 |
| + CDLM | OpenCodeInstruct | 0.091 | 0.315 | 0.249 | 0.315 |
| MBPP+ | |||
| Comparison | HumanEval+ | Primary | EvalPlus |
| Nemotron-SFT-Code | |||
| CDLM MDLM | ( ) | ( ) | ( ) |
| MDLM Base | ( ) | ( ) | ( ) |
| CDLM Base | ( ) | ( ) | ( ) |
| OpenCodeInstruct | |||
| Gap | Top-5 hit | Top-5 FP | Pass@1 | Pass@1 | ||
| 0.04 | 1,488 | 0.152 | 0.811 | 0.716 | 0.094 | 0.257 |
| 0.06 | 1,495 | 0.146 | 0.779 | 0.740 | 0.091 | 0.258 |
| 0.08 | 1,496 | 0.189 | 0.811 | 0.708 | 0.115 | 0.249 |
| 0.1 | 1,502 | 0.194 | 0.807 | 0.710 | 0.110 | 0.240 |
| 0.2 | 1,495 | 0.216 | 0.837 | 0.699 | 0.119 | 0.234 |
| 0.3 | 1,490 | 0.320 | 0.852 | 0.659 | 0.201 | 0.292 |
| Pass@1 | ||||||||
| Model | Corpus | Gap | Hit@1 | Hit@5 | ||||
| : 12 cells, 3,213 test-failing programs | ||||||||
| Base | none | 0.145 | 0.240 | 0.255 | 0.263 | 0.105 | 0.147 | 0.511 |
| MDLM-0.5B | Nemotron | 0.138 | 0.236 | 0.246 | 0.245 | 0.098 | 0.147 | 0.503 |
| CDLM-0.5B | Nemotron | 0.188 | 0.279 | 0.281 | 0.285 | 0.164 | 0.228 | 0.606 |
| MDLM-OCI | OCI | 0.140 | 0.223 | 0.235 | 0.239 | 0.099 | 0.145 | 0.520 |
| Benchmark | Decoder | Base | MDLM-OCI | CDLM-OCI | Difference [95% CI] |
|---|---|---|---|---|---|
| HumanEval | vanilla | 0.188 | 0.171 | 0.216 | [ , ] |
| HumanEval+ | vanilla | 0.167 | 0.153 | 0.196 | [ , ] |
| MBPP | vanilla | 0.157 | 0.121 | 0.116 | [ , ] |
| MBPP+ | vanilla | 0.231 | 0.190 | 0.203 | [ , ] |
| HumanEval | ReMDM | 0.182 | 0.160 | 0.221 | [ , ] |
| HumanEval+ | ReMDM | 0.164 | 0.144 | 0.198 | [ , ] |
| Pass@1, de-fenced | Pass@1, raw | ||||||
|---|---|---|---|---|---|---|---|
| Model | Gap | Hit@1 | Hit@5 | ||||
| LLaDA-8B-Base | 0.265 | 0.165 | 0.771 | 0.551 | 0.595 | 0.551 | 0.595 |
| + MDLM-OCI | 0.338 | 0.102 | 0.887 | 0.576 | 0.590 | 0.546 | 0.521 |
| + CDLM-OCI | 0.835 | 0.593 | 0.978 | 0.741 | 0.726 | 0.518 | 0.506 |
| Model | Type | Pass@1 | NFE (median) |
| gpt-4.1-nano (Plan A, full output) | AR | 0.802 | 89 |
| gpt-4.1-nano (Plan B, localize) | AR | 0.790 | 30 |
| Dream-7B-Base | DLM ( ) | 0.776 | 2 |
| LLaDA-8B-Base | DLM ( ) | 0.552 | 2 |
| LLaDA-8B CDLM † | DLM ( ) | 0.725 | 2 |
| Pass@1 | |||||
| Model | Gap | Hit@1 | Remask (%) | ||
| Base (Open-dCoder-0.5B) | 0.143 | 0.251 | 0.106 | 0.147 | 3.09 |
| MDLM (absorbing-only) | 0.137 | 0.237 | 0.095 | 0.140 | 2.95 |
| CDLM (ours) | 0.188 | 0.286 | 0.169 | 0.228 | 2.87 |
| PRISM head, | 0.132 | 0.167 | 0.087 | 0.111 | 5.72 |
| PRISM head, exact edit parity | 0.107 | 0.134 | 0.087 | 0.111 | 2.87 † |
| Feature | Passes | Hit@1 | Hit@5 | AUROC |
|---|---|---|---|---|
| Single-pass confidence | 1 | 0.359 | 0.772 | 0.891 |
| Flip rate | 5 | 0.150 | 0.480 | 0.740 |
| Convergence step | 5 | 0.142 | 0.489 | 0.742 |
| Confidence variance | 5 | 0.253 | 0.622 | 0.840 |
| Minimum confidence | 5 | 0.296 | 0.708 | 0.893 |
| Confidence slope (rising suspicious) | 5 | 0.434 | 0.752 | 0.735 |
| HumanEval | HumanEval+ | MBPP | MBPP+ | Pooled | ||||||
|---|---|---|---|---|---|---|---|---|---|---|
| Feature | Hit@1 | AUROC | Hit@1 | AUROC | Hit@1 | AUROC | Hit@1 | AUROC | Hit@1 | AUROC |
| Single-pass | 0.359 | 0.891 | 0.176 | 0.880 | 0.324 | 0.873 | 0.340 | 0.857 | 0.308 | 0.874 |
| Flip rate | 0.150 | 0.740 | 0.169 | 0.765 | 0.135 | 0.714 | 0.161 | 0.704 | 0.149 | 0.725 |
| Convergence step | 0.142 | 0.742 | 0.095 | 0.756 | 0.133 | 0.721 | 0.146 | 0.707 | 0.131 | 0.727 |
| Conf. variance | 0.253 | 0.840 | 0.269 | 0.881 | 0.215 | 0.799 | 0.236 | 0.783 | 0.235 | 0.816 |
| Min. confidence | 0.296 | 0.893 | 0.189 | 0.895 | 0.279 | 0.868 | 0.297 | 0.856 | 0.270 | 0.874 |
| Hit@1 | AUROC | |||||
| Model | Single-pass | Best traj. | Single-pass | Best traj. | ||
| Open-dCoder-0.5B (base) | 0.242 | 0.272 c | +0.031 | 0.840 | 0.892 m | +0.052 |
| MDLM, 2,000 steps | 0.263 | 0.267 c | +0.004 | 0.840 | 0.882 m | +0.042 |
| CDLM, 2,000 steps | 0.383 | 0.395 c | +0.012 | 0.873 | 0.906 m | +0.034 |
| CDLM, 10,000 steps | 0.433 | 0.478 m | +0.045 | 0.839 | 0.886 m | +0.047 |
| CDLM ctw | 0.472 | 0.472 c | +0.001 | 0.852 | 0.884 m | +0.031 |
| Artifact | Repository | Revision |
| Open-dCoder-0.5B, full fine-tuning | ||
| Base model (third party) | fredzzp/open-dcoder-0.5B | d0d86d5b |
| MDLM-0.5B | Open-Dcoder-0.5B-baseline-mdm-step2000 | fa5eef96 |
| CDLM-0.5B | Open-Dcoder-0.5B-mixture-mdm-step2000 | 5a7170e7 |
| MDLM-OCI | Open-Dcoder-0.5B-MDLM-OpenCodeInstruct | 535b3693 |
| CDLM-OCI | Open-Dcoder-0.5B-CDLM-OpenCodeInstruct | 8eb87fe2 |