LEDGE: Separating Critic and Editor Failures in Text-to-Image Self-Correction
Organizations: University of Maryland, College Park
Abstract
Text-to-image (T2I) self-correction combines a vision-language model (VLM) critic that diagnoses errors and proposes instructions with an image editor that executes them. End-to-end evaluation obscures whether failures arise from incorrect diagnoses or an editor unable to execute valid instructions. We introduce LEDGE (Learning Error Diagnosis in Generation and Editing), a framework that separates these bottlenecks. LEDGE compiles oracle edit instructions from GenEval while holding the generated images and the editor fixed, isolating the editor's correction capability from the critic's diagnostic accuracy. On Stable Diffusion 3.5 Large, oracle instructions raise GenEval from 72.0% to 81.2%, versus 74.1% for the baseline VLM critic. Yet 435 of 644 failed images remain unrepaired, exposing a substantial editor gap. We also use oracle labels from disjoint training prompts to supervise the critic. Oracle supervision raises exact diagnosis from 33.3% (the same VLM with our structured output, untrained) to 70.8%. With the same editor, the resulting critic improves GenEval from 42.2% (uncorrected) to 59.4% on Stable Diffusion 1.5, against 50.9% for the baseline critic, and from 72.0% to 76.0% on Stable Diffusion 3.5 Large, despite training only on Stable Diffusion 1.5. Improvements transfer to unseen object classes, a second editor, and independent T2I-CompBench judges and prompts. A small human evaluation confirms the ranking in estimated net corrections but suggests a smaller gain than detector-based evaluation. LEDGE thus distinguishes two limits of self-correction: knowing what to fix and being able to fix it.
Figures & tables
| Instructions from | Macro | Micro | Rep. | Reg. | |
|---|---|---|---|---|---|
| None (original) | 42.2 | 40.1 | – | – | – |
| Generic instruction | 49.9 | 48.4 | 246 | 63 | |
| Baseline (Qwen) | 50.9 | 49.3 | 226 | 23 | – |
| Molmo + rules | 52.4 | 50.9 | 301 | 64 | |
| Molmo, oracle-sup. | 56.5 | 55.2 | 345 | 12 | |
| Ours (oracle-sup.) | 59.4 | 58.3 | 420 | 18 |
| Qwen decision | Correct (888) | Failed (1,324) |
|---|---|---|
| keep | 789 (88.9%) | 164 (12.4%) |
| edit | 65 (7.3%) | 698 (52.7%) |
| review | 34 (3.8%) | 462 (34.9%) |
| Correct images | Failed images | ||||
|---|---|---|---|---|---|
| Critic | Keep | Edit | Edit | Exact | Any |
| Baseline (single error) | 88.9 | 7.3 | 52.7 | – | 42.3 † |
| Structured zero-shot | 77.5 | 22.0 | 91.4 | 33.3 | 43.6 |
| Ours (oracle-sup.) | 93.5 | 6.5 | 89.0 | 70.8 | 79.3 |
| Prompt-only rule | 0.0 | 100.0 | 100.0 | 49.1 | 57.8 |
| Task | Orig. | Qwen | Molmo | Ours | Oracle | Closed |
|---|---|---|---|---|---|---|
| Single object | 95.9 | 96.6 | 95.0 | 97.8 | 99.4 | 44% |
| Two objects | 34.3 | 58.1 | 53.0 | 69.7 | 76.5 | 63% |
| Counting | 37.2 | 40.0 | 45.0 | 43.8 | 49.1 | 41% |
| Colors | 73.7 | 81.9 | 78.7 | 84.0 | 91.5 | 22% |
| Position | 4.5 | 10.8 | 28.5 | 29.8 | 32.5 | 87% |
| Color attr. | 7.8 | 18.3 | 14.3 | 31.5 | 37.0 | 71% |
| Instructions from | Macro | Micro | Rep. | Reg. | |
|---|---|---|---|---|---|
| None (SD3.5-L) | 72.0 | 70.9 | – | – | – |
| Baseline (Qwen) | 74.1 | 73.1 | 59 | 11 | |
| Ours (oracle-sup.) | 76.0 | 75.1 | 109 | 16 | |
| Oracle (reference) | 81.2 | 80.3 | 209 | 0 |
| Instructions | Color | Object | Spatial | Numer. | Mean |
|---|---|---|---|---|---|
| None (original) | 50.75 | 68.63 | 1.46 | 68.95 | 47.45 |
| Baseline critic | 61.68 | 79.80 | 3.53 | 70.56 | 53.89 |
| Ours | 75.81 | 85.52 | 10.36 | 72.98 | 61.17 |
| Oracle (GenEval) | 77.07 | 86.58 | 10.32 | 72.18 | 61.54 |
| Editor | Instructions | Macro | Micro | Rep. | Reg. |
|---|---|---|---|---|---|
| VAREdit | Baseline | 50.9 | 49.3 | 226 | 23 |
| VAREdit | Ours | 59.4 | 58.3 | 420 | 18 |
| VAREdit | Oracle | 64.3 | 63.3 | 512 | 0 |
| FLUX.1 Kontext | Baseline | 51.6 | 49.9 | 231 | 15 |
| FLUX.1 Kontext | Ours | 61.5 | 60.3 | 450 | 5 |
| FLUX.1 Kontext | Oracle | 64.9 | 63.7 | 522 | 0 |
Appendix figures & tables11 assets
Supplementary material from the paper’s appendix.
Appendix
| Generator | Macro | Micro | Single | Two obj. | Count | Colors | Position | Color attr. |
|---|---|---|---|---|---|---|---|---|
| SD1.5 | 42.23 | 40.14 | 95.94 | 34.34 | 37.19 | 73.67 | 4.50 | 7.75 |
| SDXL | 54.36 | 52.94 | 99.38 | 74.24 | 39.06 | 83.24 | 11.00 | 19.25 |
| FLUX.1-dev | 66.75 | 65.19 | 98.75 | 81.06 | 74.06 | 77.39 | 21.25 | 48.00 |
| SD3.5-Large | 70.99 | 69.85 | 99.06 | 89.65 | 70.00 | 83.24 | 26.00 | 58.00 |
| SD3.5-Large, copy | 72.02 | 70.89 | 99.06 | 89.90 | 71.88 | 84.04 | 27.25 | 60.00 |
| Pilot tag | Images | Keep | Edit | Two edits |
|---|---|---|---|---|
| Position | 448 | 43 | 405 | 0 |
| Counting | 376 | 37 | 339 | 175 |
| Color attribution | 322 | 28 | 294 | 103 |
| Color + position | 272 | 4 | 268 | 89 |
| Colors | 74 | 62 | 12 | 0 |
| Two objects | 38 | 17 | 21 | 0 |
| Base model | Qwen3-VL-8B-Instruct (bf16), vision encoder frozen |
|---|---|
| LoRA | rank 16, , dropout 0.05; q, k, v, o, gate, up, down of the language model |
| Optimization | AdamW, lr , no weight decay, 5% warmup then cosine, grad-norm clip 1.0 |
| Batch | 1 example, 8 accumulation steps |
| Data | 1,530 labeled images; correct images repeated (1,912 examples) |
| Schedule | 3 epochs, 717 optimizer steps; adapters saved every 50 steps |
| Selection | Best DEV exact match with 10% of correct DEV images edited step 400 |
| Instructions | Color | Numer. | Spatial | Mean |
|---|---|---|---|---|
| None (original) | 36.87 | 45.50 | 11.43 | 31.27 |
| Baseline critic | 48.88 | 45.89 | 12.27 | 35.68 |
| Ours | 66.38 | 52.73 | 18.40 | 45.83 |
| Method | Macro | Micro | Rep. | Reg. | Single (6) | Two obj. (75) | Count. (58) | Colors (30) | Pos. (106) | C. attr. (99) |
|---|---|---|---|---|---|---|---|---|---|---|
| None (original) | 39.7 | 37.7 | – | – | – | – | – | – | – | – |
| Oracle-flagged resample | 49.7 | 47.7 | 60 | 0 | 66.7 | 26.7 | 25.9 | 40.0 | 2.8 | 6.1 |
| Baseline critic, resample | 43.6 | 41.5 | 33 | 10 | 0.0 | 13.3 | 22.4 | 20.0 | 0.9 | 3.0 |
| Ours, resample | 45.8 | 43.8 | 43 | 6 | 33.3 | 22.7 | 22.4 | 13.3 | 2.8 | 4.0 |
| Baseline critic, edit | 49.0 | 47.5 | 67 | 8 | 33.3 | 42.7 | 12.1 | 40.0 | 5.7 | 8.1 |
| Ours, edit | 57.6 | 56.7 | 118 | 4 | 50.0 | 57.3 | 10.3 | 40.0 | 32.1 | 20.2 |
| Category | SD1.5 | SD3.5 |
|---|---|---|
| Unrepaired / failures | 812 / 1,324 | 435 / 644 |
| Failed insertion | 56.3% | 29.0% |
| Incorrect attribute modification | 17.5% | 14.7% |
| Failed move | 18.0% | 48.7% |
| Unintended change | 9.5% | 7.6% |
| Split | Orig. | Base. | Molmo | Ours | Oracle | Closed |
|---|---|---|---|---|---|---|
| Unseen (1,428) | 41.7 | 50.3 | 55.3 | 59.1 | 64.4 | 63% |
| Mixed (464) | 15.5 | 30.2 | 40.4 | 43.2 | 47.8 | 74% |
| Seen (320) | 42.1 | 49.8 | 55.2 | 60.6 | 65.8 | 67% |
| All (2,212) | 42.2 | 50.9 | 56.5 | 59.4 | 64.3 | 63% |