High-quality source-target image pairs with precise editing instructions are essential for instruction-guided image editing, yet constructing such training triplets at scale remains costly. Recent pipelines often rely on vision-language models to synthesize editing instructions automatically, but we find that strong VLMs still struggle to describe visual transformations between image pairs. In particular, they exhibit three recurring failure modes: orientation inconsistency, viewpoint ambiguity, and missing fine-grained attributes. In a human evaluation on 400 image pairs, several open-source VLM baselines produce critical-error rates above 47%, making many synthesized instructions unsuitable for downstream training. To address this, we propose EditCaption, a two-stage post-training pipeline for image editing instruction synthesis. First, we construct a 100K supervised fine-tuning dataset through GLM-based auto-captioning, EditScore filtering, and human refinement. Second, we collect 10K human-annotated preference pairs, where each rejected instruction is labeled with its primary error type and severity. Based on this dataset, we propose Hardness-Adaptive Error-Aware DPO (HAE-DPO), a task-adapted DPO objective that introduces an adaptive margin based on human-labeled severity, failure-mode type, and reference-model hardness. Experiments across three benchmarks demonstrate that our 235B model with SFT+HAE-DPO achieves state-of-the-art performance among open-source and closed models, scoring 4.720 on Eval-400, 4.672 on HQ-Edit, and 4.651 on ByteMorph-Bench -- surpassing Gemini-3-Pro on all three. Human evaluation confirms critical error rates drop from 47.75% to 17.50%, with correct rates improving from 41.75% to 70.25%, surpassing Gemini-3-Pro (66.00%).
Figures & tables
Figure 1: Overview of our approach. (a) Task & Failure Modes: Given a source-target image pair, VLMs must generate accurate editing instructions but suffer from three systematic failures: directional confusion, viewpoint ambiguity, and missing fine-grained detail. (b) HAE-DPO: SFT model outputs serve as rejected samples; human-corrected instructions as chosen samples. Each rejection is annotated with an error type and severity score, enabling HAE-DPO to modulate the preference margin beyond uniform DPO.
Figure 2: Overview of Data Distribution. Our training dataset is divided into three categories: Semantic Editing (content-based modifications), Stylistic Editing (aesthetic adjustments), and Structural Editing(spatial arrangement and composition). Our data collection strategy ensures a balance of diversity and quality throughout the training process, providing comprehensive coverage and precise annotations to foster robust model training.
Figure 3: The three-step SFT data construction pipeline.
Dataset
#Samples
Characteristics
In-house Test Set
400
Diverse editing operations, covering spatial, viewpoint, and attribute changes
ByteMorph-Bench
600
Non-rigid motion transformations with spatial and compositional challenges
HQ-Edit
500
High-resolution images with rich details and comprehensive editing instructions
Table 1: Summary of evaluation datasets
Model
Correct ↑
P0 ↓
P1
P2
Closed-source models
Gemini-3-Pro
66.00%
21.00%
12.00%
1.00%
GPT-4.1
48.75%
42.25%
8.25%
0.75%
Open-source baseline models
GLM-4.5V (106B-A12B)
38.60%
51.20%
9.20%
1.00%
Qwen3-VL-235B-A22B (base)
41.75%
47.75%
8.50%
2.00%
Table 2: Human evaluation results on 400 image pairs. Each model generates one instruction per pair. P0: critical errors (instruction unusable); P1: moderate errors; P2: minor issues. Higher Correct and lower P0 error rates indicate better performance.
Eval-400 (In-house)
HQ-Edit
ByteMorph-Bench
Model
S
Acc
Comp
Clar
S
Acc
Comp
Clar
S
Acc
Comp
Clar
Closed-source models
Gemini-3-Pro
4.706
4.70
4.85
4.43
4.658
4.54
4.89
4.43
4.522
4.55
4.70
4.11
GPT-4.1
4.220
4.03
4.60
3.84
4.507
4.57
4.86
3.68
3.412
3.27
3.68
3.14
Open-source baseline models
Qwen3.5-397B-A17B
4.380
4.30
4.62
4.06
4.383
4.07
4.85
4.07
3.867
3.71
4.16
3.61
Table 3: Objective evaluation results on three benchmarks. Scores are reported as weighted composites ( S=0.4×Acc+0.4×Comp+0.2×Clar , scale 0–5). Best and second-best scores are highlighted in bold and underline, respectively. † marks our fine-tuned models.
Eval-400
HQ-Edit
ByteMorph
Components
S
Acc
Comp
Clar
S
Acc
Comp
Clar
S
Acc
Comp
Clar
None (Standard DPO)
4.662
4.63
4.85
4.34
4.598
4.46
4.89
4.29
4.564
4.68
4.72
4.02
Only sic(ti) (severity-type)
4.698
4.70
4.86
4.37
4.648
4.55
4.90
4.34
4.612
4.73
4.75
4.10
Only hi (hardness)
4.710
4.71
4.88
4.37
4.664
4.58
4.89
4.38
4.630
4.74
4.77
4.13
HAE-DPO
4.720
4.72
4.89
4.38
4.672
4.61
4.87
4.40
4.651
4.75
4.79
4.17
Table 4: Ablation study on HAE-DPO margin components using Qwen3-VL-235B-A22B with the SFT stage fixed. The term sic(ti) denotes the severity-type product, where si is the error severity and c(ti) is the failure-type weight. The term hi denotes model-perceived hardness. Rows “Only sic(ti) ” and “Only hi ” add the corresponding margin component alone, while HAE-DPO uses both. S=0.4× Acc +0.4× Comp +0.2× Clar.
Model
ImgEdit ↑
GEdit-EN ↑
GEdit-CN ↑
Step1X-Edit
3.062
6.444
6.658
+ Base VLM captioner
3.358
6.583
6.774
+ EditCaption (ours)
3.882
7.158
7.312
Table 5: Downstream image editing evaluation on ImgEdit-Bench and GEdit-Bench (EN and CN). All models are fine-tuned from Step1X-Edit on 1M triplets annotated by different captioning sources. Scores are overall ratings evaluated by GPT-4.1.
Appendix figures & tables5 assets
Supplementary material from the paper’s appendix.
Appendix
ID
Severity
Category
Problem Description
Examples / Notes
1
P0
(Subject) Text Recognition / Editing Error
Instruction conflicts with text in image; text unchanged, incorrectly modified, garbled, or misspelled.
Instruction: “change sign to ‘OPEN’ ”, but target still shows “SHOP”; or text becomes distorted/garbled.
P1
(Background) Text Recognition / Editing Error
Same as above.
Same as above.
2
P0
(Subject) Instruction or Attribute Error
Primary task instruction or attribute is incorrect; or affected region exceeds 1/10 of total image area. Covers: color, shape, material, style, quantity, etc.
“Change black skirt to blue” but result is grey; “Change two cats to three” but target still has two; “Enlarge to 15cm” or “Rotate 37 degrees” (unverifiable from image).
P1
(Non-subject) Instruction or Attribute Error
Non-primary task; affected area below 1/10 of image. Same instruction types as above.
Same as above.
3
P0
(Subject) Instruction or Content Omission
Primary task instruction or attribute is missing. Includes: missing add/remove action; missing specific attributes (color, position) when adding/deleting; ambiguous editing target.
Original has no hat → target has hat, but “add hat” not mentioned; “Add flower” without specifying color; “Delete person” when multiple people are present; “Remove old flower + add new flower” should use replace ; “Put object somewhere” should use move not add ; missing description of border/frame edits.
P1
(Non-subject) Instruction or Content Omission
Non-primary task; affected area below 1/10 of image. Same instruction types as above.
Same as above.
Appendix
Table 6: Annotation Quality Checklist for Image Editing Instructions
Figure 4: Qualitative examples of generated editing instructions. Given the same source-target image pairs, our model produces instructions that are more spatially precise and complete compared to Gemini-3-Pro and GLM4.5V. Green denotes key attributes where our model outperforms baselines; Red denotes errors or omissions.
Images
Annotation
[Primary Changes] 1. The dress color changes from grey to pink. 2. The shoulder strap style changes from wide straps to thin spaghetti straps. 3. The dress hem length is shortened from midi-length (approximately mid-calf) to a mini skirt (approximately mid-thigh). [Secondary Changes] 1. The gold necklace around the neck is replaced with a thin silver necklace. [Overall Description] The woman in the image, originally wearing a grey wide-strap midi dress, has been edited to wear a pink spaghetti-strap mini dress. Her gold necklace has been replaced with a thin silver one. Her pose and white handbag remain unchanged.
Appendix
Table 7: An annotation example of image editing instructions.
Score
Criteria
Case
Reasoning
5
Perfect accuracy • All facts correct • Coverage ≥ 50%
“Change dress to pink spaghetti-strap; necklace to silver.”
All facts correct: pink ✓, spaghetti-strap ✓, silver necklace ✓. Coverage 2/3 (67%) ≥ 50%. Note: dress length not mentioned, but accuracy is perfect and coverage is sufficient.
4
Core correct, minor flaw • Major facts correct • Coverage ≥ 30% • Error on unchanged elements
“Change dress to pink; bag to red.”
Dress pink ✓. Bag unchanged but turned red × . Non-core error: − 1.
3
Single error or conservative penalty • 1 major factual error • Or coverage < 30%
A: “Dress to blue.” B: “Necklace to silver.”
A: GT is pink; blue × . B: Dress coverage =0× . Prevents skipping primary edits.
2
Multiple errors or hallucination • ≥ 2 major errors • Or fabricated operations
A: “Long blue dress.” B: “Pink dress; add hat.”
A: Color × + length × . B: “Add hat” not in GT × . Severely misleads.
1
Completely wrong • Wrong object • Unrelated to image
“Change car to red.”
No car in image × .
Appendix
Table 8: Representative scoring examples for the Accuracy dimension, with corresponding instructions and reasoning.
Hyperparameter
SFT
HAE-DPO
Learning rate
1×10−5
5×10−6
Global batch size
512
128
Training epochs
6
4
Max sequence length
8192
8192
Learning-rate schedule
Cosine decay
Cosine decay
Warmup ratio
0.03
0.03
Appendix
Table 11: Training hyperparameters for the SFT and HAE-DPO stages. Batch size denotes the global effective batch size after gradient accumulation.
Nanjing University China · Beijing Institute of Technology China · College of Artificial Intelligence, China University of Petroleum (Beijing) China +3