Organizations: Beijing University of Posts and Telecommunications · Beijing Key Laboratory of Multimodal Data Intelligent Perception and Governance · Tencent Hunyuan · King Abdullah University of Science and Technology
Existing image editing methods struggle to perceive where to edit, especially under complex scenes and nuanced spatial instructions. To address this issue, we propose Generative Visual Chain-of-Thought (GVCoT), a unified framework that performs native visual reasoning by first generating spatial cues to localize the target region and then executing the edit. Unlike prior text-only CoT or tool-dependent visual CoT paradigms, GVCoT jointly optimizes visual tokens generated during the reasoning and editing phases in an end-to-end manner. This way fosters the emergence of innate spatial reasoning ability and enables more effective utilization of visual-domain cues. The main challenge of training GCVoT lies in the scarcity of large-scale editing data with precise edit region annotations; to this end, we construct GVCoT-Edit-Instruct, a dataset of 1.8M high-quality samples spanning 19 tasks. We adopt a progressive training strategy: supervised fine-tuning to build foundational localization ability in reasoning trace before final editing, followed by reinforcement learning to further improve reasoning and editing quality. Finally, we introduce SREdit-Bench, a new benchmark designed to comprehensively stress-test models under sophisticated scenes and fine-grained referring expressions. Experiments demonstrate that GVCoT consistently outperforms state-of-the-art models on SREdit-Bench and ImgEdit. We hope our GVCoT will inspire future research toward interpretable and precise image editing.
Figures & tables
Figure 2: Comparing spatial cue representation for image editing on ImgEdit [ 68 ] . We study two ways of injecting spatial information: (1) text modality uses bounding box coordinates, and (2) visual modality providing a binary mask. Providing spatial information in the visual modality yields a greater improvement in both instruction adherence and background preservation.
Figure 3: Supervised Fine-Tuning of our GVCoT training recipe. Stage 1: Multi-Task Visual Manipulation, where the model’s generation expert is trained in a multi-task setup to inject the newly masking skill. Stage 2: Visual Reason-aided Editing, where the entire model is trained to generate a faithful and interpretable visual reasoning image and then an edited image within a single sequence.
Figure 4: GVCoT-Edit-Instruct Data Pipeline. Left: We design a scalable multi-stage data pipeline to curate high-quality samples with faithful editing region annotations, i.e. , bounding boxes and masks. Right: The distribution of GVCoT-Edit-Instruct spanning 19 tasks.
Figure 5: Illustration of the SREdit-Bench . Left: We provide challenging scenarios featuring complex scenes and fine-grained referring expressions. Right: (a) We quantify scene complexity by counting editable objects and regions. Results show that SpaEdit-Bench concentrates on more sophisticated scenes than ImgEdit [ 68 ] and GEdit-Bench [ 40 ] . (b) Referral type distribution. (c) Edit tasks distribution.
Model
SREdit-Bench ↑
SCg
PQg
Og
Product-level models
GPT Image 1 [High] [ 47 ]
9.02
8.42
8.56
FLUX.1 Kontext [Pro]
8.69
8.40
8.41
Qwen-Image [ 63 ]
8.57
8.43
8.32
Generation Only
Table 1: Quantitative results on SREdit-Bench. SCg , PQg , and Og indicate scores on Semantic Consistency, Perceptual Quality, and Overall. The best and second-best results are highlighted in bold and underlined, respectively.
Method
CLIP-I ↑
LPIPS ↓
PSNR ↑
SSIM ↑
SmartEdit [ 24 ]
0.8841
0.2915
17.1728
0.6828
BrushNet [ 28 ]
0.8986
0.1830
19.2172
0.7877
MagicQuill [ 42 ]
0.9381
0.1162
22.2380
0.8981
MIND-Edit [ 61 ]
0.9310
0.1245
22.2714
0.8517
Bagel [ 12 ]
0.9124
0.1721
22.1843
0.8640
Bagel-GVCoT (Ours)
0.9451
0.1066
23.0161
0.8943
Table 2: Quantitative results in HumanEdit. Our Bagel-GVCoT allowing native visual reasoning outperforms previous approaches that rely on additional mask guidance.
Model
Add
Adjust
Extract
Replace
Remove
Background
Style
Hybrid
Action
Overall ↑
Product-level models
FLUX.1 Kontext [Pro] [ 31 ]
4.25
4.15
2.35
4.56
3.57
4.26
4.57
3.68
4.63
4.00
GPT Image 1 [High] [ 47 ]
4.61
4.33
2.90
4.35
3.66
4.57
4.93
3.96
4.89
4.20
Qwen-Image [ 63 ]
4.38
4.16
3.43
4.66
4.14
4.38
4.81
3.82
4.69
4.27
Generation Only
MagicBrush [ 70 ]
2.84
1.58
1.51
1.97
1.58
1.75
2.38
1.62
1.22
1.90
Table 3: Quantitative results on ImgEdit. We use GPT-4.1 to evaluate all metrics. “Overall” is calculated by averaging all scores across tasks. The best and second-best results in open-sourced models are highlighted in bold and underlined , respectively.
Figure 6: Quantitative comparison of different visual reasoning paradigms on SREdit-Bench. The performance is measured by Og . Under both visual cue forms, our method consistently surpasses the text CoT and Visual CoT with considerable margins.
Figure 7: Qualitative comparison in SREdit-Bench. Our method demonstrates superior spatial reasoning and instruction adherence compared to existing open-source models, especially when handling complex, multi-object editing tasks.
Model
IoU ↑
SCg↑
PQg↑
Og↑
Bagel *
–
6.21
6.09
5.71
TCoT
–
6.54
6.22
5.97
Mask
VCoT
0.68
7.55
7.92
7.31
GVCoT
0.60
8.53
8.62
8.42
VCoT w/ TF-thought
–
8.03
7.96
7.75
GVCoT w/ TF-thought
–
8.79
8.95
8.72
Table 4: Comparative results of GVCoT and Visual CoT using SREdit-Bench. TF-thought denotes skipping the reasoning step by using teacher-forcing visual thought.
Step 1
Step 2
IoU ↑
SCg↑
PQg↑
Og↑
✓
0.64
7.92
8.01
7.75
✓
0.61
8.21
8.33
8.10
✓
✓
0.66
8.53
8.62
8.42
Table 5: Ablation results on progressive SFT.
Setting
IoU ↑
SCg↑
PQg↑
Og↑
SFT Only
0.60
8.53
8.62
8.42
SFT+RL
0.67
8.57
8.76
8.53
(1) Multi-stage RL training
w/o Stage 2
0.67
8.32
8.47
8.25
(2) Visual thought reward design
Full reward set
0.67
8.32
8.47
8.25
Table 6: Ablation studies on Reinforcement Learning (RL). We analyze (1) multi-stage training strategy, (2) visual-thought reward design, and (3) editing reward design.
Appendix figures & tables15 assets
Supplementary material from the paper’s appendix.
Appendix
Model
GEdit-Bench-EN (Full set)
G_SC ↑
G_PQ ↑
G_O ↑
Product-level models
FLUX.1 Kontext [Pro] [ 31 ]
7.02
7.60
6.56
Gemini 2.0 [ 10 ]
6.73
6.61
6.32
Gemini 2.5 Flash Image Preview [ 11 ]
7.28
7.83
6.93
Gemini 2.5 Flash Image [ 11 ]
7.41
7.96
7.10
Appendix
Table 1: Quantitative results on GEdit-Bench We use Semantic Consistency (G_SC), Perceptual Quality (G_PQ), and Overall Score (G_O) metrics measured by GPT 4.1.
Figure 1: Limitation of our Bagel-GVCoT. Our approach constrains under style transfer tasks compared with the baseline.
Emu Edit Test set
MagicBrush Test Set
Method
CLIPdir↑
CLIPim↑
CLIPout↑
L1↓
DINO ↑
CLIPdir↑
CLIPim↑
CLIPout↑
L1↓
DINO ↑
InstructPix2Pix [ 4 ]
0.078
0.834
0.219
0.121
0.762
0.115
0.837
0.245
0.093
0.767
MagicBrush [ 70 ]
0.090
0.838
0.222
0.100
0.776
0.123
0.883
0.261
0.058
0.871
PnP [ 55 ]
0.028
0.521
0.089
0.304
0.153
0.025
0.568
0.101
0.289
0.220
Null-Text Inv. [ 45 ]
0.101
0.761
0.236
0.075
0.678
0.121
0.752
0.263
0.077
0.664
UltraEdit [ 73 ]
0.107
0.793
0.283
0.071
0.844
-
0.868
-
0.088
0.792
Appendix
Table 2: Quantitative results on Emu Edit and MagicBrush test sets. All results except Bagel [ 12 ] and our Bagel-GVCoT are directly from Unireal [ 7 ] . The best and second-best results are highlighted in bold and underlined , respectively.
Figure 2: Statistical analysis of GVCoT-Edit-Instruct. Left: Length distribution of textual editing instructions. Middle: Word cloud of all editing instructions. Right: Proportion of edit-region masks that cover the entire image.
Figure 3: User study. We conduct GSB (Good/Same/Bad) to compare our method with state-of-the-art models.
Figure 4: More examples of our training dataset GVCoT-Edit-Instruct. Each sample consists of four components: (1) the source image, (2) the textual edit instruction, (3) the edit mask, a pink overlay indicating the edit region, and (4) the target image after editing.
Figure 5: More examples of SREdit-Bench. We focus on fine-grained referring expressions and sophisticated visual scenes.
Hyperparameter
Stage 1
Stage 2
Optimizer & Scheduler
Learning Rate (LR)
2×10−5
1×10−5
LR Scheduler
Cosine Decay
Cosine Decay
Min Learning Rate
1×10−7
1×10−7
Warmup Steps
1,000
1,000
Total Training Steps
80,00
12,000
Appendix
Table 3: Key hyperparameters for the two-stage training process in our Progressive Supervised Fine-Tuning. “ N/A ” indicates that the parameter did not apply to that stage.
Hyperparameter
Stage 1
Stage 2
Optimizer & Scheduler
Learning Rate (LR)
1×10−6
1×10−6
LR Scheduler
Cosine Decay
Cosine Decay
Min Learning Rate
1×10−7
1×10−7
Warmup Steps
50
50
Total Training Steps
500
450
Appendix
Table 4: Key hyperparameters for the two-stage training process in our Reinforcement-based Refining. “ N/A ” indicates that the parameter did not apply to that stage.
Figure 6: Qualitative comparison in SREdit-Bench. Our method demonstrates superior spatial reasoning and instruction adherence compared to existing open-source models, especially when handling complex, multi-object editing tasks.
Table 5: The prompt template used by Qwen2.5-VL-72B [ 2 ] to measure the consistency of the edited image with CoT visual thought.
Table 6: The prompt template used by Qwen2.5-VL-72B [ 2 ] to measure the perceptual quality of source and edited images.
Table 7: The prompt template used by Qwen2.5-VL-72B [ 2 ] for generating diverse image editing instructions.
Table 8: The prompt template used by Qwen2.5-VL-72B [ 2 ] to measure the semantic consistency of triplets, i.e. , source image, edited image, and edit instruction, during our data curation procedure.
Table 9: The prompt template used by Qwen2.5-VL-72B [ 2 ] for mining bounding box coordinates of the edit region.
Emerging unified editing models have demonstrated strong capabilities in general object editing tasks. However, it remains a significant challenge to perform fine-grained editing in complex multi-entity scenes, particularly those where targets are not visually salient and require spatial reasoning. To this end, we propose InterCoG, a novel text-vision Interleaved Chain-of-Grounding reasoning framework for fine-grained image editing in complex real-world scenes. The key insight of InterCoG is to first perform object position reasoning solely within text that includes spatial relation details to explicitly deduce the location and identity of the edited target. It then conducts visual grounding via highlighting the editing targets with generated bounding boxes and masks in pixel space, and finally rewrites the editing description to specify the intended outcomes. To further facilitate this paradigm, we propose two auxiliary training modules: multimodal grounding reconstruction supervision and multimodal grounding reasoning alignment to enforce spatial localization accuracy and reasoning interpretability, respectively. We also construct GroundEdit-45K, a dataset comprising 45K grounding-oriented editing samples with detailed reasoning annotations, and GroundEdit-Bench for grounding-aware editing evaluation. Extensive experiments substantiate the superiority of our approach in highly precise edits under spatially intricate and multi-entity scenes.
Yecong Wan, Fan Li, Chunwei Wang +3
Faculty of Computing, Harbin Institute of Technology, Harbin, 150001, China. · Zhengzhou Advanced Research Institute of Harbin Institute of Technology, Zhengzhou, 450000, China. · Huawei Noah’s Ark Lab, Shenzhen, 518100, China. +1
Existing image editing methods can be generally categorized into textual instruction-based and visual prompt-based ones. Textual instructions are semantically expressive, but are limited by the coarse granularity of spatial control of the editing results. In contrast, visual prompts such as drag and point can provide precise spatial guidance, but are limited by the inherent ambiguity in semantic intent. To unify the strength of textual and visual prompts, we present Text-Vision Co-Instructed Image Editing, which jointly models textual instructions as semantic intent and sparse visual instructions as spatial guidance, aiming to achieve precise and intent-faithful image manipulation. To this end, we first construct a textual-visual instruction paired dataset with more than 23K samples derived from dynamic videos, enabling aligned supervision for cross-modal instruction. We then propose TV-Edit, a Textual-Visual instruction unified Editing framework to contextualize drag or point-based visual instructions with image-text semantics and lift them into semantic-aware control representations for pretrained editing backbones. By integrating semantic intent and spatial constraints, TV-Edit leads to more precise spatial control, less instruction ambiguity, and stronger structural consistency than text-only or drag-based alternatives. Finally, we establish TV-Edit-Bench, a deliberately designed benchmark to evaluate semantic faithfulness, spatial alignment, and visual consistency with ground-truth references and controlled textual-visual variations for reliable assessment. Our experiments across multiple editing backbones demonstrate that TV-Edit consistently yields more precise and intent-faithful edits, significantly outperforming state-of-the-art instruction-based and drag-based baselines.
Chenxi Xie, Yuhui Wu, Qiaosi Yi +1
The Hong Kong Polytechnic University · OPPO Research Institute
Editing images via instruction provides a natural way to generate interactive content, but it is a big challenge due to the higher requirement of scene understanding and generation. Prior work utilizes a chain of large language models, object segmentation models, and editing models for this task. However, the understanding models provide only a single modality ability, restricting the editing quality. We aim to bridge understanding and generation via a new multi-modality model that provides the intelligent abilities to instruction-based image editing models for more complex cases. To achieve this goal, we individually separate the instruction editing task with the multi-modality chain of thought prompts, i.e., Chain-of-Thought (CoT) planning, editing region reasoning, and editing. For Chain-of-Thought planning, the large language model could reason the appropriate sub-prompts considering the instruction provided and the ability of the editing network. For editing region reasoning, we train an instruction-based editing region generation network with a multi-modal large language model. Finally, a hint-guided instruction-based editing network is proposed for editing image generations based on the sizeable text-to-image diffusion model to accept the hints for generation. Extensive experiments demonstrate that our method has competitive editing abilities on complex real-world images.