Organizations: Faculty of Computing, Harbin Institute of Technology, Harbin, 150001, China. · Zhengzhou Advanced Research Institute of Harbin Institute of Technology, Zhengzhou, 450000, China. · Huawei Noah’s Ark Lab, Shenzhen, 518100, China. · Artificial Intelligence Research Institute, Shenzhen University of Advanced Technology, Shenzhen, 518107, China.
Emerging unified editing models have demonstrated strong capabilities in general object editing tasks. However, it remains a significant challenge to perform fine-grained editing in complex multi-entity scenes, particularly those where targets are not visually salient and require spatial reasoning. To this end, we propose InterCoG, a novel text-vision Interleaved Chain-of-Grounding reasoning framework for fine-grained image editing in complex real-world scenes. The key insight of InterCoG is to first perform object position reasoning solely within text that includes spatial relation details to explicitly deduce the location and identity of the edited target. It then conducts visual grounding via highlighting the editing targets with generated bounding boxes and masks in pixel space, and finally rewrites the editing description to specify the intended outcomes. To further facilitate this paradigm, we propose two auxiliary training modules: multimodal grounding reconstruction supervision and multimodal grounding reasoning alignment to enforce spatial localization accuracy and reasoning interpretability, respectively. We also construct GroundEdit-45K, a dataset comprising 45K grounding-oriented editing samples with detailed reasoning annotations, and GroundEdit-Bench for grounding-aware editing evaluation. Extensive experiments substantiate the superiority of our approach in highly precise edits under spatially intricate and multi-entity scenes.
Figures & tables
Fig. 1 : We present InterCoG , a novel interleaved chain-of-grounding reasoning framework capable of performing spatially precise image editing in complex scenes, particularly those demanding referential reasoning for precise target localization. InterCoG first performs textual grounding to parse the user-specified editing target, followed by visual grounding with explicit and interpretable localization cues in the form of mask and bounding boxe tags. Leveraging these grounded cues, the model is able to conduct target-faithful editing under complex multi-entity scenarios, thus enjoying better functionality and practicality in realistic applications.
Fig. 2 : GroundEdit-45K Dataset Construction Pipeline and Statistics. Left: The pipeline consists of three steps: (1) target selection and visual grounding generation; (2) instruction and text reasoning generation; and (3) context-preserving local target editing. Right: Our dataset contains 45K samples covering 8 categories of local editing types.
Fig. 3 : Overview of the Proposed InterCoG Framework. Left: Our framework performs text–vision interleaved chain-of-grounding reasoning to interpret and locate user-intended targets and formulate editing descriptions, ultimately producing precise and semantically aligned editing results. Right: Illustration of the proposed text–vision and vision–vision reasoning alignment schemes, designed to enforce coherent multimodal grounding reasoning.
Method
Color Change
Remove
Replace
Add
Appearance
Others
Average
EGA ↑
ES ↑
EGA ↑
ES ↑
EGA ↑
ES ↑
EGA ↑
ES ↑
EGA ↑
ES ↑
EGA ↑
ES ↑
EGA ↑
ES ↑
Human ↑
FLUX.2 [Pro] [ 54 ]
0.53
2.61
0.57
2.48
0.48
2.76
0.61
3.08
0.26
3.28
0.38
2.92
0.53
2.65
7.4
GPT-Image-1 [ 55 ]
0.61
2.78
0.74
2.89
0.55
3.20
0.42
2.82
0.83
3.56
0.65
3.72
0.66
2.93
8.4
GPT-Image-1.5 [ 56 ]
0.83
4.09
0.84
3.82
0.82
3.76
0.80
3.58
0.89
3.62
0.83
4.06
0.81
3.92
8.7
Nano Banana [ 57 ]
0.56
2.91
0.66
3.15
0.53
2.64
0.45
2.58
0.71
4.08
0.59
3.40
0.61
2.98
8.1
Nano Banana Pro [ 57 ]
0.82
4.11
0.85
3.94
0.80
3.96
0.83
3.72
0.92
3.38
0.90
4.30
0.87
4.02
9.2
TABLE I : Quantitative comparison on the proposed GroundEdit-Bench. EGA and ES denote the editing grounding accuracy and editing score, respectively. The “Average” column indicates the overall mean across all samples, rather than the mean of the listed categories.
Fig. 4 : Qualitative comparisons on our proposed GroundEdit-Bench. Our method consistently delivers highly precise and spatially accurate edits, particularly in multi-entity and fine-grained reasoning scenarios. Best viewed at screen!
Fig. 5 : Visualization of the interleaved chain-of-grounding reasoning processing. InterCoG first interprets user-intended referential targets via textual reasoning and then highlights the object via visualizing bounding boxes and masks in pixel space. By interleaving these multimodal grounding cues, InterCoG is able to precisely locate the editing regions and achieve spatially exact modifications. Best viewed at screen!
Method
SmartEdit Reasoning Scenarios
PSNR ↑
SSIM ↑
LPIPS ↓
CLIP Score ↑
InstructPix2Pix [ 1 ]
24.234
0.707
0.083
19.413
MagicBrush [ 2 ]
22.101
0.694
0.113
19.755
InstructDiffusion [ 64 ]
21.453
0.666
0.117
19.523
SmartEdit-7B [ 15 ]
25.258
0.742
0.055
20.950
SmartEdit-13B [ 15 ]
25.757
0.747
0.051
20.777
TABLE II : Quantitative comparison results on SmartEdit benchmark (Reasoning Scenarios).
Method
SmartEdit Understanding Scenarios
PSNR ↑
SSIM ↑
LPIPS ↓
CLIP Score ↑
InstructPix2Pix [ 1 ]
21.576
0.721
0.089
22.762
MagicBrush [ 2 ]
18.120
0.680
0.143
22.620
InstructDiffusion [ 64 ]
23.258
0.743
0.067
23.080
SmartEdit-7B [ 15 ]
22.049
0.731
0.087
23.611
SmartEdit-13B [ 15 ]
23.596
0.751
0.068
23.536
TABLE III : Quantitative comparison results on SmartEdit benchmark (Understanding Scenarios).
Method
GEdit-Bench-EN
SC ↑
PQ ↑
O ↑
InstructPix2Pix [ 1 ]
3.58
5.49
3.68
MagicBrush [ 2 ]
4.68
5.66
4.52
AnyEdit [ 3 ]
3.18
5.82
3.21
UniWorld-v1 [ 16 ]
4.93
7.43
4.85
OmniGen2 [ 29 ]
7.16
6.77
6.41
TABLE IV : Comparison of Semantic Consistency (SC), Perceptual Quality (PQ), and Over all Score (O) on GEdit-Bench.
Fig. 6 : Visualization of model performance on lobal and multi-object editing.
Variant
EGA ↑
ES ↑
w/o Reasoning
0.70
3.15
w Text-Only Grounding Reasoning
0.76
3.57
w Vision-Only Grounding Reasoning
0.74
3.23
w/o Grounding Reconstruction
0.85
3.83
w/o Reasoning Alignment
0.83
3.71
InterCoG (Ours)
0.89
4.02
TABLE V : Ablation studies on the GroundEdit-Bench.
Fig. 7 : Visual comparison between text-only grounding reasoning and interleaved chain-of-grounding reasoning.
Fig. 8 : Visual comparison with or without our proposed multimodal grounding reconstruction supervision.
Fig. 9 : Visual comparison with or without our proposed multimodal grounding reasoning alignment.
Layer
EGA ↑
ES ↑
3
0.79
3.70
10
0.83
3.95
14
0.89
4.02
28
0.84
3.89
TABLE VI : Ablation studies of layer choices for multimodal grounding reasoning alignment on GroundEdit-Bench.
Method
Latency
Peak GPU Mem.
EGA
ES
Qwen-Image-Edit
1m 50s
60.6GB
0.59
2.97
Bagel
1m 03s
30.6GB
0.64
2.70
InterCoG w/o Reasoning
1m 03s
30.6GB
0.75
3.51
InterCoG (Ours)
2m 16s
31.9GB
0.89
4.02
TABLE VII : Average latency and peak GPU memary comparison under 1024 × 1024 resolution on an NVIDIA A100 GPU.
Fig. 10 : Failure cases with ambiguous position and boundary.
Beijing University of Posts and Telecommunications · Beijing Key Laboratory of Multimodal Data Intelligent Perception and Governance · Tencent Hunyuan +1