InEdit-Bench: Benchmarking Intermediate Logical Pathways for Intelligent Image Editing Models
Organizations: State Key Laboratory of Optical Field Manipulation Science and Technology, Institute of Optics and Electronics, Chinese Academy of Sciences, Chengdu, 610209, China · National Laboratory on Adaptive Optics, Chengdu, 610209, China · University of Chinese Academy of Sciences, Beijing, 101408, China
Abstract
Multimodal generative models have made significant strides in image editing, demonstrating impressive performance on a variety of static tasks. However, their proficiency typically does not extend to complex scenarios requiring dynamic reasoning, leaving them ill-equipped to model the coherent, intermediate logical pathways that constitute a multi-step evolution from an initial state to a final one. This capacity is crucial for unlocking a deeper level of procedural and causal understanding in visual manipulation. To systematically measure this critical limitation, we introduce InEdit-Bench, the first evaluation benchmark dedicated to reasoning over intermediate pathways in image editing. InEdit-Bench comprises meticulously annotated test cases covering four fundamental task categories: state transition, dynamic process, temporal sequence, and scientific simulation. Additionally, to enable fine-grained evaluation, we propose a set of assessment criteria to evaluate the logical coherence and visual naturalness of the generated pathways, as well as the model's fidelity to specified path constraints. Our comprehensive evaluation of 14 representative image editing models on InEdit-Bench reveals significant and widespread shortcomings in this domain. By providing a standardized and challenging benchmark, we aim for InEdit-Bench to catalyze research and steer development towards more dynamic, reason-aware, and intelligent multimodal generative models.
Figures & tables
| Dataset | Publication | Size | Dimensions | Perspective |
| AURORA-Bench [ 20 ] | NeurIPS 2024 | 400 | – | Action |
| SmartEdit [ 18 ] | CVPR 2024 | 219 | 2 | Attribute |
| RISEBench [ 61 ] | NeurIPS 2025 | 360 | 4 | T-C-S-L |
| IntelligentBench [ 7 ] | arXiv 2025.5 | 350 | – | Deep |
| InEdit-Bench | – | 237 | 4 | Pathway |
| Models | Appearance Consistency | Perceptual Quality | Semantic Consistency | Logical Coherence | Scientific Plausibility | Process Plausibility | Overall Average | Bootstrap 95% CIs | Accuracy |
| Proprietary Models | |||||||||
| GPT-Image-1 | 92.24 | 92.36 | 72.04 | 71.06 | 71.31 | 88.97 | 81.33 | [79.04, 83.61] | 16.75% |
| Nano-Banana | 86.45 | 92.49 | 62.93 | 60.22 | 73.58 | 75.74 | 75.23 | [72.40, 77.96] | 13.30% |
| Flux-Kontext-pro | 64.66 | 89.11 | 33.99 | 30.17 | 43.75 | 47.06 | 51.46 | [48.59, 54.45] | 0.99% |
| Doubao-SeedEdit-3.0 | 44.43 | 69.70 | 22.54 | 22.41 | 34.94 | 25.00 | 36.50 | [34.04, 39.10] | 0.00% |
| Open-Source Models | |||||||||
Appendix figures & tables26 assets
Supplementary material from the paper’s appendix.
Appendix
| Metric | Proprietary Models | Open-Source Models | |||||||||||||||
| GPT-Image-1 | Nano-Banana | Flux-Kontext-pro | Doubao-SeedEdit-3.0 | Qwen-Image-Edit | Emu1 | Emu2 | Bagel | Bagel-Think | OmniGen | OmniGen2 | Step1X-Edit (v1.0) | Step1X-Edit (v1.1) | InstructPix2Pix | ||||
| State Transition | AC | 95.41 | 79.59 | 56.12 | 42.71 | 53.06 | 3.06 | 30.10 | 38.27 | 48.47 | 5.61 | 31.63 | 12.76 | 21.94 | 24.49 | ||
| PQ | 92.35 | 94.39 | 94.79 | 69.39 | 81.63 | 44.90 | 83.33 | 69.39 | 78.06 | 29.08 | 80.10 | 42.71 | 47.45 | 72.96 | |||
| SC | 72.45 | 58.67 | 31.63 | 21.94 | 24.49 | 1.53 | 8.67 | 23.98 | 19.90 | 2.04 | 19.39 | 7.65 | 10.71 | 4.59 | |||
| LC | 64.29 | 47.96 | 24.49 | 19.39 | 19.90 | 1.02 | 11.73 | 21.43 | 20.92 | 5.10 | 22.45 | 5.61 | 13.78 | 12.24 | |||
| Avg | 81.12 | 70.15 | 51.76 | 38.36 | 44.77 | 12.63 | 33.46 | 38.27 | 41.84 | 10.46 | 38.39 | 17.18 | 23.47 | 28.57 | |||
| SubTasks | Proprietary Models | Open-Source Models | |||||||||||||||
| GPT-Image-1 | Nano-Banana | Flux-Kontext-pro | Doubao-SeedEdit-3.0 | Qwen-Image-Edit | Emu1 | Emu2 | Bagel | Bagel-Think | OmniGen | OmniGen2 | Step1X-Edit (v1.0) | Step1X-Edit (v1.1) | InstructPix2Pix | ||||
| State Transition | CA | 14.29 | 7.14 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | ||
| DP | 8.33 | 8.33 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | |||
| OA | 40.00 | 10.00 | 10.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | |||
| PD | 7.69 | 15.38 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | |||
| Avg | 16.33 | 10.20 | 2.04 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | |||
Image Editing Evaluation
Image editing evaluation measures whether edits follow instructions while preserving identity, context, and visual quality.
See all 92 papers