With the rapid advancement of image editing models and their widespread application across various domains, there is an increasingly urgent need to deploy these model capabilities directly into real-world scenarios. However, existing benchmarks remain confined to simple single-image tasks, suffering from limited coverage dimensions and an inability to effectively differentiate performance among diverse models. Consequently, they fail to reliably evaluate model performance in complex multi-image editing, highly demanding reasoning instructions, and practical deployment settings. To address these limitations, we propose CPI-Bench, a Comprehensive, Practical and Intelligent benchmark for real-world image editing. CPI-Bench comprises three core subsets: CPI-General-Bench, which comprehensively covers diverse editing tasks and introduces multi-image editing evaluation; CPI-Practical-Bench, which focuses on high-frequency real-user application scenarios; and CPI-Intelligent-Bench, which is dedicated to evaluating capabilities in highly demanding reasoning-based editing. Evaluation results of mainstream image editing models based on CPI-Bench demonstrate that CPI-Bench enhances performance differentiation among models. It provides a comprehensive and reliable quantification of gaps in general editing capabilities, practical deployment efficacy, and advanced reasoning-based editing, offering invaluable guidance for the future optimization of image editing models. Crucially, our ranking analysis reveals that CPI-Bench achieves the highest alignment with the Arena Image Edit Leaderboard, indicating stronger consistency with public human preference rankings, serving as an effective proxy for public human evaluations.
Figures & tables
Figure 1: Left: Model ranking trends across benchmarks compared against the Arena Image Edit Leaderboard Hugging Face [2024] . Our proposed CPI-Bench demonstrates the closest alignment with Arena. Right: Spearman correlation coefficients and Mean Absolute Error (MAE) with Arena rank. CPI-Bench outperforms other benchmarks, achieving the highest correlation and the lowest error margin.
Figure 2: The overview of CPI-Bench. CPI-Bench is decomposed into three distinct dimensions: CPI-General-Bench for comprehensive editing performance, CPI-Practical-Bench for real-world deployment efficacy, and CPI-Intelligent-Bench for reasoning-based editing tasks.
Figure 3: Hierarchical evaluation of model capabilities across CPI-Bench: (a) Basic editing performance (CPI-General); (b) Real-world application robustness (CPI-Practical); and (c) Highly demanding reasoning tasks (CPI-Intelligent). The difficulty escalates progressively from general, practical to intelligent levels.
Figure 4: The construction pipeline of CPI-Bench. The process comprises four sequential stages: (1) Taxonomy Definition, (2) Image Collection and Curation, (3) Instruction Generation and Quality Assurance, and (4) Privacy Preservation and Face Anonymization.
Figure 6: CPI-Practical-Bench distribution. CPI-Practical-Bench comprises 51 common application scenarios, covering 4 top-level categories and 13 sub-categories.
Table 1: Comparison of image editing models across three public benchmarks and our proposed CPI-Bench suite. For multilingual benchmarks, we report results on the English split.