Image Editing Evaluation

Unified multimodal models (UMMs) have shown impressive capabilities in complex natural image generation. However, their potential in computer-use planning tasks, which are closely related to our lives, remains rarely explored. Image generation and editing in computer-use tasks require capabilities like fine-grained perception and procedural understanding, and it is still unknown whether UMMs can finish these tasks well. Therefore, we propose PlanViz, a new benchmark designed to evaluate image generation and editing for computer-use tasks. We focus on sub-tasks which frequently involve in daily life and require planning steps. Specifically, three representative sub-tasks are designed: route planning, work diagramming, and web&UI displaying. Data quality is ensured by including large human efforts on question, reference and criteria annotation. Additionally, a task-adaptive score, PlanScore, is proposed for comprehensive and exact evaluation. PlanScore helps understanding the correctness, visual quality and efficiency of generated images. Through experiments, we highlight key limitations and opportunities for future research on this topic. Codes and supplementary materials are in https://github.com/lijunxian111/PlanViz.

Image editing models can modify visual content through natural language, but editing dense documents requires precise local changes amid tightly coupled text, tables, and layout. We introduce VDE Bench, an evaluation resource for studying instruction-guided editing of Chinese and English document images. VDE Bench contains 3,506 instruction-guided image editing samples, including 1,578 text-region edits and 1,928 table-region edits. The benchmark pairs editing instructions and reference images with annotated editing regions and structured OCR outputs. Its evaluation protocol combines region-level and full-page measurements to assess textual correctness, structural fidelity, and visual content preservation. By focusing on the interaction between local edits and their surrounding document context, VDE Bench complements existing visual text editing benchmarks and provides a resource for diagnosing document-specific editing failures.