PlanViz: Evaluating Planning-Oriented Image Generation and Editing for Computer-Use Tasks

Junxian Li; Kai Liu; Leyang Chen; Weida Wang; Zhixin Wang; Jiaqi Xu; Fan Li; Renjing Pei

cs.CVFeb 6, 2026

PlanViz: Evaluating Planning-Oriented Image Generation and Editing for Computer-Use Tasks

Authors: Junxian Li, Kai Liu, Leyang Chen, Weida Wang, Zhixin Wang, Jiaqi Xu, Fan Li, Renjing Pei,

Organizations: Shanghai Jiao Tong University Shanghai, China · Fudan University Shanghai, China · Huawei Technologies Ltd Shenzhen, China

Abstract

Unified multimodal models (UMMs) have shown impressive capabilities in complex natural image generation. However, their potential in computer-use planning tasks, which are closely related to our lives, remains rarely explored. Image generation and editing in computer-use tasks require capabilities like fine-grained perception and procedural understanding, and it is still unknown whether UMMs can finish these tasks well. Therefore, we propose PlanViz, a new benchmark designed to evaluate image generation and editing for computer-use tasks. We focus on sub-tasks which frequently involve in daily life and require planning steps. Specifically, three representative sub-tasks are designed: route planning, work diagramming, and web&UI displaying. Data quality is ensured by including large human efforts on question, reference and criteria annotation. Additionally, a task-adaptive score, PlanScore, is proposed for comprehensive and exact evaluation. PlanScore helps understanding the correctness, visual quality and efficiency of generated images. Through experiments, we highlight key limitations and opportunities for future research on this topic. Codes and supplementary materials are in https://github.com/lijunxian111/PlanViz.

Figures & tables

Figure 1: Examples of generation (left) and editing (right). The queries are “generating a flowchart on how to apply a VISA” and “show what happens if setting ‘Chinese(Simplified)’ to the display language”. Both UMMs make mistakes: Bagel doesn’t provide a complete workflow and texts are meaningless; GPT-Image-1 provides garbled characters and fails to keep the total layout.

Figure 2: The overview of PlanViz. Our evaluation includes image generation and editing, with three proposed sub-tasks: route planning, work diagramming, and web&UI displaying. Compared with prior benchmarks, PlanViz focuses on planning-oriented generation and editing in computer-use scenarios, supported by human-annotated references and keypoints.

Benchmarks	Domain	Type	Planning	Keypoints
ROVER [ 28 ]	Natural/science image	Edit	✗	✗
GenExam [ 43 ]	Science(exam) image	Gen	✗	✓
WiseEdit [ 32 ]	Mostly natural image	Edit	✗	✗
Weave [ 11 ]	Natural/science/Logic	Gen&Edit	partial	✓
PlanViz (ours)	Computer-use image	Gen&Edit	✓	✓

Table 1: Comparison with other benchmarks.

Panel of Figure 3 — Figure 3: The distribution (left) and the word cloud (right) of our benchmark. The green part represents route planning, while the orange part represents workflow diagramming, and the blue part represents web&UI displaying. The word cloud shows the hot topics in all questions of our benchmark.

Figure 4: Pipeline of data construction. It consists of four stages: high-quality data collecting and cleaning, human annotation, quality check, and prompt style transformation. These stages are displayed from the top-left to the bottom right.

Models	OmniGen2			GPT-Image-1
& Metrics	Cor	Vis	Ef	Cor	Vis	Ef
MLLM Mean	0.20	0.71	0.52	0.65	0.84	0.86
Human Mean	0.18	0.76	0.48	0.68	0.78	0.81
Human-MLLM r	0.92	0.83	0.85	0.88	0.79	0.83
Human-MLLM MAE	0.02	0.05	0.04	0.03	0.08	0.05

Table 2: Correlation between human and model judgments.

Models	Params	Route Planning				Workflow Diagraming				Web&UI Displaying				Overall
Models	Params	Cor $\uparrow$	Vis $\uparrow$	Ef $\uparrow$	Avg $\uparrow$	Cor $\uparrow$	Vis $\uparrow$	Ef $\uparrow$	Avg $\uparrow$	Cor $\uparrow$	Vis $\uparrow$	Ef $\uparrow$	Avg $\uparrow$	Avg $\uparrow$
Open-source Edit-only Models
AnyEdit [ 51 ]	1B	0.00	0.82	0.70	0.51	0.00	0.50	0.31	0.27	0.01	0.56	0.52	0.36	0.36
OmniGen [ 48 ]	4B	0.00	0.77	0.62	0.46	0.00	0.58	0.16	0.25	0.01	0.34	0.31	0.22	0.31
UltraEdit [ 54 ]	8B	0.02	0.44	0.29	0.25	0.05	0.54	0.17	0.25	0.03	0.71	0.58	0.44	0.31
SD-3.5-large [ 33 ]	8B	0.02	0.41	0.29	0.24	0.01	0.30	0.17	0.16	0.02	0.14	0.22	0.13	0.18

Table 3: Main results on image editing tasks. “Avg” means the average of scores. All scores are between 0 and 1. We highlight the best performance of Cor , for it is the most important score. We also add scores from our annotated reference images.

Figure 5: Comparison of previous judgments and ours.

Models	Params	Route Planning				Workflow Diagramming				Web & UI Displaying				Overall
Models	Params	Cor $\uparrow$	Vis $\uparrow$	Ef $\uparrow$	Avg $\uparrow$	Cor $\uparrow$	Vis $\uparrow$	Ef $\uparrow$	Avg $\uparrow$	Cor $\uparrow$	Vis $\uparrow$	Ef $\uparrow$	Avg $\uparrow$	Avg $\uparrow$
Open-source Generation-only Models
OmniGen [ 48 ]	4B	0.01	0.85	0.75	0.54	0.01	0.43	0.20	0.21	0.21	0.75	0.24	0.40	0.38
SD-3.5-large [ 33 ]	8B	0.24	0.81	0.60	0.55	0.26	0.45	0.31	0.34	0.67	0.58	0.45	0.57	0.49
FLUX.1 dev [ 22 ]	12B	0.10	0.95	0.51	0.52	0.05	0.46	0.18	0.23	0.70	0.76	0.60	0.69	0.48
Hidream-I1-Full [ 6 ]	17B	0.03	0.85	0.27	0.38	0.00	0.56	0.11	0.22	0.06	0.54	0.12	0.24	0.28

Table 4: Main results on image generation tasks. “Avg” means the average of scores. All scores are between 0 and 1. We highlight the best performance of Cor , for it is the most important score on these tasks. “Generation” means image generation here.

Models	Open (Gen)			Closed (Gen)			Open (Edit)			Closed (Edit)
Models	Cor	Vis	Ef	Cor	Vis	Ef	Cor	Vis	Ef	Cor	Vis	Ef
FLUX.1 dev	0.20	0.71	0.39	0.67	0.79	0.63	0.05	0.55	0.39	0.05	0.47	0.31
Ovis-U1	0.14	0.58	0.40	0.23	0.50	0.93	0.05	0.34	0.30	0.14	0.35	0.19
OmniGen2	0.23	0.66	0.47	0.78	0.76	0.60	0.14	0.79	0.58	0.08	0.75	0.60
Bagel	0.11	0.66	0.43	0.47	0.61	0.45	0.08	0.08	0.14	0.09	0.17	0.07
GPT-Image-1	0.83	0.94	0.94	0.96	1.00	0.93	0.39	0.55	0.75	0.44	0.84	0.78

Table 5: Performance comparison between open-ended questions and closed-ended questions. Each score is the average of three sub-tasks. Best correctness is highlighted .

Panel of Figure 6 — Figure 6: Score distribution across different models. We choose Cor (top) and Vis (bottom) of route planning.

Panel of Figure 7 — Figure 7: Qualitative examples of Route Planning, Workflow Diagramming, and Web&UI Displaying (top to bottom). Each row shows Generation on the left and Editing on the right, separated by a small gap. Editing columns show Original, Reference, and outputs.

Sub-tasks	Gen			Edit
Sub-tasks	Cor	Vis	Ef	Cor	Vis	Ef
Route Planning	0.99	0.99	0.94	0.75	0.83	0.93
Workflow Diagramming	1.00	0.99	0.98	0.55	0.27	0.94
Web & UI Displaying	0.99	0.99	0.84	0.77	0.68	0.92

Table 6: Performance of code rendering. All evaluation follows the main table. We only count the successfully rendered samples here.

Figure 8: Gap between knowing how to plan and real visualization of UMMs. The query is: Design a process-oriented visual chart to explain how to please generate a clear and detailed flowchart illustrating the process of training a convolutional neural network (CNN). We reports the textual planning outputs using understanding mode, and the related generated images in generation mode.

Image Generation Evaluation

Image generation evaluation measures the quality, fidelity, diversity, and instruction-following ability of AI-generated images.

See all 113 papers

From Plans to Pixels: Learning to Plan and Orchestrate for Open-Ended Image Editing
May 14, 2026Anirudh Sundara Rajan, Krishna Kumar Singh, Yong Jae LeeText-Guided Image Editing Tool-Using Vision-Language Agents
University of Wisconsin–Madison, WI, USA · Adobe Research, San Jose, CA, USA
Probing Visual Planning in Image Editing Models
Apr 23, 2026Zhimu Zhou, Yanpeng Zhao, Qiuyu Liao +2Visual Spatial Reasoning Diffusion-Based Planning
Shanghai Jiao Tong University · State Key Laboratory of General Artificial Intelligence, BIGAI · Renmin University of China
PhotoAgent: Exploratory Visual Aesthetic Planning with Large Vision Models
Feb 26, 2026Mingde Yao, Zhiyuan You, King-Man Tam +2Text-Guided Image Editing Image Editing
MMLab, CUHK · CPII under InnoHK · Institute of Science Tokyo +2

May 14, 2026J/K move · Enter open · S save

From Plans to Pixels: Learning to Plan and Orchestrate for Open-Ended Image Editing

Anirudh Sundara Rajan, Krishna Kumar Singh, Yong Jae Lee

University of Wisconsin–Madison, WI, USA · Adobe Research, San Jose, CA, USA

Text-Guided Image Editing Tool-Using Vision-Language Agents Image Editing RL for Image Editing

Open paper AskPDF arXiv

Modern image editing models produce realistic results but struggle with abstract, multi step instructions (e.g., ``make this advertisement more vegetarian-friendly''). Prior agent based methods decompose such tasks but rely on handcrafted pipelines or teacher imitation, limiting flexibility and decoupling learning from actual editing outcomes. We propose an experiential framework for long-horizon image editing, where a planner generates structured atomic decompositions and an orchestrator selects tools and regions to execute each step. A vision language judge provides outcome-based rewards for instruction adherence and visual quality. The orchestrator is trained to maximize these rewards, and successful trajectories are used to refine the planner. By tightly coupling planning with reward driven execution, our approach yields more coherent and reliable edits than single-step or rule-based multistep baselines.

Abstract

Image Generation Evaluation

Explore similar work

From Plans to Pixels: Learning to Plan and Orchestrate for Open-Ended Image Editing

Probing Visual Planning in Image Editing Models

PhotoAgent: Exploratory Visual Aesthetic Planning with Large Vision Models