EvoDesign: Agentic Editable Diagram Creation via Design Expertise Evolution

Tianfu Wang; Leilei Ding; Ziyang Tao; Yi Zhan; Zhiyuan Ma; Wei Wu; Yuxuan Lei; Junyang Wang

cs.HCFeb 20, 2026

EvoDesign: Agentic Editable Diagram Creation via Design Expertise Evolution

Authors: Tianfu Wang, Leilei Ding, Ziyang Tao, Yi Zhan, Zhiyuan Ma, Wei Wu, Yuxuan Lei, Junyang Wang, +7 more

Organizations: Hong Kong University of Science and Technology (Guangzhou) · University of Science and Technology of China · Peking University · Hong Kong University of Science and Technology

Abstract

High-fidelity diagram creation requires the complex orchestration of semantic topology, visual styling, and spatial layout, posing a significant challenge for automated systems. Existing methods also suffer from a representation gap: pixel-based models often lack precise control, while code-based synthesis limits intuitive flexibility. To bridge this gap, we introduce EvoDiagram, an agentic framework that generates object-level editable diagrams via an intermediate canvas schema. EvoDiagram employs a coordinated multi-agent system to decouple semantic intent from rendering logic, resolving conflicts across heterogeneous design layers. Additionally, we propose a design knowledge evolution mechanism that distills execution traces into a hierarchical memory of domain guidelines, enabling agents to retrieve context-aware expertise adaptively. We further release CanvasBench, a benchmark consisting of both data and metrics for canvas-based diagramming. Extensive experiments demonstrate that EvoDiagram exhibits excellent performance and balance against baselines in generating editable, structurally consistent, and aesthetically coherent diagrams. Our code is available at https://github.com/AuraX-AI/EvoDiagram.

Figures & tables

Figure 1 : Comparison of diagram generation paradigms. Unlike pixel-based generation (limited control) or code-based synthesis (high barrier), Canvas-based creation unifies AI actionability with human-interpretable UI editing, bridging the representation gap.

Figure 2 : The overview of EvoDesign framework. (a) Agentic Creation System : A multi-agent pipeline where specialized agents for structure, style, and layout coordinate via a shared symbolic schema, followed by a closed-loop refinement agent to resolve cross-layer conflicts. (b) Hybrid Experience Search : A structural exploration of the design space using both vertical refinement and horizontal comparison. (c) Design Knowledge Distillation : A hierarchical process that distills execution traces into specialized domain guidelines and universal design principles for storage in the design expertise Memory.

Figure 3 : The dataset construction pipeline and the overview of CanvasBench.

Category	Method	Content			Visual				Cognitive
Category	Method	$\text{C}_{\textit{CF}}$ ( $\uparrow$ )	$\text{C}_{\textit{CL}}$ ( $\uparrow$ )	$\text{C}_{\textit{SR}}$ ( $\uparrow$ )	$\text{V}_{\textit{VA}}$ ( $\uparrow$ )	$\text{V}_{\textit{SC}}$ ( $\uparrow$ )	$\text{V}_{\textit{FC}}$ ( $\uparrow$ )	$\text{V}_{\textit{SB}}$ ( $\uparrow$ )	$\text{G}_{\textit{CE}}$ ( $\uparrow$ )	$\text{G}_{\textit{CA}}$ ( $\uparrow$ )	$\text{G}_{\textit{SE}}$ ( $\uparrow$ )
Diffusion	GPT-4o-Image	2.200	3.110	2.710	3.990	4.354	4.153	3.923	4.301	2.751	3.043
	NanoBanano	1.986	2.871	2.657	3.767	3.838	3.533	3.671	3.761	2.665	2.990
	Flux.2 flex	1.871	2.782	2.643	3.836	3.868	3.614	3.704	3.780	2.712	2.922
L a T e X Tikz	GPT-5.2	4.096	3.933	4.067	2.833	3.257	2.933	2.543	3.295	3.171	2.505
	Gemini-3-Pro	4.201	4.129	4.086	3.368	3.670	3.641	3.124	3.871	3.416	2.541

Table 1 : Main results on CanvasBench. Performance is measured across content, visual, and cognitive dimensions.

Figure 4 : Visual comparison of different baseline methods on a representative diagramming task.

Appendix figures & tables13 assets

Supplementary material from the paper’s appendix.

Appendix

Chart Type	Best For	Key Characteristics
Architecture Diagram	System design, software structure	Modules, components, and interfaces
Class Diagram	OOP modeling, database schemas	Classes, attributes, methods, inheritance
Concept Map	Knowledge organization, learning	Nodes connected by labeled relationships
Data Flow Diagram	System information flow analysis	Processes, data stores, external entities
Entity Relationship	Database design, modeling data	Entities, attributes, relationships (1:N, M:N)
Fishbone Diagram	Root cause analysis	Spine (problem) with ribs (causes)

Appendix

Table 2 : Comprehensive Summary of Diagram Types.

Category	Property	Candidate Parameters Values
Color	TLColor	black, grey, light-violet, violet, blue, light-blue, yellow, orange, green, light-green, light-red, red, white
Geometry	TLGeo	rectangle, ellipse, triangle, diamond, pentagon, hexagon, octagon, star, cloud, arrowLeft, arrowRight, arrowUp, arrowDown
Size	TLSize	s, m, l, xl
Fill Style	TLFill	none, semi, solid, pattern
Dash Style	TLDash	draw, solid, dashed, dotted
Typography	TLFont	draw, sans, serif, mono

Appendix

Table 3: Styling-related properties and categorical parameter spaces of tldraw shape elements

Category	Property	Restricted Parameter Space
Position	x, y	Positive Integers
Dimensions	w, h	Positive Integers
Rotation	rotation	Integer Degrees
Alignment	align	start , middle , end
Layering	index	Sequential Rank Index

Appendix

Table 4: Layout-related properties and discretized parameter spaces of tldraw shape elements

Panel of Table 5 — Table 5 : Detailed Taxonomy for Diagram Image Retrieval. The taxonomy intersects 21 structural types with 30 semantic domains.

Figure 5 : Information density by diagram type. The box plots illustrate the distribution of character counts for each category, revealing high semantic variation and significant textual depth across the dataset.

Figure 6 : Hierarchical distribution of vertical domains. The sunburst chart depicts the balanced coverage across six primary disciplines and 30 granular sub-domains, ensuring the benchmark tests generalizability across diverse knowledge fields.

Figure 7 : Data examples in CanvasBench.

Dimension	Metric Name	Symbol	Focus	Sub-items
Content Integrity	Content Fidelity	$\text{C}_{\textit{CF}}$	Trustworthiness & Fact-checking	Factual Consistency, Hallucination Rate, Evidence Traceability
	Concept Logic	$\text{C}_{\textit{CL}}$	Logical Structure & Relationships	Edge Correctness, Relation Type Accuracy, Self-Consistency, Hierarchy Rationality
	Semantic Relevance	$\text{C}_{\textit{SR}}$	Information Density & Alignment	Coverage, Conciseness, Context Alignment
Visual Presentation	Visual Aesthetics	$\text{V}_{\textit{VA}}$	Overall Look & Feel	Composition & Whitespace, Harmony, Professionalism
	Styling Consistency	$\text{V}_{\textit{SC}}$	Standardization & Uniformity	Font Consistency, Line & Shape Consistency, Color Theme Consistency
	Flow Coherence	$\text{V}_{\textit{FC}}$	Reading Path & Navigation	Main Reading Direction, Low Path Ambiguity, Key Path Trackability

Appendix

Table 6 : Overview of Evaluation Metrics

Figure 8 : A case study of refinement iterations.

Panel of Table 7 — Table 7 : Example of sample strategy ( $K^{s}$ ) within the design memory $\mathcal{M}$ .

Panel of Table 8 — Table 8 : Examples of domain guideline ( $K^{s}$ ) within the design memory $\mathcal{M}$ .

Panel of Table 9 — Table 9 : Examples of general principle ( $K^{p}$ ) within the design memory $\mathcal{M}$ .

Panel of Figure 9 — Figure 9 : Human intuitively refines the generated diagram via UI-friendly operations in our web application. The interface facilitates a fluid transition from agentic generation to manual manipulation.

Explore similar work

DiagramRAG: A Lightweight Framework to Retrieve Scientific Diagram for Figure Generation
May 27, 2026Xinjiang Yu, Junyi Han, Zhuofan Chen +7Diagrams Sketches
Beijing Institute of Technology
DIAGRAMS: A Review Framework for Reasoning-Level Attribution in Diagram QA
Apr 29, 2026Anirudh Iyengar Kaniyar Narayana Iyengar, Tampu Ravi Kumar, Manan Suri +4Diagrams Reasoning Graphs
Arizona State University · University of Maryland · IIITDM +1
PaperBanana-Interact: Scientific Diagram Refinement with Multi-Turn Human Feedback
Aug 31, 2026Xueqing Wu, Ashwin Balasubramanian, Bingxuan Li +7Diagrams Paperpilot
University of California, Los Angeles · Google · University of Illinois Urbana-Champaign +1

May 27, 2026, cs.AIJ/K move · Enter open · S save

DiagramRAG: A Lightweight Framework to Retrieve Scientific Diagram for Figure Generation

Xinjiang Yu, Junyi Han, Zhuofan Chen, Chi Zhang+6

Beijing Institute of Technology

Diagrams Sketches Graph-Based Retrieval Augmented Generation Retrievers Lightweight

Open paper AskPDF arXiv

Scientific diagrams are essential for communicating complex methodologies in academic papers. A natural way for researchers to specify such diagrams is through rough sketches, where text labels, connectors, and spatial arrangements express early semantic and topological intentions. However, sketches are usually incomplete, making them insufficient for directly producing publication-quality diagrams. Existing sketch-based generation methods mainly reconstruct the sketch itself, while recent text-driven diagram generation frameworks rely on textual semantics and do not fully exploit the topological structure contained in sketches. In this paper, we introduce DiagramRAG, a lightweight retrieval-augmented framework for sketch-based scientific diagram completion. Given a user sketch, DiagramRAG retrieves reference diagrams that are both semantically relevant to the sketch content and topologically compatible with its structure, and uses them to guide downstream diagram generation. To enable efficient structure-aware retrieval, we represent diagrams as knowledge graphs, synthesize sketch variants at different simplification levels, and train an embedding model to align sketches with compatible diagrams in a shared space. The retrieved references further provide content, topology, and visual priors for completing and rendering the final diagram. Experiments show that DiagramRAG achieves F1-scores of 0.848 and 0.802 on DiagramBank and FigureBench, respectively, and improves generation quality with the best VLM-as-a-Judge score of 7.170, while reducing inference latency to 35.48 seconds per sample. Our code and data are available at https://anonymous.4open.science/r/DiagramRAG-A262 and https://huggingface.co/datasets/anonymous-review-a262/DiagramSketch.

Abstract

Explore similar work

DiagramRAG: A Lightweight Framework to Retrieve Scientific Diagram for Figure Generation

DIAGRAMS: A Review Framework for Reasoning-Level Attribution in Diagram QA

PaperBanana-Interact: Scientific Diagram Refinement with Multi-Turn Human Feedback