While infographics have become a powerful medium for communicating data-driven stories, authoring them from scratch remains challenging, especially for novice users. Retrieving relevant exemplars from a large corpus can provide design inspiration and promote reuse, substantially lowering the barrier to infographic authoring. However, effective retrieval is difficult because users often express design intent in ambiguous natural language, while infographics embody rich and multi-faceted visual designs. As a result, keyword-based search often fails to capture design intent, and general-purpose vision-language retrieval models trained on natural images are ill-suited to the text-heavy, multi-component nature of infographics. To address these challenges, we develop an intent-aware infographic retrieval framework that better aligns user queries with infographic designs. We first conduct a formative study of how people describe infographics and derive an intent taxonomy spanning content and visual design facets. This taxonomy is then leveraged to enrich and refine free-form user queries, guiding the retrieval process with intent-specific cues. Building on the retrieved exemplars, users can adapt the designs to their own data with high-level edit intents, supported by an interactive agent that performs low-level adaptation. Both quantitative evaluations and user studies are conducted to demonstrate that our method improves retrieval quality over baseline methods while better supporting intent satisfaction and efficient infographic authoring.
Figures & tables
Figure 1: Four infographic exemplars used in the formative study to elicit participants’ queries of desired infographic exemplars.
Facet
What it captures
Typical cues in user queries
Content
Communicative goal and information organization: what should be shown and what takeaway should be emphasized.
category breakdown/share; trend or growth over time; compare A vs. B; highlight the largest component; include key numbers or summaries
Style
Holistic visual aesthetics: overall tone and look-and-feel, including palette and typography cues.
clean or minimalist; editorial or magazine-like; warm or playful; brand-like; pastel or muted colors; 3D or colorful; typography-forward
Layout
Spatial composition and narrative organization: how the infographic is arranged and read, beyond chart type alone.
vertical poster; clear hierarchy or sections; center chart with side summaries; radial layout with fixed-angle segments; dense labels or annotations; start at the positive y -axis and go clockwise
Illustration
Whether and how icons or illustrations are used, including their density and explanatory versus decorative role.
icons for each category; illustration-heavy; replace labels with pictorial symbols; scene-based background; minimal decoration
Chart Type
Primary chart or visual form used to encode data, often as shorthand for the intended visual encoding.
bar chart; stacked bars; pie or donut; rose/Nightingale chart; radial chart; ring heatmap; timeline-like chart
Table 1: Five-facet intent taxonomy derived from the formative study. The taxonomy summarizes recurring facets in users’ natural-language descriptions of desired infographic exemplars. Facets can be partially specified: a query may omit one or more facets, or express them only implicitly.
Figure 2: Facet patterns in participants’ queries. (a) A facet co-occurrence matrix computed from the natural language queries and keyword queries. (b) An annotated example query illustrates how a single query can contain multiple facet signals, motivating facet-specific rewriting in retrieval.
Figure 3: Overview of our intent-aware infographic retrieval framework. Top (training): ChartGalaxy images are described by a multimodal LLM and distilled into short facet-specific captions (two per facet). We fine-tune the full model (text encoder, image encoder and facet MLP heads) with a multi-facet in-batch contrastive loss (Eq. 5 ). Bottom (inference): A user query q is parsed into facet rewrites {qf} , facet weights w , and an optional multi-choice chart-type set Tq . We compute facet-conditioned text embeddings and compare them with precomputed facet-specific image embeddings to obtain {sf}f∈Fe , and compute a multi-choice discrete chart-type similarity s\textscT . We fuse all facets by a weighted sum to rank exemplars.
Figure 4: System UI for conversational exemplar retrieval and SVG-based adaptation. (A) chat panel for intent articulation, assistant feedback, and iterative refinement; (B) committed infographic exemplars that persist as references during adaptation; (C) rendered SVG outputs with version history; and (D) retrieval panel for exemplar discovery and commitment.
Method
Synthetic (General)
Synthetic (Multi-facet)
Human (Short)
Human (Long)
R@1 ↑
R@5 ↑
MRR@10 ↑
R@1 ↑
R@5 ↑
MRR@10 ↑
R@1 ↑
R@5 ↑
MRR@10 ↑
R@1 ↑
R@5 ↑
MRR@10 ↑
CLIP [ 26 ]
62.47
84.74
0.7203
46.42
70.83
0.5696
35.00
58.00
0.4375
41.00
67.33
0.5150
SigLIP2 [ 33 ]
19.38
38.05
0.2739
67.59
86.45
0.7566
33.00
50.00
0.4042
20.67
33.33
0.2632
MegaPairs [ 44 ]
67.42
88.18
0.7630
50.08
74.87
0.6075
39.33
65.67
0.5036
50.33
73.67
0.5913
Ours w/o In-dom. Align.
70.22
87.43
0.7760
51.38
75.76
0.6168
30.00
47.00
0.3732
35.00
57.33
0.4424
Ours w/o Facets
92.79
98.88
0.9545
89.67
97.60
0.9332
53.67
77.33
0.6345
65.00
82.33
0.7237
Table 2: Single-round query-to-image retrieval. We report Recall@1/5 (%) and MRR@10 (fraction) on synthetic general queries, synthetic multi-facet queries, and human-written paired short/long queries. ↑ higher is better.
Mean score (1–5) ↑
Preference vs. MegaPairs (%)
Method
Short
Long
Overall
Win
Tie
Loss
MegaPairs
3.09
3.26
3.17
–
–
–
Ours
4.20
4.51
4.35
57.3
37.3
5.3
Table 3: Human scoring of top-5 retrieved results. ↑ higher is better.
Category
Metric
Baseline
Ours
p
Efficiency
Rounds-to-stop ↓
3.29 ± 2.27
1.42 ± 0.50
0.0029
Retrieval
FoundRate ↑
45.8% (11/24)
91.7% (22/24)
0.0018
dCRR@10 ↑
0.21 ± 0.34
0.73 ± 0.38
5.7×10−5
Utility (1–7)
BestSatisfaction ↑
5.50 ± 1.62
6.88 ± 0.45
0.0029
Table 4: Multi-round retrieval. We report task-level mean ± SD for continuous metrics and percentage (count) for binary metrics. ↓ lower is better; ↑ higher is better.
Category
Metric
Baseline
Ours
p
Process
Dialogue turns
4.83 ± 1.70
5.17 ± 1.47
0.594
Workload (0–20, ↓ )
Overall workload
11.23 ± 4.13
7.88 ± 2.08
0.034
Mental demand
9.25 ± 5.74
6.00 ± 3.33
0.036
Temporal demand
12.50 ± 5.16
7.75 ± 3.62
0.028
Effort
13.50 ± 3.48
10.67 ± 3.42
0.055
Frustration
9.67 ± 6.14
7.08 ± 3.50
0.160
Table 5: Retrieval-based authoring. We report mean ± SD for continuous metrics and percentage for preference rate. ↓ lower is better; ↑ higher is better. p values are from paired two-sided t -tests.
Reliable infographic generation requires facts, symbols, and visual relations to remain consistent through rendering and revision. Correcting one element also requires tracking its supporting evidence and the dependencies affected by the change. We present InfoAgent, a training-free framework for evidence-bound visual-symbolic program synthesis. Its Infographic Visual Description (IVD) records factual payloads, evidence provenance, execution routes, and verification obligations in a typed dependency graph. Retrieved design priors guide compilation, and layered execution combines raster synthesis with editable symbolic and binding objects while retaining their traces. Dependency-aware repair localizes corrections, rechecks affected dependencies, and requires protected obligations to remain satisfied under the declared checkers. Unresolved obligations remain explicit. On IGenBench, InfoAgent achieves 93.0 Q-ACC and 59.0 I-ACC. We also introduce InfoGraphicBench-Evidence, where complete-checklist pass rates on 200 test requests increase from 21.5% for Same-IVD Prompt to 23.5% for the initial layered output and 28.5% after repair, using the same evidence and initial IVD. On 120 audited repair cases, localized repair edits 12.4% of the canvas on average, compared with 67.3% for global regeneration.
Yifan Li, Tong Li, Qi Zeng +6
University of Science and Technology of China · Meituan · Peking University, Peking University
Multimodal foundation models edit natural photographs at production quality, yet the same models struggle with structured visual content such as infographics. Unlike photographs, infographics encode information through logical relations; editing one element often requires surrounding elements to be adapted. We refer to this global layout reasoning capability as reflow. Existing image-editing benchmarks neither provide a dedicated setting for structured visual content nor evaluate the reflow capability. We introduce InfoEdit, a novel benchmark of 1,000 infographics across eight logical-relation families, paired with 4,000 editing instructions across four editing tasks, and a reflow-aware evaluation protocol. Across eight frontier editors, only GPT-Image-2 clears 60% average success rate; most models fall below 7%, and no editor exceeds 36% on the Swap-Block task even with perfect target localization. We further show that code-level editing can match the strongest pixel-level editor, revealing complementary strengths across tasks. InfoEdit identifies reflow as a central challenge in structured visual content editing and provides a diagnostic benchmark to facilitate future progress.
Cheng Yang, Chufan Shi, Huijuan Wang +6
University of California San Diego · University of Southern California · University of Illinois Urbana-Champaign +1
Large Language Models (LLMs) can produce detailed answers to complex queries, but these answers are typically presented as dense linear text, which makes fine-grained inspection, navigation, and return visits difficult. We present ChatImage, a system that converts long-form LLM answers into interactive visual images. Given a textual answer, ChatImage first normalizes its content into structured visual modules, plans a visual layout, and renders a coherent image. It then applies a second grounding pass to the rendered image with vision grounding models such as LocateAnything and MiMo-Vision, with optional SAM-style mask refinement, to identify the visible regions that should support interaction. From these grounded regions, ChatImage overlays transparent clickable hotspots on the image. Each hotspot opens a detail panel and a region-scoped follow-up thread, allowing the user to inspect and query a specific part of the answer without re-reading the full response. Instead of treating planned coordinates as the final interaction geometry, ChatImage uses them as priors and grounds the interaction targets after rendering, which improves consistency between visual content and clickable regions. We release a reference implementation and introduce a 30-question benchmark covering infographic, map, and scene-based answer formats. Evaluation with configured external models reports interaction-loop completion, a strict visual-alignment gate, and a SAM-based mask-completeness diagnostic.