Show Me the Infographic I Imagine: Intent-Aware Infographic Retrieval for Authoring Support

Jing Xu; Jiarui Hu; Zhihao Shuai; Yiyun Chen; Weikai Yang

cs.IRApr 9, 2026

Show Me the Infographic I Imagine: Intent-Aware Infographic Retrieval for Authoring Support

Authors: Jing Xu, Jiarui Hu, Zhihao Shuai, Yiyun Chen, Weikai Yang

Organizations: The Hong Kong University of Science and Technology (Guangzhou) · Nanjing University

Abstract

While infographics have become a powerful medium for communicating data-driven stories, authoring them from scratch remains challenging, especially for novice users. Retrieving relevant exemplars from a large corpus can provide design inspiration and promote reuse, substantially lowering the barrier to infographic authoring. However, effective retrieval is difficult because users often express design intent in ambiguous natural language, while infographics embody rich and multi-faceted visual designs. As a result, keyword-based search often fails to capture design intent, and general-purpose vision-language retrieval models trained on natural images are ill-suited to the text-heavy, multi-component nature of infographics. To address these challenges, we develop an intent-aware infographic retrieval framework that better aligns user queries with infographic designs. We first conduct a formative study of how people describe infographics and derive an intent taxonomy spanning content and visual design facets. This taxonomy is then leveraged to enrich and refine free-form user queries, guiding the retrieval process with intent-specific cues. Building on the retrieved exemplars, users can adapt the designs to their own data with high-level edit intents, supported by an interactive agent that performs low-level adaptation. Both quantitative evaluations and user studies are conducted to demonstrate that our method improves retrieval quality over baseline methods while better supporting intent satisfaction and efficient infographic authoring.

Figures & tables

Figure 1: Four infographic exemplars used in the formative study to elicit participants’ queries of desired infographic exemplars.

Facet	What it captures	Typical cues in user queries
Content	Communicative goal and information organization: what should be shown and what takeaway should be emphasized.	category breakdown/share; trend or growth over time; compare A vs. B; highlight the largest component; include key numbers or summaries
Style	Holistic visual aesthetics: overall tone and look-and-feel, including palette and typography cues.	clean or minimalist; editorial or magazine-like; warm or playful; brand-like; pastel or muted colors; 3D or colorful; typography-forward
Layout	Spatial composition and narrative organization: how the infographic is arranged and read, beyond chart type alone.	vertical poster; clear hierarchy or sections; center chart with side summaries; radial layout with fixed-angle segments; dense labels or annotations; start at the positive $y$ -axis and go clockwise
Illustration	Whether and how icons or illustrations are used, including their density and explanatory versus decorative role.	icons for each category; illustration-heavy; replace labels with pictorial symbols; scene-based background; minimal decoration
Chart Type	Primary chart or visual form used to encode data, often as shorthand for the intended visual encoding.	bar chart; stacked bars; pie or donut; rose/Nightingale chart; radial chart; ring heatmap; timeline-like chart

Table 1: Five-facet intent taxonomy derived from the formative study. The taxonomy summarizes recurring facets in users’ natural-language descriptions of desired infographic exemplars. Facets can be partially specified: a query may omit one or more facets, or express them only implicitly.

Figure 2: Facet patterns in participants’ queries. (a) A facet co-occurrence matrix computed from the natural language queries and keyword queries. (b) An annotated example query illustrates how a single query can contain multiple facet signals, motivating facet-specific rewriting in retrieval.

Figure 3: Overview of our intent-aware infographic retrieval framework. Top (training): ChartGalaxy images are described by a multimodal LLM and distilled into short facet-specific captions (two per facet). We fine-tune the full model (text encoder, image encoder and facet MLP heads) with a multi-facet in-batch contrastive loss (Eq. 5 ). Bottom (inference): A user query $q$ is parsed into facet rewrites $\{q_{f}\}$ , facet weights $\mathbf{w}$ , and an optional multi-choice chart-type set $\mathcal{T}_{q}$ . We compute facet-conditioned text embeddings and compare them with precomputed facet-specific image embeddings to obtain $\{s_{f}\}_{f\in\mathcal{F}_{e}}$ , and compute a multi-choice discrete chart-type similarity $s_{\textsc{T}}$ . We fuse all facets by a weighted sum to rank exemplars.

Figure 4: System UI for conversational exemplar retrieval and SVG-based adaptation. (A) chat panel for intent articulation, assistant feedback, and iterative refinement; (B) committed infographic exemplars that persist as references during adaptation; (C) rendered SVG outputs with version history; and (D) retrieval panel for exemplar discovery and commitment.

Method	Synthetic (General)			Synthetic (Multi-facet)			Human (Short)			Human (Long)
	R@1 $\uparrow$	R@5 $\uparrow$	MRR@10 $\uparrow$	R@1 $\uparrow$	R@5 $\uparrow$	MRR@10 $\uparrow$	R@1 $\uparrow$	R@5 $\uparrow$	MRR@10 $\uparrow$	R@1 $\uparrow$	R@5 $\uparrow$	MRR@10 $\uparrow$
CLIP [ 26 ]	62.47	84.74	0.7203	46.42	70.83	0.5696	35.00	58.00	0.4375	41.00	67.33	0.5150
SigLIP2 [ 33 ]	19.38	38.05	0.2739	67.59	86.45	0.7566	33.00	50.00	0.4042	20.67	33.33	0.2632
MegaPairs [ 44 ]	67.42	88.18	0.7630	50.08	74.87	0.6075	39.33	65.67	0.5036	50.33	73.67	0.5913
Ours w/o In-dom. Align.	70.22	87.43	0.7760	51.38	75.76	0.6168	30.00	47.00	0.3732	35.00	57.33	0.4424
Ours w/o Facets	92.79	98.88	0.9545	89.67	97.60	0.9332	53.67	77.33	0.6345	65.00	82.33	0.7237

Table 2: Single-round query-to-image retrieval. We report Recall@1/5 (%) and MRR@10 (fraction) on synthetic general queries, synthetic multi-facet queries, and human-written paired short/long queries.

\uparrow

higher is better.

	Mean score (1–5) $\uparrow$			Preference vs. MegaPairs (%)
Method	Short	Long	Overall	Win	Tie	Loss
MegaPairs	3.09	3.26	3.17	–	–	–
Ours	4.20	4.51	4.35	57.3	37.3	5.3

Table 3: Human scoring of top-5 retrieved results.

\uparrow

higher is better.

Category	Metric	Baseline	Ours	$p$
Efficiency	Rounds-to-stop $\downarrow$	3.29 $\pm$ 2.27	1.42 $\pm$ 0.50	0.0029
Retrieval	FoundRate $\uparrow$	45.8% (11/24)	91.7% (22/24)	0.0018
	dCRR@10 $\uparrow$	0.21 $\pm$ 0.34	0.73 $\pm$ 0.38	$5.7\times 10^{-5}$
Utility (1–7)	BestSatisfaction $\uparrow$	5.50 $\pm$ 1.62	6.88 $\pm$ 0.45	0.0029

Table 4: Multi-round retrieval. We report task-level mean

\pm

SD for continuous metrics and percentage (count) for binary metrics.

\downarrow

lower is better;

\uparrow

higher is better.

Category	Metric	Baseline	Ours	$p$
Process	Dialogue turns	4.83 $\pm$ 1.70	5.17 $\pm$ 1.47	0.594
Workload (0–20, $\downarrow$ )	Overall workload	11.23 $\pm$ 4.13	7.88 $\pm$ 2.08	0.034
	Mental demand	9.25 $\pm$ 5.74	6.00 $\pm$ 3.33	0.036
	Temporal demand	12.50 $\pm$ 5.16	7.75 $\pm$ 3.62	0.028
	Effort	13.50 $\pm$ 3.48	10.67 $\pm$ 3.42	0.055
	Frustration	9.67 $\pm$ 6.14	7.08 $\pm$ 3.50	0.160

Table 5: Retrieval-based authoring. We report mean

\pm

SD for continuous metrics and percentage for preference rate.

\downarrow

lower is better;

\uparrow

higher is better.

p

values are from paired two-sided

t

-tests.

Information Retrieval

Information Retrieval studies how to find, rank, and organize relevant information in large collections of data.

See all 471 papers

InfoAgent: Traceable Generation and Repair of Evidence-Grounded Infographics
Sep 30, 2026Yifan Li, Tong Li, Qi Zeng +6Data Visualization Data Provenance
University of Science and Technology of China · Meituan · Peking University, Peking University
InfoEdit: Probing Global Layout Reasoning in Infographic Editing
Sep 27, 2026Cheng Yang, Chufan Shi, Huijuan Wang +6Image Editing Evaluation Text-Guided Image Editing
University of California San Diego · University of Southern California · University of Illinois Urbana-Champaign +1
ChatImage: Navigating Long-Form LLM Answers through Interactive Images
Jul 6, 2026Wencan Jiang, Jiangning Zhang, Yong Liu
1Zhejiang University

Sep 30, 2026J/K move · Enter open · S save

InfoAgent: Traceable Generation and Repair of Evidence-Grounded Infographics

Yifan Li, Tong Li, Qi Zeng, Lishuai Gao+5

University of Science and Technology of China · Meituan · Peking University, Peking University

Data Visualization Data Provenance Program Synthesis Evidence-Grounded Generation

Open paper AskPDF arXiv

Reliable infographic generation requires facts, symbols, and visual relations to remain consistent through rendering and revision. Correcting one element also requires tracking its supporting evidence and the dependencies affected by the change. We present InfoAgent, a training-free framework for evidence-bound visual-symbolic program synthesis. Its Infographic Visual Description (IVD) records factual payloads, evidence provenance, execution routes, and verification obligations in a typed dependency graph. Retrieved design priors guide compilation, and layered execution combines raster synthesis with editable symbolic and binding objects while retaining their traces. Dependency-aware repair localizes corrections, rechecks affected dependencies, and requires protected obligations to remain satisfied under the declared checkers. Unresolved obligations remain explicit. On IGenBench, InfoAgent achieves 93.0 Q-ACC and 59.0 I-ACC. We also introduce InfoGraphicBench-Evidence, where complete-checklist pass rates on 200 test requests increase from 21.5% for Same-IVD Prompt to 23.5% for the initial layered output and 28.5% after repair, using the same evidence and initial IVD. On 120 audited repair cases, localized repair edits 12.4% of the canvas on average, compared with 67.3% for global regeneration.

Abstract

Information Retrieval

Explore similar work

InfoAgent: Traceable Generation and Repair of Evidence-Grounded Infographics

InfoEdit: Probing Global Layout Reasoning in Infographic Editing

ChatImage: Navigating Long-Form LLM Answers through Interactive Images