PosterIQ: A Design Perspective Benchmark for Poster Understanding and Generation

Yuheng Feng; Wen Zhang; Haodong Duan; Xingxing Zou

cs.CVMar 25, 2026

PosterIQ: A Design Perspective Benchmark for Poster Understanding and Generation

Authors: Yuheng Feng, Wen Zhang, Haodong Duan, Xingxing Zou

Organizations: The Hong Kong Polytechnic University · Snapchat Inc. · ByteDance Seed

Abstract

We present PosterIQ, a design-driven benchmark for poster understanding and generation, annotated across composition structure, typographic hierarchy, and semantic intent. It includes 7,765 image-annotation instances and 822 generation prompts spanning real, professional, and synthetic cases. To bridge visual design cognition and generative modeling, we define tasks for layout parsing, text-image correspondence, typography/readability and font perception, design quality assessment, and controllable, composition-aware generation with metaphor. We evaluate state-of-the-art MLLMs and diffusion-based generators, finding persistent gaps in visual hierarchy, typographic semantics, saliency control, and intention communication; commercial models lead on high-level reasoning but act as insensitive automatic raters, while generators render text well yet struggle with composition-aware synthesis. Extensive analyses show PosterIQ is both a quantitative benchmark and a diagnostic tool for design reasoning, offering reproducible, task-specific metrics. We aim to catalyze models' creativity and integrate human-centred design principles into generative vision-language systems.

Figures & tables

Figure 1: Overview of the benchmark, which includes over a dozen tasks

		Logo OCR	Poster OCR	Simple OCR	hard OCR	$\Delta$	Font Size OCR
Model		$AC$ $\uparrow$	$AC$ $\uparrow$	$WR$ $\uparrow$	$WR$ $\uparrow$	$\delta$ $\downarrow$	$WR$ $\uparrow$	$Std$ $\downarrow$
Closed	GPT-5	0.952	0.922	0.965	0.496	0.469	0.885	0.113
	Claude-Sonnet-4.5	0.902	0.884	0.951	0.579	0.372	0.878	0.035
	Gemini-2.5-Pro	0.923	0.952	0.997	0.472	0.525	0.879	0.037
	Grok-4-fast	0.440	0.834	0.769	0.044	0.725	0.288	0.105
Open	MiniCPM-V-4.5	0.883	0.932	0.989	0.521	0.468	0.865	0.023

Table 1: Comprehensive OCR benchmark results across multiple visual text recognition tasks.

AC

denotes the accuracy, and

WR

denotes the word-level recall rate.

\Delta

represents the performance gap between the simple and hard OCR settings, reflecting model robustness.

Std

denotes the standard deviation of

WR

across different font sizes, indicating stability.

Table 3

		Text Localization				Text Positioning		Empty-Space		Layout Comparison	Layout Generation
Model		Top-1 IoU $\uparrow$	Top-3 IoU $\uparrow$	Mean IoU $\uparrow$	Recall $\uparrow$	Alignment $\uparrow$	Rotation $\uparrow$	Mean IoU $\uparrow$	Match Acc. $\uparrow$	Score $\uparrow$	Center Bias $\downarrow$	Area Ratio $\uparrow$	Recall $\uparrow$
Closed	GPT-5	0.432	0.308	0.171	0.971	0.347	0.480	0.384	0.820	0.719	0.084	0.484	1.000
	Claude-Sonnet-4.5	0.163	0.104	0.060	0.905	0.468	0.488	0.300	0.976	0.648	0.145	0.423	0.953
	Gemini-2.5-Pro	0.491	0.404	0.295	0.899	0.325	0.576	0.491	0.982	0.680	0.084	0.569	1.000
	Grok-4-fast	0.087	0.065	0.033	0.978	0.205	0.129	0.241	0.784	0.172	0.130	0.439	1.000
Open	MiniCPM-V-4.5	0.269	0.228	0.150	0.992	0.017	0.232	0.280	0.407	0.305	0.243	0.376	0.991

Table 4: Comparison of layout reasoning tasks across models.

		Overall Rating
Model		Sim $\uparrow$
Closed	GPT-5	0.347
	Claude-Sonnet-4.5	0.384
	Gemini-2.5-Pro	0.399
	Grok-4-fast	0.465
Open	MiniCPM-V 4.5	0.095

Table 5: Comparison of Overall Rating

Figure 2: Qualitative comparison of four models on three layout-related tasks. For Text Localization and Layout Generation , the predicted bounding boxes are shown in red. For the Empty Space task, the selected patch IDs are highlighted in the image.

	Dense	Font	Style	Composition	Intention
Model	$PS$ $\uparrow$	$R$ $\uparrow$	$S$ $\uparrow$	$PS$ $\uparrow$	$PS$ $\uparrow$	Average $\uparrow$
Seedream-4.0	0.618	0.342	0.591	0.848	0.645	0.609
Gemini-2.5-Flash-Image	0.622	0.391	0.590	0.866	0.663	0.626
GPT-Image-1	0.508	0.299	0.633	0.856	0.670	0.593
Qwen-Image	0.464	0.286	0.620	0.801	0.589	0.552

Table 6: Performance of image generation models across five evaluation tasks. Point Score (

PS

), Score (

S

), Richness (

R

)

Figure 3: Qualitative comparison of four models on five generation tasks.

Figure 4: Qualitative comparison of model outputs over supervision-guided iterations.

Model	OCR	Font	Layout	Understanding	Overall Rating	Average
GPT-5	0.838	0.615	0.511	0.801	0.347	0.622
Claude-Sonnet-4.5	0.838	0.603	0.453	0.727	0.384	0.601
Gemini-2.5-Pro	0.855	0.606	0.571	0.806	0.399	0.647
Grok-4-fast	0.475	0.420	0.313	0.682	0.465	0.471
MiniCPM-V-4.5	0.838	0.421	0.325	0.652	0.095	0.466
Gemma-3n-e4b-it	0.744	0.381	0.287	0.538	0.483	0.486

Table 7: Average scores across understanding tasks. The average is computed by aggregating only these positively correlated scores.

Appendix figures & tables24 assets

Supplementary material from the paper’s appendix.

Appendix

Panel of Figure 5 — Figure 5: Benchmark statistics for understanding tasks (top) and generation tasks (bottom).

Image Generation

Image generation studies models that create or edit images from text, examples, layouts, and other structured controls.

See all 444 papers

PosterHarness: Turning Scientific Poster Generation into an Auditable Instruction-Following Benchmark
Jul 3, 2026Tianyi Yang, Dawei Fu, Youpeng Wu +5AI-Assisted Scientific Research Image Generation
School of Physics, Peking University
Any2Poster: Any-Source Poster Generation Across Modalities and Domains
Jun 1, 2026Amogh Vinaykumar, Aiden Li, Suozhi Huang +1Image Generation Multimodal Generation
Flower Mound High School · University College London · Princeton University
From Prompting to Composing: A Spatial Canvas Interface for Poster Generation
Oct 8, 2026Yitong Wang, Fangyun Wei, Jinjing Zhao +5Controllable Image Generation Image Generation
Fudan University · Microsoft Research · The University of Sydney +3

Jul 3, 2026J/K move · Enter open · S save

PosterHarness: Turning Scientific Poster Generation into an Auditable Instruction-Following Benchmark

Tianyi Yang, Dawei Fu, Youpeng Wu, Zixun Kou+4

School of Physics, Peking University

AI-Assisted Scientific Research Image Generation Image Generation Evaluation Controllable Image Generation

Open paper AskPDF arXiv

Text-rich image models can now design poster-scale layouts, but we lack ways to measure whether they honor scientific communication contracts: legible labels, prescribed aspect ratios, and -- above all -- abstaining from fabricated scientific figures. We present POSTERHARNESS, an auditable harness reframing poster generation as measurable instruction-following tasks, with a pilot benchmark and failure taxonomy. POSTERHARNESS uses a placeholder-first contract to separate two jobs models otherwise conflate. The model performs visual-summary design: typography, reading path, color, and background -- but never draws data-bearing figures. Every figure region must be an empty labeled placeholder; a deterministic compositor inserts real source-paper figures at detected coordinates. This makes properties measurable: placeholder count and ID accuracy, blankness, aspect-ratio compliance, abstention from synthesized graphics, public-text hygiene, and source-figure provenance -- with failures logged as explicit rejections, not hidden in plausible-looking output. We instantiate the harness on 12 papers (6 HEP, 6 AI/ML-adjacent) and report three findings. (i) A counterfactual probe shows the placeholder contract drives VLM-counted synthesized figures from 34 to 0 across three papers. (ii) A failure taxonomy identifies blocking contracts: placeholder geometry, placeholder QA, template critic, and public text. (iii) Comparison with Paper2Poster shows a trade-off: PosterHarness yields higher-resolution artifacts, lower white-canvas fraction, and stronger VLM visual preference; the deterministic baseline retains slightly more PosterQuiz-style information and runs faster. We report this as regime characterization, not a superiority claim. All artifacts, prompts, manifests, and audit scripts are released as a reusable evaluation component.

Abstract

Image Generation

Explore similar work

PosterHarness: Turning Scientific Poster Generation into an Auditable Instruction-Following Benchmark

Any2Poster: Any-Source Poster Generation Across Modalities and Domains

From Prompting to Composing: A Spatial Canvas Interface for Poster Generation