WorldGenBench: A World-Knowledge-Integrated Benchmark for Reasoning-Driven Text-to-Image Generation

Ruoshi Xu; Daoan Zhang; Che Jiang; Biaoxiang Chen; Zijian Jin; Yutian Lu; Jianguo Zhang; Liang Yong

cs.CVMay 2, 2025

WorldGenBench: A World-Knowledge-Integrated Benchmark for Reasoning-Driven Text-to-Image Generation

Authors: Ruoshi Xu, Daoan Zhang, Che Jiang, Biaoxiang Chen, Zijian Jin, Yutian Lu, Jianguo Zhang, Liang Yong, +2 more

Organizations: Southern University of Science and Technology · University of Rochester · New York University · Datawhale org. · Chinese Medicine Guangdong Laboratory

Abstract

Recent text-to-image (T2I) models achieve high visual fidelity and accurate explicit grounding from input prompts. However, image generation is fundamentally a process of semantic expansion. Beyond surface-level descriptions, generating authentic visual content requires supplementing prompts with unstated cues, a crucial step that current models frequently overlook. In response, we introduce WorldGenBench, a benchmark designed to evaluate how well T2I systems integrate factual grounding with contextual inference. Unlike prior work that isolates these abilities, WorldGenBench emphasizes their synergy, recognizing that true understanding requires both knowing and reasoning. To enable interpretable evaluation, we propose the Knowledge Checklist Score, which assesses whether generated images faithfully reflect key explicit and implicit semantic elements. Evaluating 24 models reveals persistent gaps: proprietary models like GPT-4o show strong integration, while most open-source systems lag, particularly in culturally diverse and scientific scenarios. Our analysis further identifies common failure modes, including semantic hallucination and the underrepresentation of non-Western knowledge, underscoring the limitations of current models in achieving real-world readiness. These findings highlight the need to guide future T2I models toward deeper, knowledge-aware generation. Project Page: https://dwanzhang-ai.github.io/WorldGenBench/

Figures & tables

Figure 1. The presence of world knowledge and the emergence of implicit reasoning capabilities are fundamental to building a high-quality text-to-image model, and are essential prerequisites for achieving real-world ready generation.

Benchmark	World Knowledge	Implicit Reasoning	Knowledge Diversity	Checklist Score
Imagine-E ( Lei et al., 2025 )	✗	✗	✗	✗
PhyBench ( Meng et al., 2024 )	✗	✓	✗	✗
Commonsense-T2I ( Fu et al., 2024 )	✓	✗	✗	✗
WISE ( Niu et al., 2025 )	✓	✗	✓	✗
MMMG ( Luo et al., 2025 )	✓	✗	✗	✗
R2I-Bench ( Chen et al., 2025a )	✓	✗	✗	✗

Table 1. Comparison of the proposed WorldGenBench with existing T2I benchmarks.

Figure 2. WorldGenBench: Humanities and Nature prompts by region and category.

Figure 3. The construction and evaluation pipeline of WorldGenBench.

Panel of Figure 4 — Figure 4. Each WorldGenBench case includes a natural-language prompt and three key visual checkpoints (out of 10). Left : Humanities (historical grounding); Right : Nature (scientific schematic fidelity). Within each checklist, the bolded part represents the expected visual item , while the gray text provides the corresponding explanation or rationale .

Figure 5. An example case from GPT4o. In the checklist, red text indicates that the model did not score on the corresponding item, while green text indicates that the model received a score. The red-text items underscore a critical gap in the model’s capabilities: the discrepancy between generating images from general knowledge versus expert, domain-specific information. For instance, the model failed to render accurate botanical details (’African pearwood’) and specific indigenous technologies (’Bark ropes’), demonstrating a limitation in factual and cultural fidelity, even while the overall scene appears plausible.

Panel of Table 2 — Table 2. Knowledge Checklist Score on the Humanities split. The scores on the left and right of the slash (/) are evaluated by GPT-4o and Gemini 2.5 Pro, respectively. Columns report per-continent scores ( AF =Africa, AN =Antarctica, AS =Asia, EU =Europe, NA =North America, OC =Oceania, SA =South America; Avg. =mean across continents). The upper block lists diffusion models, the middle block lists autoregressive models, and the lower block contains proprietary systems.

Panel of Table 3 — Table 3. Knowledge Checklist Score on the Nature split. The scores on the left and right of the slash (/) are evaluated by GPT-4o and Gemini 2.5 Pro, respectively. Columns denote scientific disciplines ( ASTR =Astronomy, BIO&MED =Biology & Medicine, CHEM =Chemistry, EASC =Earth Sciences, PHYS =Physics, XDIS =Cross-Disciplinary, Avg. =mean across disciplines). The upper block lists diffusion models, the middle block lists autoregressive models, and the lower block contains proprietary systems.

Figure 6. Correlation on 50 sampled prompts between human checklist scores and automated scores from three VLMs. The high Pearson correlations for GPT-4o, Gemini 2.5 Pro, and Qwen2.5-VL 72B demonstrate strong human-VLM consistency, with most data points falling along the y=x line (Blue). In contrast, the VLM-as-a-judge method fails to align with the human checklist score, as its sample points are scattered and do not follow the diagonal (Red).

Figure 7. Mean Knowledge Score profiles for diffusion, autoregressive, and proprietary models on the WorldGenBench benchmark. The top chart compares performance across six natural science fields, and the bottom chart compares performance across seven continents for humanities. Shaded areas represent variance across models within each model type. Proprietary models consistently outperform diffusion and autoregressive models across all domains.

Figure 8. Correlation of GPT-4o and human evaluations. Results across six text-to-image models show strong alignment (Pearson $r$ from 0.70 to 0.91), demonstrating the evaluation’s reliability against loop bias across different model families.

Panel of Table 4 — Table 4. Performance on the Humanities split. Columns report per-continent scores ( AF =Africa, AN =Antarctica, AS =Asia, EU =Europe, NA =North America, OC =Oceania, SA =South America; Avg. =mean across continents). The upper block lists diffusion-based models, the middle block lists AR-based models, and the lower block contains proprietary systems.

Panel of Table 5 — Table 5. Performance on the Nature split. Columns denote scientific disciplines ( ASTR =Astronomy, BIO =Biology, CHEM =Chemistry, EASC =Earth Sciences, PHYS =Physics, XDIS =Cross-Disciplinary, Avg. =mean across disciplines). The upper block lists diffusion-based models, the middle block lists AR-based models, and the lower block contains proprietary systems.

Model	Aesthetic	ImageReward	PickScore
GoT	0.9537	0.6343	0.7947
JanusFlow1.3B	0.8827	0.3800	0.7432
Lumina-gpt2	0.9362	-0.3082	0.3778
Ming	0.9764	0.7692	0.8669
SDXL	0.9729	-0.1842	0.9211
T2I-R1	0.7854	0.8011	0.7318

Table 6. Pearson correlation coefficients between WorldGenBench scores and three traditional metrics (Aesthetic Score, ImageReward, and PickScore) across all evaluated models.

Figure 9. Prompt used for benchmarking with the default model, GPT-4o

Figure 10. Visual Case 1: An example case from the humanities domain, including the prompt and the results from GPT-4o, HiDreaml1-Full, and Show-o. Checklist is in 7.1

Figure 11. Visual Case 2: An example case from the Nature domain, including the prompt and the results from GPT-4o, HiDreaml1-Full, and Show-o. Checklist is in 7.2

Class / Model	JF1.3B	JP1B	JP7B	Lu	fdev	fsch	got	4o	hm	hdl1	igv2	mj	pg	px	sd35l	sd35m	sdxl	showo	sarrl	sarsft	vl
Algeria	0.00	0.03	0.00	0.13	0.03	0.20	0.07	0.47	0.17	0.17	0.23	0.03	0.23	0.17	0.17	0.07	0.03	0.20	0.03	0.07	0.10
Angola	0.00	0.00	0.00	0.07	0.07	0.05	0.00	0.13	0.00	0.05	0.00	0.05	0.15	0.00	0.00	0.05	0.10	0.10	0.05	0.00	0.00
Benin	0.03	0.00	0.03	0.07	0.00	0.10	0.07	0.07	0.05	0.13	0.13	0.10	0.05	0.00	0.10	0.05	0.10	0.07	0.07	0.07	0.00
Botswana	0.03	0.07	0.07	0.00	0.17	0.23	0.13	0.30	0.15	0.20	0.03	0.30	0.30	0.00	0.17	0.13	0.10	0.13	0.13	0.10	0.10
Burkina Faso	0.00	0.00	0.03	0.05	0.10	0.10	0.10	0.30	0.10	0.13	0.00	0.30	0.10	0.20	0.07	0.07	0.10	0.03	0.03	0.03	0.00
Burundi	0.00	0.00	0.10	0.00	0.13	0.17	0.13	0.20	0.17	0.27	0.07	0.25	0.17	0.10	0.10	0.20	0.17	0.20	0.10	0.10	0.00

Table 7. Detailed breakdown of average performance scores for each model across specific countries. Best viewed digitally due to the large number of entries.

Explore similar work

Knowledge Visualization: A Benchmark and Method for Knowledge-Intensive Text-to-Image Generation
Apr 24, 2026Ran Zhao, Sheng Jin, Size Wu +5T2I Generation Image Generation
Huazhong University of Science and Technology · The University of Hong Kong · S-Lab, Nanyang Technological University
Are Text-to-Image Models Inductivist Turkeys? A Counterfactual Benchmark for Causal Reasoning
Jun 23, 2026Jiayi Lei, Yuandong Pu, Xingyu Han +8Causal Counterfactual Generation T2I Generation
Shanghai Jiao Tong University · Shanghai AI Laboratory · The Chinese University of Hong Kong
ImagenWorld: Stress-Testing Image Generation Models with Explainable Human Evaluation on Open-ended Real-World Tasks
Mar 29, 2026Samin Mahdizadeh Sani, Max Ku, Nima Jamali +23Image Generation Evaluation T2I Generation Evaluation
University of Waterloo · G-G-G · Independent +1

Apr 24, 2026J/K move · Enter open · S save

Knowledge Visualization: A Benchmark and Method for Knowledge-Intensive Text-to-Image Generation

Ran Zhao, Sheng Jin, Size Wu, Kang Liao+4

Huazhong University of Science and Technology · The University of Hong Kong · S-Lab, Nanyang Technological University

T2I Generation Image Generation T2I Generation Evaluation Controllable Image Generation

Open paper AskPDF arXiv

Recent text-to-image (T2I) models have demonstrated impressive capabilities in photorealistic synthesis and instruction following. However, their reliability in knowledge-intensive settings remains largely unexplored. Unlike natural image generation, knowledge visualization requires not only semantic alignment but also strict adherence to domain knowledge, structural constraints, and symbolic conventions, exposing a critical gap between visual plausibility and scientific correctness. To systematically study this problem, we introduce KVBench, a curriculum-grounded benchmark for evaluating knowledge-intensive T2I generation. KVBench covers six senior high-school subjects: Biology, Chemistry, Geography, History, Mathematics, and Physics. The benchmark consists of 1,800 expert-curated prompts derived from over 30 authoritative textbooks. Using this benchmark, we evaluate 14 state-of-the-art open- and closed-source models, revealing substantial deficiencies in logical reasoning, symbolic precision, and multilingual robustness, with open-source models consistently underperforming proprietary systems. To address these limitations, we further propose KE-Check, a two-stage framework that improves scientific fidelity via (1) Knowledge Elaboration for structured prompt enrichment, and (2) Checklist-Guided Refinement for explicit constraint enforcement through violation identification and constraint-guided editing. KE-Check effectively mitigates scientific hallucinations, narrowing the performance gap between open-source and leading closed-source models. Data and codes are publicly available at https://github.com/zhaoran66/KVBench.

Abstract

Explore similar work

Knowledge Visualization: A Benchmark and Method for Knowledge-Intensive Text-to-Image Generation

Are Text-to-Image Models Inductivist Turkeys? A Counterfactual Benchmark for Causal Reasoning

ImagenWorld: Stress-Testing Image Generation Models with Explainable Human Evaluation on Open-ended Real-World Tasks