Although image generation has boosted various applications via its rapid evolution, whether the state-of-the-art models are able to produce ready-to-use academic illustrations for papers is still largely unexplored. Directly comparing or evaluating the illustration with VLM is native but requires oracle multi-modal understanding ability, which is unreliable for long and complex texts and illustrations. To address this, we propose AIBench, the first benchmark using VQA for evaluating logic correctness of the academic illustrations and VLMs for assessing aesthetics. In detail, we designed four levels of questions proposed from a logic diagram summarized from the method part of the paper, which query whether the generated illustration aligns with the paper on different scales. Our VQA-based approach raises more accurate and detailed evaluations on visual-logical consistency while relying less on the ability of the judger VLM. With our high-quality AIBench, we conduct extensive experiments and conclude that the performance gap between models on this task is significantly larger than general ones, reflecting their various complex reasoning and high-density generation ability. Further, the logic and aesthetics are hard to optimize simultaneously as in handcrafted illustrations. Additional experiments further state that test-time scaling on both abilities significantly boosts the performance on this task.
Figures & tables
Figure 1: Overview of AIBench. We introduce a comprehensive benchmark to evaluate academic illustration generation through two primary dimensions: question-answering-based logical evaluation and model-based aesthetic assessment. To systematically assess logical accuracy, we collect top-tier conference papers, construct text-to-logic directed graphs, and manually annotate QA pairs across four hierarchical levels. Ultimately, our AIBench establishes a new evaluation standard for generated academic illustrations.
Benchmark
Data Construction
# Papers
# Avg. Eval Unit
Eval Method
Granularity
PaperBanana [ 55 ]
Auto
292
1
VLM-as-Judge
Coarse
AutoFigure [ 57 ]
Auto
3,300
1
VLM-as-Judge
Coarse
AIBench (Ours)
Auto + Human
300
19.01
QA-Based
Fine-grained
Table 1: Comparison between our AIBench and related benchmarks.
Figure 2: QA data construction pipeline of AIBench. We construct the multi-level QA pairs with the help of Gemini, followed by careful human annotation.
Figure 3: Statistics analysis of AIBench. (a) Papers are curated from four representative 2025 conferences (CVPR, ICCV, NeurIPS, ICLR). (b) Models are evaluated on four hierarchical QA levels: Component, Topology, Phase, and Semantics. (c) Word cloud shows lexical frequency and topic diversity of our AIBench. (d) Top research topics in 2025 research papers, covering diffusion, LLMs, 3D reconstruction, etc.
Figure 4: The evaluation pipeline of AIBench. Models generate academic illustrations based on method descriptions, and are subsequently evaluated on two primary dimensions: logical accuracy, assessed via paper-specific QA pairs across four hierarchical levels (L1–L4, ranging from component existence to global semantics), and visual appeal, measured by a model-based aesthetic score.
Method
Aesthetic Score
Pick Score
VLM-as -Judge
UniPercept
AesR1
Human Rank
Original Figure
5.193
19.423
8.92
51.11
62.39
3
Nano Banana Pro [ 17 ]
5.181
19.904
8.63
55.04
64.82
2
GPT-Image-1.5 [ 35 ]
5.210
19.717
8.87
57.50
64.71
1
Wan 2.6 [ 43 ]
5.151
19.866
7.60
51.50
59.09
4
FLUX2-dev [ 4 ]
5.150
20.127
5.53
42.40
46.48
5
Table 2: Comparison of different aesthetic evaluation metrics and human ranking on generated academic illustrations.
Methods
Size
Component
Topology
Phase
Semantics
Aesthetics
Score
Original Image
-
82.65
57.98
79.57
79.13
51.11
70.09
Closed-Source Models
Seedream 4.5 [ 41 ]
-
67.89
52.42
48.14
74.47
55.48
59.68
Wan 2.6 [ 43 ]
-
68.60
56.11
72.56
80.43
51.50
65.84
Seedream 5.0 [ 39 ]
-
82.93
72.81
72.10
86.53
51.78
73.23
GPT-Image-1.5 [ 35 ]
-
66.23
50.87
55.95
77.55
57.50
61.62
Table 3: The evaluation results on AIBench of different state-of-the-art models. We report level-wise and overall performance across our defined five dimensions.
Figure 5: Qualitative examples of typical generation failure modes leading to incorrect answers: (a) missing components, (b) layout errors, (c) hallucinated reasoning/incorrect logic, and (d) unclear text rendering.
Methods
Component
Topology
Phase
Semantics
Aesthetics
Overall
Rewriting
Qwen-Image-2512 [ 49 ]
32.27
29.11
39.95
56.39
56.45
42.83
Rewritten Qwen-Image-2512
56.97
45.93
57.71
71.97
59.35
58.39
AutoFigure Pipeline [ 57 ]
SVG Code (Gemini-2.5-Flash [ 10 ] )
87.51
79.66
81.90
91.98
43.12
76.83
Nano Banana Pro [ 17 ]
87.80
74.81
82.67
88.54
55.04
77.77
Table 4: Exploration on the different test time scaling methods to increase the performance on AIBench. We use different models based on their distinct bottlenecks.
Figure 6: (a) shows the sensitivity analysis on different VLM solvers. (b) shows ranking correlation with human evaluation. ρ denotes Spearman’s rank correlation coefficient.
Appendix figures & tables14 assets
Supplementary material from the paper’s appendix.
Appendix
Figure A1: Examples of rendered text-to-logic directed graph derived from method-text. Purple boxes of two shapes represent nodes: rectangular purple boxes denote processing modules ( e.g. , VAE encoder, MLLM), while elliptical purple boxes represent data entities ( e.g. , InD images, latent features). Black arrows denote edges, indicating the direction of data flow between nodes. Yellow boxes indicate phases, corresponding to distinct procedural steps ( e.g. , Image Generation, Feature Encoding).
Figure A2: Different inference prompts for closed- and open-source models.
Figure A3: Prompt of QA evaluation.
Methods
Component
Topology
Phase
Semantics
Aesthetics
Overall
Rewriting
Wan2.6 w/o Prompt Enhance
35.29
42.70
64.66
79.64
42.58
52.97
Gemini Rewritten Wan2.6
47.61
49.55
66.56
79.98
58.87
60.51
Wan2.6 [ 43 ]
68.60
56.11
72.56
80.43
51.50
65.84
AutoFigure Pipeline [ 57 ]
SVG Code (Gemini-2.5-Flash [ 10 ] )
87.51
79.66
81.90
91.98
43.12
76.83
Appendix
Table A1: Extended evaluation of Test-Time Scaling strategies on AIBench. This table details the impact of the explicit reasoning phase ( e.g. , text rewriting and SVG-based structural) priors across models with varying inherent capabilities, illustrating how the effectiveness of these enhancements heavily depends on the model’s native comprehension and generation capacity.
Figure A4: Qualitative comparison of academic illustrations generated by Seedream 5.0, Qwen-Image-2512, and BAGEL on AIBench.
Figure A5: Qualitative comparison of academic illustrations generated by different models from the same method text on AIBench. The orange box shows the input method text, while the green, blue, and purple boxes present outputs from closed-source, open-source, and unified models, respectively.
Figure A6: Qualitative comparison of academic illustrations generated by different models from the same method text on AIBench.
Figure A7: Qualitative comparison of academic illustrations generated by different models from the same method text on AIBench.
Figure A8: A complete AIBench case example. The case contains the Method Text (left), the generated pipeline diagram summarizing the method (middle), and an AIBench-curated QA List (right) that includes the evaluation questions and the corresponding answers grounded in the method text and pipeline.
Figure A9: Prompt of Text-to-Logic Directed Graph during the QA construction.
Figure A10: Prompt of Level 1: Component Existence during the QA construction.
Figure A11: Prompt of Level 2: Local Topology during the QA construction.
Figure A12: Prompt of Level 3: Phase Architecture during the QA construction.
Figure A13: Prompt of Level 4: Global Semantics during the QA construction.