DesignTaste: Benchmarking AI-Generator Visual Design Against Award-Wining Works
Organizations: Research Institute of Tsinghua University in Shenzhen · Academy of Arts & Design, Tsinghua University
Abstract
Text-to-image benchmarks commonly assess text-image alignment or overall preference. Whether models can produce good designs when a brief leaves the appearance open remains less well understood. We introduce DesignTaste, a visual design benchmark that compares generated images with award-winning works. From these works and their publicly available context, we derive 403 design briefs: 100 poster tasks from 100 Beste Plakate 2025 and 303 tasks spanning 20 categories from ADC 2026. Six leading commercial text-to-image models each generate an image from the same brief. A single model judge scores each output and the corresponding award-winning work independently on five evaluation dimensions. On each dataset, only ArtArk achieves a higher mean overall score than the award-winning works. All six models score lower in Creative Originality on both datasets. These results show that producing finished, visually appealing images does not imply that a model can develop good design solutions.
Figures & tables
| Research area | Representative work | Primary task |
| Text–image alignment | GenEval, T2I-CompBench, TIFA, DSG, VQAScore ( Ghosh et al., 2023 ; Huang et al., 2023 ; Hu et al., 2023 ; Cho et al., 2023 ; Lin et al., 2024b ) | Check objects, relationships, and compositional semantics against the prompt |
| Multidimensional quality | HEIM ( Lee et al., 2023 ) | Evaluate alignment, image quality, aesthetics, originality, and other dimensions separately |
| Human preference | ImageReward, Pick-a-Pic / PickScore, HPS v2 ( Xu et al., 2023 ; Kirstain et al., 2023 ; Wu et al., 2023 ) | Learn image preference models from human ratings or choices |
| Design understanding | DesignProbe ( Lin et al., 2024a ) | Recognize design elements and interpret style and visual metaphor |
| Editable design generation | COLE, CreatiPoster ( Jia et al., 2023 ; Hong et al., 2025 ) | Generate editable multilayer designs from descriptions of intent or supplied assets |
| Poster evaluation | PosterIQ, PosterHarness ( Feng et al., 2026 ; Yang et al., 2026 ) | Evaluate poster understanding and generation, or scientific poster content and layout requirements |
| Dimension | Evaluation focus | Scoring principle |
| Thematic Expression | Whether imagery, text, or symbols coherently convey the work’s subject or mood. | The brief provides context. Alternative solutions are valid, but clear conflicts with the work’s own subject or purpose warrant lower scores. |
| Creative Originality | Whether the central idea is distinctive and clearly developed through relationships among imagery, text, or form. | A familiar subject can become original through a distinctive treatment. Additional detail or novelty alone does not merit a higher score. |
| Visual Organization | How composition, hierarchy, contrast, and spacing establish focus and guide the viewer’s attention. | When text is present, assess legibility and hierarchy according to its role. Neither its presence nor its absence merits a higher score. |
| Aesthetic Expression | Whether color, image style, and atmosphere convey the intended expression and create visual impact. | Realism, complexity, and surface polish do not automatically increase the score. |
| Formal Completion | Whether visual elements work together and individual details are consistent with the design as a whole. | Distinguish deliberate roughness or experimental treatments from unintended errors and unfinished details. |
| Source | Poster | ADC | ||||
| Score | H/T/L | Score | H/T/L | |||
| Award-winning works | 78.275 | — | — | 80.343 | — | — |
| Seedream 5.0 Pro | 72.440 | 12 / 6 / 82 | 72.566 | 37 / 7 / 259 | ||
| Nano Banana Pro | 75.180 | 20 / 10 / 70 | 77.028 | 86 / 18 / 199 | ||
| GPT-Image-2.5 Flare | 74.710 | 13 / 10 / 77 | 78.094 | 98 / 9 / 196 | ||
| GPT-Image-2 | 75.060 | 18 / 7 / 75 | 79.477 | 118 / 17 / 168 | ||
| Case 1 Poster Create a poster announcing the exhibition “Death by Landscape” by Dorota Gawęda and Eglė Kulbokaitė at Haus für Kunst Uri. Its purpose is to introduce the exhibition and identify the two artists, without suggesting a political message or a specific interpretation of the work. Award-winning work: 77.0 ArtArk: 78.5 Other models (Seedream 5.0 Pro / Nano Banana Pro / GPT-Image-2.5 Flare / GPT-Image-2 / GPT-Image-2.5 Sunburst): 74.0 / 77.0 / 78.5 / 72.0 / 74.0 |
| Case 2 Poster Create a poster announcing the 2025 swimming parade on Vienna’s Danube Canal and encouraging participation. Communicate the event as a collective swim and make clear that a ticket is required to join the swimming parade. Award-winning work: 67.0 ArtArk: 82.5 Other models (Seedream 5.0 Pro / Nano Banana Pro / GPT-Image-2.5 Flare / GPT-Image-2 / GPT-Image-2.5 Sunburst): 69.5 / 74.0 / 74.0 / 74.0 / 74.0 |
| Case 3 Poster Create a poster promoting the Festival d’Art Brut, centred on the project Regarde bien ce que je suis, which brings together theatre, installation and music. Communicate the festival’s focus on art brut and the multidisciplinary nature of this featured work. Award-winning work: 72.0 ArtArk: 85.5 Other models (Seedream 5.0 Pro / Nano Banana Pro / GPT-Image-2.5 Flare / GPT-Image-2 / GPT-Image-2.5 Sunburst): 77.0 / 77.0 / 81.0 / 77.0 / 77.0 |
| Case 4 ADC Advertising Create one poster for a Squarespace Domains advertising campaign aimed at the general public considering buying a domain name. Position Squarespace as the preferred place to get a domain, addressing its low awareness as a domain provider and communicating that a domain is the essential first step toward building an online presence. Present a distinctive, clearly branded idea in a single still image that could serve as one execution within a poster series. Award-winning work: 65.0 ArtArk: 87.0 Other models (Seedream 5.0 Pro / Nano Banana Pro / GPT-Image-2.5 Flare / GPT-Image-2 / GPT-Image-2.5 Sunburst): 64.0 / 78.5 / 74.0 / 80.5 / 72.5 |
| Case 5 ADC Brand / Communication Design Create a poster or billboard for the GDC Award 2025, conceived as one piece within a promotional series. Reconsider “Graphic Design in China” as a living practice embedded in everyday urban life rather than a fixed style or professional category, communicating that perspective to the award’s design community and wider public. Submit one still image of the proposed piece that clearly identifies the event and expresses a distinctive interpretation of this theme. Award-winning work: 81.5 ArtArk: 90.0 Other models (Seedream 5.0 Pro / Nano Banana Pro / GPT-Image-2.5 Flare / GPT-Image-2 / GPT-Image-2.5 Sunburst): 79.0 / 77.0 / 74.0 / 80.0 / 82.0 |
| Case 6 ADC Illustration Create a single editorial illustration for a double-page spread in Inc. Magazine about businesses that continue to prioritize employee benefits and safe, thriving work environments despite a broader trend of rolling back those commitments. For the magazine’s readers, communicate the value of sustained investment in employee well-being and a supportive workplace culture. Present the illustration as one still image suitable for use across the spread. Award-winning work: 78.5 ArtArk: 87.0 Other models (Seedream 5.0 Pro / Nano Banana Pro / GPT-Image-2.5 Flare / GPT-Image-2 / GPT-Image-2.5 Sunburst): 73.5 / 72.0 / 77.0 / 77.0 / 74.0 |
| Case 7 ADC Book Design Develop an innovative book design for ArtCenter College of Design’s “Seeing the Unseeable: Data, Design, Art,” a publication examining how contemporary art and culture respond to the pervasive role of big data in modern life. The design should give this inquiry a compelling visual identity and invite readers to explore the relationship between data and creative practice. Present one still image of the proposed book or a relevant design detail that clearly communicates the concept. Award-winning work: 77.0 ArtArk: 88.5 Other models (Seedream 5.0 Pro / Nano Banana Pro / GPT-Image-2.5 Flare / GPT-Image-2 / GPT-Image-2.5 Sunburst): 74.0 / 88.5 / 77.0 / 88.5 / 85.5 |
Appendix figures & tables13 assets
Supplementary material from the paper’s appendix.
Appendix
| Dimension (0–10) | Award- winning works | Seedream 5.0 Pro | Nano Banana Pro | GPT-Image- 2.5 Flare | GPT- Image-2 | GPT-Image- 2.5 Sunburst | ArtArk |
| Thematic Expression | 8.040 | 7.790 | 7.990 | 7.920 | 8.020 | 8.090 | 8.270 |
| Creative Originality | 7.720 | 6.180 | 6.540 | 6.150 | 6.290 | 6.360 | 7.140 |
| Visual Organization | 7.010 | 7.590 | 7.760 | 8.210 | 8.020 | 8.120 | 8.290 |
| Aesthetic Expression | 8.230 | 7.850 | 8.030 | 7.970 | 8.030 | 8.080 | 8.320 |
| Formal Completion | 8.040 | 7.490 | 7.920 | 8.110 | 8.040 | 8.160 | 8.400 |
| Dimension (0–10) | Award- winning works | Seedream 5.0 Pro | Nano Banana Pro | GPT-Image- 2.5 Flare | GPT- Image-2 | GPT-Image- 2.5 Sunburst | ArtArk |
| Thematic Expression | 8.125 | 7.825 | 8.162 | 8.287 | 8.403 | 8.399 | 8.356 |
| Creative Originality | 8.023 | 6.525 | 7.238 | 7.073 | 7.333 | 7.403 | 7.581 |
| Visual Organization | 8.102 | 7.465 | 7.908 | 8.036 | 8.102 | 8.139 | 8.254 |
| Aesthetic Expression | 8.175 | 7.426 | 7.716 | 8.069 | 8.145 | 8.158 | 8.323 |
| Formal Completion | 7.680 | 7.528 | 7.799 | 8.073 | 8.152 | 8.116 | 8.168 |
| ADC award category | Seedream 5.0 Pro | Nano Banana Pro | GPT-Image- 2.5 Flare | GPT- Image-2 | GPT-Image- 2.5 Sunburst | ArtArk | |
| AI Visual Design - Dreamina AI | 4 | ||||||
| Advertising | 15 | ||||||
| Architecture / Interior / Environmental Design | 14 | ||||||
| Artificial Intelligence | 9 | ||||||
| Book Design | 29 | ||||||
| Brand / Communication Design | 59 |
| Dataset and setting | Seedream 5.0 Pro | Nano Banana Pro | GPT-Image- 2.5 Flare | GPT- Image-2 | GPT-Image- 2.5 Sunburst | ArtArk | |
| Poster / equal task weights | 100 | ||||||
| Poster / equal related-work group weights | 100 | ||||||
| Poster / excluding manually revised briefs | 86 | ||||||
| ADC / equal task weights | 303 | ||||||
| ADC / equal related-work group weights | 303 | ||||||
| ADC / excluding the fallback evaluation | 302 |
| Model | Poster: H/T/L | ADC: H/T/L |
| ArtArk | 57 / 8 / 35 | 148 / 14 / 141 |
| GPT-Image-2.5 Sunburst | 23 / 11 / 66 | 122 / 18 / 163 |
| GPT-Image-2 | 18 / 7 / 75 | 118 / 17 / 168 |
| GPT-Image-2.5 Flare | 13 / 10 / 77 | 98 / 9 / 196 |
| Nano Banana Pro | 20 / 10 / 70 | 86 / 18 / 199 |
| Seedream 5.0 Pro | 12 / 6 / 82 | 37 / 7 / 259 |