cs.CVNov 16, 2026

PixelBench: Evaluating Language Models for Pixel-Art Creation

Authors: Mutian Hong, Jianxuan Yao, Mingkun Yue, Changkai Li, Hower Yang, Haibin Ling

Organizations: Westlake University · Mind Lab · ShanghaiTech University

Abstract

Pixel art is widely used in games and digital content creation, requiring readable forms, coherent spatial relationships, and harmonious colors under limited resolution and palette constraints. Language models show growing promise in visual creation, yet their pixel art creation capabilities lack systematic evaluation. We introduce PixelBench, formalizing text-to-image generation and instruction-guided editing for pixel art and providing a benchmark for these tasks. We further present a creation environment in which language model agents invoke drawing tools through code to generate and edit pixel art. We assess the quality of generated artwork through human evaluation and employ a multimodal large language model as a judge, validating the agreement between its automated assessments and human preferences. We compare language models and configurations across diverse subjects and canvas sizes. In our experiments, GPT-6 Astra achieves both the highest average judge score and the best average human rank, highlighting the potential of general-purpose language models for pixel art creation. We release the tasks, all artworks, the human rankings, and the judge protocol.

Figures & tables

Appendix figures & tables18 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. PixelBench: Can LLMs Understand and Generate Images Pixel by Pixel?

    Nov 2, 2026Haoquan Zhang, Feiyang Liu, Juanxi Tian +3Image GenerationT2I Generation

  2. Pixel is All You Need

    Nov 17, 2026Jie Chu, Haoting Shi, Fucheng Zhang +4Generative ModelingUnified Multimodal Models

  3. PainterBench: A Figural Divergent-Thinking Benchmark for Tool-Using Language Models

    Sep 28, 2026Shane K. A. Dalumura Hettige, Jonas OppenlaenderCreativity AssessmentAgentic Image Generation