cs.CVJan 23, 2026

ColorConceptBench: A Benchmark for Probabilistic Color-Concept Understanding in Text-to-Image Models

Authors: Chenxi Ruan, Yihan Hou, Yu Xiao, Guosheng Hu, Wei Zeng

Organizations: The Hong Kong University of Science and Technology (Guangzhou), Guangzhou, China · China Academy of Art, Hangzhou, China · The Hong Kong University of Science and Technology, Hong Kong SAR, China

Abstract

Text-to-image (T2I) models have advanced considerably in generating high-quality images from textual descriptions. However, their ability to associate colors with concepts remains largely constrained to explicit color names or codes, while their capacity to handle implicit concepts, such as emotions and visual states, remains underexplored. To address this gap, we introduce ColorConceptBench, an expert-annotated benchmark that systematically evaluates color-concept associations through probabilistic color distributions. ColorConceptBench moves beyond explicit color specifications by examining how models interpret 1,281 implicit color concepts, grounded in 6,584 human annotations. Our evaluation of nine leading T2I models reveals that performance varies substantially across semantic categories, and models exhibit a significant lack of sensitivity to abstract semantics. These limitations persist even when applying classifier-free guidance scaling at inference time, suggesting that achieving human-like color understanding demands a shift in how models learn and represent implicit semantic meaning.

Figures & tables

Appendix figures & tables29 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. ACVA-Bench: Beyond Prompt Following in Abstract Concept Visualization Ability for Text-to-Image Models

    Oct 23, 2026Mengyan Wang, Jiyuan An, Yining Liu +2T2I Generation EvaluationT2I Diffusion Models

  2. NegT2IBench: When Negation Changes the Picture. A Polarity Benchmark for Text-to-Image Models

    Oct 2, 2026Omar Elfatairy, Maria A. Bravo, Jessica Bader +1T2I GenerationT2I Generation Evaluation