Image Generation Evaluation

Overview

Image generation evaluation asks whether generated images look realistic, match prompts and references, preserve identities, and satisfy constraints such as spatial layout or readable text. Common approaches combine distributional metrics such as Fréchet Inception Distance and precision–recall, vision-language and feature-based scores, human judgments, and benchmarks designed for specific tasks. Recent work expands evaluation to fine-grained capabilities—including multi-subject identity and attribute binding, editing consistency, world plausibility, artistic style, and medical-image utility—and uses multimodal models to provide scalable judgments or explanations. A central challenge is making automated scores reliable across datasets and domains: a single metric can miss important failures or favor one quality dimension, while human and expert assessments are costly and difficult to standardize.

Latest papers

CardsList

113 papers18 papers in the last four weeks, up 80% on the four weeks before. 0.1% of all new papers.

  1. MoI-Bench: A Benchmark for Image Generation and Editing on Mobile Devices

    Oct 27, 2026Kailai Feng, Haoxiang Rao, Yuxiang Wei +5Image Editing EvaluationImage Generation Evaluation

  2. DiffAuction: An Auction-Driven Framework for Generative Image Advertising

    Oct 26, 2026Yijie Su, Ruiyang Si, Tonghan WangImage GenerationControllable Image Generation

  3. MDMR-Bench: A Multi-Dimensional Benchmark for Multi-Reference Image Generation

    Oct 23, 2026Bowen Zheng, Jintao Lin, Changheng Yi +5Image GenerationImage Generation Evaluation

  4. SciDiagBench: A Type-Specific Benchmark for Human-Aligned Evaluation of Scientific Diagrams

    Oct 20, 2026Hao Wang, Zixuan Li, Long Bai +4VLM EvaluationImage Generation Evaluation

  5. LogicVal:A Logic-Formalized Benchmark for Constraint Satisfaction in Text-to-Image Generation

    Oct 16, 2026Savitha Sam Abraham, Marjan Alirezaie, Hossein RahnamaImage Generation EvaluationT2I Generation Evaluation

  6. Can We Use Saliency Maps for the Evaluation of Generative Image Models?

    Oct 16, 2026Eleonora Ristori, Luca Bindini, Marco Lippi +1Image Generation EvaluationSaliency Map Evaluation

  7. TRIG: Toward Measuring Texture Realism in Image Generation

    Oct 10, 2026Zhiwei Xu, Jiali Cui, Zhiqiang Lao +2Image Generation EvaluationImage Quality Assessment

  8. UltraText Bench: A Comprehensive Bilingual Benchmark for Evaluating Visual Text Rendering in Image Generation

    Oct 7, 2026Deyuan Liu, Yihao Hu, Jingxuan Zhang +13Text Rendering in T2I GenerationT2I Generation Evaluation

  9. Visual Jev Rewards: Reference-Bound Verification for Multi-Subject Image Generation

    Oct 7, 2026Baoteng Li, Wenzhuo Wu, Kongming Liang +1Multi-Subject Image GenerationReward Modeling

  10. What Does Fréchet Distance Measure? A Directional Decomposition

    Oct 4, 2026Yunghee Lee, Jaeyeon KimImage Generation EvaluationOptimal Transport

  11. TerraVis: Towards Evaluation of World-Grounded Visual Consistency in Text-to-Image Generation via MLLM Workflows

    Oct 2, 2026Shuai Fu, Jing Gu, Jian Zhou +3Image Generation EvaluationT2I Generation Evaluation

  12. VIEScore2: Unified Image Evaluation with Spatially Grounded Explanations

    Oct 1, 2026Xianda Du, Max Ku, Weiming Ren +5VLM EvaluationImage Editing Evaluation

  13. Personalized Image Generation with Reasoning and Reflection

    Sep 30, 2026Bo Ni, Ngoc N. Tran, Qinwen Ge +10Image GenerationAgentic Image Generation

  14. VIF-Bench: Evaluating Visual Instruction Following in Multi-Reference Image Generation

    Sep 29, 2026Yuta Oshima, Masakazu Yoshimura, Masahiro Suzuki +2Image GenerationCompositional Image Generation

  15. Evaluating the Evaluators: Diagnosing Large Multimodal Models for AI-Generated Image Assessment

    Sep 29, 2026Yu Zhao, Jiarui Wang, Huiyu Duan +5LLM-as-a-JudgeImage Generation Evaluation

  16. MaLiang-Harness: A Programmable Path to Image and Video Generation

    Sep 28, 2026Haoyu Zhao, Zihao Zhang, Xudong Wang +4Image GenerationLarge Vision-Language Models

  17. M3-Score: Fidelity, Memorization and Coverage as Separate Axes for Evaluating Generative Radiology Image Models

    Sep 27, 2026Sathiyamohan Nishankar, Pubudu Sanjeewani, Asanka PereraMaximum Mean DiscrepancyMedical Image Synthesis

  18. SciGen-Verifier: A Multimodal Reasoner for Explainable Verification in Scientific Image Generation

    Sep 27, 2026Jiali Chen, Zhengteng Lin, Zuqi Wang +6Image Generation EvaluationScientific Reasoning in Language Models

  19. What Makes a Good Medical Image Tokenizer? Rethinking Reconstruction and Generation in Medical Image Tokenization

    Sep 21, 2026Niklas Bubeck, Yundi Zhang, Vasiliki Sideri-Lampretsa +4Image TokenizationMedical Image Synthesis

  20. UFO: Chain-of-Evaluation for Omni-Condition Alignment in Multi-Modal Image Generation

    Sep 11, 2026Danning Zhang, Yijing Lin, Shuhan Zhuang +4Subject-Driven Image GenerationMultimodal Generation

  21. Shedding Light: A Benchmark for Evaluating Lighting Understanding in Generative Image Models

    Sep 9, 2026Justine Giroux, Jack Oliver Hilliard, Yannick Hold-Geoffroy +2Image Generation Evaluation

  22. Are Image Generators Zero-Shot Perceivers? A Rigorous Evaluation

    Sep 7, 2026Shangzhe Di, Zhaokai Wang, Weidi XieDistribution Shift RobustnessZero-Shot Learning

  23. GenPuzzle: Benchmarking Visual Reasoning in Image Generation Models

    Sep 5, 2026Changpeng Zhao, Yiren Song, Jinpeng WangImage GenerationImage Generation Evaluation

  24. Thinking in Pictures: A Systematic Benchmark for Reasoning-driven Image Generation

    Sep 2, 2026Yutong Liu, Nan Huang, Xu Cao +1Image GenerationImage Generation Evaluation

  25. GenScale: A Benchmark for Relative Object Scale in Image Generation and Editing

    Sep 1, 2026Lingxiao Li, Max Whitton, Ledell Wu +1Image Editing EvaluationImage Generation