cs.CVOct 22, 2025

D2D: Detector-to-Differentiable Critic for Improved Numeracy in Text-to-Image Generation

Authors: Nobline Yoo, Olga Russakovsky, Ye Zhu

Organizations: Department of Computer Science, Princeton University · LIX, École Polytechnique, IP Paris

Abstract

Text-to-image (T2I) diffusion models have achieved strong performance in semantic alignment, yet they still struggle with generating the correct number of objects specified in prompts. Existing approaches typically incorporate auxiliary counting networks as external critics to enhance numeracy. However, since these critics must provide gradient guidance during generation, they are restricted to regression-based models that are inherently differentiable, thus excluding detector-based models with superior counting ability, whose count-via-enumeration nature is non-differentiable. To overcome this, we propose Detector-to-Differentiable (D2D), a novel framework that transforms non-differentiable detection models into differentiable critics, leveraging their superior counting ability to guide numeracy generation. Specifically, we design custom activation functions to convert detector logits into soft binary indicators, which are then used to optimize the noise prior at inference time with pre-trained T2I models. Our extensive experiments on SDXL-Turbo, SD-Turbo, and Pixart-DMD across four benchmarks of varying complexity (low-density, high-density, and multi-object scenarios) demonstrate consistent and substantial improvements in object counting accuracy (e.g., boosting up to 13.7% on D2D-Small, a 400-prompt, low-density benchmark), with minimal degradation in overall image quality and computational overhead. Project page: https://nobliney.github.io/d2d/.

Figures & tables

Appendix figures & tables37 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. NumBench: Diagnosing Counting Failures in Text-to-Image Models

    Aug 28, 2026Sandeep Wadhwa, Mayank Vatsa, Richa Singh +2T2I GenerationT2I Generation Evaluation

  2. DiffusionBench: On Holistic Evaluation of Diffusion Transformers

    Jun 23, 2026Xingjian Leng, Jaskirat Singh, Zhanhao Liang +5Diffusion TransformerImage Generation Evaluation

  3. ATHENA: Adaptive Test-Time Steering for Improving Count Fidelity in Diffusion Models

    Mar 20, 2026Mohammad Shahab Sepehri, Asal Mehradfar, Berk Tinaz +2T2I GenerationT2I Diffusion Models