cs.CVNov 5, 2026

Improving Long-Tail Detection via Layout-Instruction Conditioned Diffusion Synthesis

Authors: Xinyu Zhang, Xiaodi Wang, Yanpeng Sun, Lingfeng Yang, Jingdong Wang, Lingqiao Liu

Organizations: University of Auckland, Auckland, New Zealand · Dcar, Beijing, China · Singapore University of Technology and Design, Singapore · WIZYELAB PTE. LTD., Singapore · 

Abstract

Synthetic data from diffusion models can improve object detection, but long-tail performance is still limited by scarce rare classes and inflexible conditioning (e.g., text-only or layout-only inputs). In this paper, we propose Tail-SD, a layout-instruction conditioned stable diffusion framework that supports text, layouts, or their hybrid, enabling controllable synthesis from boxes, masks, and/or prompts to better cover the long-tail objects. To train a single model across heterogeneous conditions, we introduce a multi-task cross-attention module and multi-task embedding tokens, improving generation quality and diversity. Extensive experiments on COCO and LVIS demonstrate that diffusion synthesis with hybrid conditioning consistently benefits detection, e.g., on LVIS val, it achieves +4.89% / +5.68% rare Box/Mask AP, highlighting strong potential for addressing long-tail challenges.

Figures & tables

Explore similar work

CardsList
  1. Class-specific diffusion models improve military object detection in a low-data domain

    Apr 20, 2026Ella P. Fokkinga, Jan Erik van Woerden, Thijs A. Eker +4Synthetic Data AugmentationDiffusion Model Guidance

  2. Contrastive Conditional-Unconditional Alignment for Long-tailed Diffusion Model

    Jul 11, 2025Fang Chen, Alex Villa, Gongbo Liang +3Diffusion Model AlignmentConditional Image Generation