PaQ-DETR: Learning Pattern and Quality-Aware Dynamic Queries for Object Detection

Zhengjian Kang; Jun Zhuang; Kangtong Mo; Qi Chen; Rui Liu; Ye Zhang

cs.CVMar 6, 2026

PaQ-DETR: Learning Pattern and Quality-Aware Dynamic Queries for Object Detection

Authors: Zhengjian Kang, Jun Zhuang, Kangtong Mo, Qi Chen, Rui Liu, Ye Zhang

Organizations: New York University · Boise State University · University of Illinois at Urbana-Champaign · University of California, Irvine ·

Abstract

Detection Transformer (DETR) has redefined object detection by casting it as a set prediction task within an end-to-end framework. Despite its elegance, DETR and its variants still rely on fixed learnable queries and suffer from severe query utilization imbalance, which limits adaptability and leaves the model capacity underused. We propose PaQ-DETR (Pattern and Quality-Aware DETR), a unified framework that enhances both query adaptivity and supervision balance. It learns a compact set of shared latent patterns capturing global semantics and dynamically generates image-specific queries through content-conditioned weighting. In parallel, a quality-aware one-to-many assignment strategy adaptively selects positive samples based on localizatio-classification consistency, enriching supervision and promoting balanced query optimization. Experiments on COCO, CityScapes, and other benchmarks show consistent gains of 1.5%-4.2% mAP across DETR backbones, including ResNet and Swin-Transformer. Beyond accuracy improvement, our method provides interpretable insights into how dynamic patterns cluster semantically across object categories.

Figures & tables

Panel of Figure 1 — Figure 1: Query activation distributions of baseline DETRs vs. PaQ-DETR. Standard DETRs show highly skewed, long-tailed activations, while PaQ-DETR reduces imbalance and lowers Gini coefficients.

Figure 2: Overview of PaQ-DETR . Our framework integrates (1) a content-aware weight generator that adapts query composition to image features, (2) a pattern-based representation module that learns shared semantic bases, and (3) a quality-aware one-to-many assignment that provides balanced supervision.

Method	Backbone	Epochs	Queries	mAP	$\textrm{AP}_{50}$	$\textrm{AP}_{75}$	$\textrm{AP}_{S}$	$\textrm{AP}_{M}$	$\textrm{AP}_{L}$
H-DETR [ 19 ]	ResNet-50	12	300	48.7	66.4	52.9	31.2	51.5	63.5
MS-DETR [ 45 ]	ResNet-50	12	300	48.8	66.2	53.2	31.5	52.3	63.7
Deformable-DETR++ [ 47 ]	ResNet-50	12	300	46.9	65.6	51.1	30.1	50.4	60.3
PaQ-Deformable-DETR (ours)	ResNet-50	12	300	48.4	66.9	52.6	32.2	52.2	63.1
DAB-DETR++ [ 24 ]	ResNet-50	12	300	48.0	66.2	52.4	31.9	51.4	61.7
PaQ-DAB-DETR (ours)	ResNet-50	12	300	49.2	67.7	53.6	32.8	52.6	63.9

Table 1: The performance on COCO val 2017 based on ResNet-50 backbone under 12 and 24 epochs. ++ denotes we re-implement the methods and report the corresponding results. Rows marked in gray show the results integrating our methods.

Method	Backbone	Epochs	Queries	mAP	$\textrm{AP}_{50}$	$\textrm{AP}_{75}$	$\textrm{AP}_{S}$	$\textrm{AP}_{M}$	$\textrm{AP}_{L}$
$\mathcal{H}$ -Def-DETR [ 19 ]	Swin-L	12	300	55.9	75.2	61.0	39.1	59.9	72.2
DINO [ 41 ]	Swin-L	12	900	56.8	75.4	62.3	41.1	60.6	73.5
Salience-DETR [ 15 ]	Swin-L	12	900	56.5	75.0	61.5	40.2	61.2	72.8
Rank-DETR [ 31 ]	Swin-L	12	900	57.6	76.0	63.4	41.6	61.4	73.8
DAC-DETR [ 16 ]	Swin-L	12	900	57.3	75.7	62.7	40.1	61.5	74.4
Stable-DINO [ 25 ]	Swin-L	12	900	57.7	75.7	63.4	39.8	62.0	74.7

Table 2: The performance on COCO val 2017 based on Swin-L backbone under 12 epochs.

Method	mAP	$\textrm{AP}_{50}$	$\textrm{AP}_{75}$	$\textrm{AP}_{S}$	$\textrm{AP}_{M}$	$\textrm{AP}_{L}$
Group-DETR [ 4 ]	49.8	-	-	32.4	53.0	64.2
$\mathcal{H}$ -Def-DETR [ 19 ]	48.7	66.4	52.9	31.2	51.5	63.5
MS-DETR [ 45 ]	50.0	67.3	54.4	31.6	53.2	64.0
Co-DETR [ 48 ]	52.1	69.4	57.1	35.4	55.4	65.9
PaQ-DETR (ours)	52.4	69.8	57.4	36.1	55.6	66.2

Table 3: The performance on COCO val 2017 with one-to-many assignment DETRs using ResNet-50 backbone under 12 epochs.

Method	mAP	$\textrm{AP}_{50}$	$\textrm{AP}_{75}$	$\textrm{AP}_{S}$	$\textrm{AP}_{M}$
DAB-DETR [ 24 ]	50.2	88.5	51.0	48.2	38.2
DN-DETR [ 22 ]	49.9	88.0	51.2	47.6	37.7
H-DETR [ 19 ]	53.0	90.6	55.7	51.2	39.2
DINO [ 41 ]	53.4	91.0	55.8	50.9	39.6
PaQ-DINO (ours)	54.2	91.8	56.2	51.4	40.2

Table 4: The performance on CSD based on ResNet-50 backbone under 60 epochs.

Method	mAP	$\textrm{AP}_{50}$	$\textrm{AP}_{S}$	$\textrm{AP}_{M}$	$\textrm{AP}_{L}$
DAB-DETR [ 24 ]	33.7	60.0	15.9	26.7	29.0
DN-DETR [ 22 ]	45.6	74.1	18.6	31.9	41.0
H-DETR [ 19 ]	46.9	76.8	20.3	45.3	40.7
DINO [ 41 ]	51.0	80.0	20.0	47.4	44.8
PaQ-DINO (ours)	55.2	80.2	27.2	48.2	45.2

Table 5: The performance on MSSD based on ResNet-50 backbone under 120 epochs.

Panel of Figure 3 — Figure 3: Ablation study on key hyperparameters of PaQ-DETR: (a) number of patterns, (b) diversity loss weight $\beta$ , (c) top- $k$ in quality-aware assignment, and (d) balancing weight $\gamma$ .

D	Q	mAP	$\textrm{AP}_{50}$	$\textrm{AP}_{75}$	$\textrm{AP}_{S}$	$\textrm{AP}_{M}$	$\textrm{AP}_{L}$	Gini
		50.3	67.9	55.3	34.1	53.7	63.7	0.97
✓		51.4	69.0	56.0	35.3	55.2	66.5	0.90
	✓	51.1	68.4	55.5	34.8	55.1	65.6	0.95
✓	✓	51.9	69.1	56.3	35.1	56.0	66.6	0.89

Table 6: Ablation study on different components. ‘D’: dynamic query, ‘Q’: Quality-aware one-to-many assignment.

Figure 4: Convergence curves comparing baseline (dashed lines) with our methods (solid lines). The horizontal axis shows the number of the epochs, while the vertical axis shows mAP metrics.

Panel of Figure 5 — Figure 5: Pattern activation visualization. Left: activation heat-map; Right: statistical distribution of weights over successful detection for (a) person and (b) cat categories.

Figure 6: t-SNE visualization of $\mathbf{W}^{D}$ on 200 images from COCO val 2017. Zoom in to see details.

Method	Params (M)	FLOPs (G)	Mem (GB)	FPS
Deformable-DETR++ [ 47 ]	40.5	188	3.63	16.4
DN-DETR++ [ 22 ]	41.2	192	3.64	16.6
DINO++ [ 41 ]	42.0	196	3.67	15.8
PaQ-DINO (ours)	43.8	205	4.15	15.4

Table 7: Computational efficiency analysis.

Dataset: COCO val2017
Method	Epochs	Mask mAP	Box mAP
Baseline [ 47 ]	12	32.4	46.5
PaQ-Def-DETR (ours)	12	34.8	48.4
Baseline [ 47 ]	24	35.1	48.6
PaQ-Def-DETR (ours)	24	37.2	50.6
Dataset: CityScapes 2016

Table 8: Instance segmentation results on COCO val 2017 and CityScapes 2016 with 300 queries and ResNet-50 backbone.

Object Detection

Object detection locates and classifies objects in images or video, balancing localization accuracy, robustness, and inference speed.

See all 624 papers

Beyond Hungarian: Match-Free Supervision for End-to-End Object Detection
Mar 9, 2026Shoumeng Qiu, Xinrun Li, Yang LongObject Detection Detection Transformer
BOSCH · Durham University
Dynamic Query as Budgeting for Tiny Object Detection
Oct 24, 2026Yi Zhang, Di Xiong, Ge Gao +4Small Object Detection Detection Transformer
School of Intelligence Science and Technology, Nanjing University, China · Center for Advanced Intelligence Project, RIKEN, Japan
Order Matters: Competition-Guided Query Ordering for RNN-Based Object Detection
Oct 4, 2026Shengjian Wu, Li Sun, Yu Shangguan +1Detection Transformer Object Detection
Shanghai Key Laboratory of Multidimensional Information Processing, East China Normal University · FinVolution Group · Key Laboratory of Advanced Theory and Application in Statistics and Data Science, East China Normal University

Mar 9, 2026J/K move · Enter open · S save

Beyond Hungarian: Match-Free Supervision for End-to-End Object Detection

Shoumeng Qiu, Xinrun Li, Yang Long

BOSCH · Durham University

Object Detection Detection Transformer Efficient Object Detection

Open paper AskPDF arXiv

Recent DEtection TRansformer (DETR) based frameworks have achieved remarkable success in end-to-end object detection. However, the reliance on the Hungarian algorithm for bipartite matching between queries and ground truths introduces computational overhead and complicates the training dynamics. In this paper, we propose a novel matching-free training scheme for DETR-based detectors that eliminates the need for explicit heuristic matching. At the core of our approach is a dedicated Cross-Attention-based Query Selection (CAQS) module. Instead of discrete assignment, we utilize encoded ground-truth information to probe the decoder queries through a cross-attention mechanism. By minimizing the weighted error between the queried results and the ground truths, the model autonomously learns the implicit correspondences between object queries and specific targets. This learned relationship further provides supervision signals for the learning of queries. Experimental results demonstrate that our proposed method bypasses the traditional matching process, significantly enhancing training efficiency, reducing the matching latency by over 50%, effectively eliminating the discrete matching bottleneck through differentiable correspondence learning, and also achieving superior performance compared to existing state-of-the-art methods.

Abstract

Object Detection

Explore similar work

Beyond Hungarian: Match-Free Supervision for End-to-End Object Detection

Dynamic Query as Budgeting for Tiny Object Detection

Order Matters: Competition-Guided Query Ordering for RNN-Based Object Detection