HGSQ: Heatmap-Guided Sparse Query Detector for Real-Time Aerial Small Object Detection

Yangchen Zeng

cs.CVSep 10, 2026

HGSQ: Heatmap-Guided Sparse Query Detector for Real-Time Aerial Small Object Detection

Authors: Yangchen Zeng

Organizations: School of Cyber Science and Engineering, Southeast University, Nanjing, 211189, China.

Abstract

Real-time aerial small object detection is an important visual signal and image processing problem, requiring a detector to preserve fine-grained localization while avoiding redundant computation on large background regions. This paper focuses on this deployment-oriented aerial/UAV setting rather than claiming a universal detector for all object detection scenarios. Existing Transformer-based detectors provide strong global modeling, but their dense query initialization and multi-layer decoder still spend substantial computation on background tokens, which is inefficient when small objects occupy only sparse image regions. To address this problem, this paper proposes HGSQ, a Heatmap-Guided Sparse Query Detector for real-time aerial small object detection. HGSQ uses a lightweight Heatmap Budget Predictor (HBP) to predict a foreground budget map in a single forward pass. The predicted heatmap is then used by three fixed components: Heatmap-Guided Sparse Query Selection (HSQS), which initializes decoder queries from high-confidence foreground positions; Heatmap-Gated Lite Snake Convolution (HGLSConv), which performs local shape refinement only on heatmap-activated small-object regions; and Adaptive Query-Decoder Budgeting (AQDB), which adjusts the query budget and decoder depth according to the estimated object density. Unlike post-hoc heatmap generation, HGSQ treats the heatmap as a real-time computation budget rather than a visualization map during deployment. Experiments on NWPU VHR-10 and VisDrone2019 show that HGSQ achieves 95.10 mAP50 on NWPU VHR-10 and 54.8 mAP50 on VisDrone2019, while reducing GFLOPs to 48.6 and running at 96.0 FPS on an RTX 4070 under our TensorRT FP16 deployment protocol.

Figures & tables

Figure 1: Motivation of HGSQ. Dense Transformer decoding allocates many queries to background regions. HGSQ predicts a heatmap budget map and selects sparse foreground queries for real-time detection.

Figure 2: Overview of the proposed HGSQ framework. The Heatmap Budget Predictor (HBP) produces a foreground budget map that guides foreground-oriented query construction, heatmap-gated local refinement, and adaptive query-decoder budgeting in a single forward pass. The heatmap-guided path allocates more computation to high-budget small-object regions while reducing redundant processing of background regions.

Figure 3: Illustration of Heatmap-Gated Lite Snake Convolution (HGLSConv). HGLSConv activates local refinement only in high-budget regions and uses bounded axis-wise snake sampling for the horizontal and vertical branches.

Input:	Image $I$ ; query bounds $K_{min},K_{max}$ ; decoder depths $D_{min},D_{mid},D_{max}$
Output:	Detections $\mathcal{D}=\{b_{k},c_{k},s_{k}\}_{k=1}^{N}$

Algorithm 1 Single-forward inference of HGSQ

Method	Backbone / Scale	Datasets					Efficiency
		NWPU VHR-10		VisDrone2019			Params(M)	GFLOPs	FPS
		mAP50	mAP50:95	mAP50	mAP50:95	AP-small	Params(M)	GFLOPs	FPS
YOLOv8s	CSPDarknet-s	91.80	64.70	43.0	26.0	14.6	11.1	28.5	123.0
YOLOv11s	C3k2/CSP-s	92.30	66.00	41.6	25.2	13.9	9.4	21.3	128.0
YOLOv11m	C3k2/CSP-m	92.68	68.89	47.3	29.3	18.4	20.0	67.7	108.0
SOD-YOLO	YOLOv8-m based	93.80	67.80	52.6	35.1	23.7	22.6	94.9	72.5

Table 1: Main comparison on NWPU VHR-10 and VisDrone2019. Methods marked with † are re-evaluated under our RTX 4070 TensorRT FP16 protocol. Other published detectors are included as reference points for the accuracy-speed trade-off when identical TensorRT reproduction is unavailable.

Variant	Backbone	Decoder	Modules				Accuracy		Efficiency
Variant	Backbone	Decoder	HBP	HSQS	HGLSConv	AQDB	mAP50	mAP50:95	GFLOPs	FPS
RT-DETR-HGNetv2 baseline	HGNetv2	6	–	–	–	–	92.90	62.10	–	80.0
Baseline	HGNetv2	original	–	–	–	–	92.60	60.20	62.4	82.0
Baseline, 3 decoder	HGNetv2	3	–	–	–	–	91.80	58.90	–	112.0
+ HBP + HSQS	HGNetv2	3	✓	✓	–	–	93.80	64.20	51.0	108.0
+ HBP + HSQS + HGLSConv	HGNetv2	3	✓	✓	✓	–	94.70	66.70	52.4	100.0

Table 2: Ablation and same-backbone fairness study under the HGNetv2 backbone and identical training setting.

Query strategy	Budget		mAP50:95	FPS
Query strategy	$K$	Layers	mAP50:95	FPS
Learned dense query	fixed	6	61.50	80.0
Encoder top- $K$ query	300	6	62.10	83.3
Heatmap top- $K$ query	100	3	64.00	128.0
Heatmap top- $K$ query	300	3	67.50	99.0
Heatmap top- $K$ query	500	3	68.00	85.0

Table 3: HSQS query-budget ablation with different query selection strategies.

Method	Query quality			Avg. queries
Method	FG ratio	GT recall	Duplicate ratio	Avg. queries
Learned dense query	18.4	82.1	31.5	300
Encoder top- $K$ query	24.7	86.8	27.2	300
Heatmap top- $K$ query	41.3	91.6	18.5	300
Adaptive heatmap query	45.8	92.4	15.9	224

Table 4: Query quality analysis for dense, encoder-selected, and heatmap-selected queries.

Module	Gate	mAP50	mAP50:95	AP-s	GFLOPs	FPS
Std. Conv	–	93.70	64.80	21.8	50.1	104.0
Dilated	–	94.00	65.50	22.6	51.0	101.0
DCNv2/3	–	94.80	67.70	24.4	61.4	72.0
DSConv	–	94.20	66.10	23.1	54.8	88.0
LSConv	–	93.91	63.69	24.0	57.0	84.0
Lite Snake	–	94.50	66.80	24.8	52.9	95.0

Table 5: Comparison of local refinement modules.

Component	Latency (ms)
Component	RT-DETR-HGNetv2-X	HMPE	HGSQ
Preprocessing	0.6	0.6	0.6
Backbone	4.8	4.2	4.2
Neck	1.5	1.2	1.2
Heatmap head	–	0.4	0.3
LSConv / HGLSConv	–	1.8	0.8

Table 6: End-to-end latency breakdown measured with batch size 1 and TensorRT FP16 deployment.

Group	HBP	$K$	$D$	AP50	Lat.	Worst	FPS
	Rec.				(ms)	(ms)
Sparse	92.8	152	1.8	57.0	9.1	13.2	109.9
Medium	91.1	242	2.3	55.0	10.5	15.4	95.2
Dense	89.3	403	3.0	47.5	14.2	22.7	70.4
Overall	90.6	224	2.2	54.8	10.4	22.7	96.0

Table 7: Density-aware AQDB behavior and latency on VisDrone2019.

Object Detection

Object detection locates and classifies objects in images or video, balancing localization accuracy, robustness, and inference speed.

See all 624 papers

Learning Where to Embed: Noise-Aware Positional Embedding for Query Retrieval in Small-Object Detection
Apr 16, 2026Yangchen Zeng, Zhenyu Yu, Dongming Jiang +5Small Object Detection Efficient Object Detection
Southeast University · Fudan University · The University of Texas at Dallas +6
UHR-DETR: Efficient End-to-End Small Object Detection for Ultra-High-Resolution Remote Sensing Imagery
Apr 23, 2026Jingfang Li, Haoran Zhu, Wen Yang +4Remote Sensing Small Object Detection Small Object Detection
School of Electronic Information, Wuhan University, Wuhan, 430072 China
FSDETR: Frequency-Spatial Feature Enhancement for Small Object Detection
Apr 16, 2026Jianchao Huang, Fengming Zhang, Haibo Zhu +1Frequency-Domain Feature Learning Small Object Detection
School of Artificial Intelligence and Computer Science, Jiangnan University, Wuxi, China

Apr 16, 2026J/K move · Enter open · S save

Learning Where to Embed: Noise-Aware Positional Embedding for Query Retrieval in Small-Object Detection

Yangchen Zeng, Zhenyu Yu, Dongming Jiang, Wenbo Zhang+4

Southeast University · Fudan University · The University of Texas at Dallas+6

Small Object Detection Efficient Object Detection Detection Transformer Efficient ViTs Positional Encoding

Open paper AskPDF arXiv

Transformer-based detectors have advanced small-object detection, but they often remain inefficient and vulnerable to background-induced query noise, which motivates deep decoders to refine low-quality queries. We present HELP (Heatmap-guided Embedding Learning Paradigm), a noise-aware positional-semantic fusion framework that studies where to embed positional information by selectively preserving positional encodings in foreground-salient regions while suppressing background clutter. Within HELP, we introduce Heatmap-guided Positional Embedding (HPE) as the core embedding mechanism and visualize it with a heatbar for interpretable diagnosis and fine-tuning. HPE is integrated into both the encoder and decoder: it guides noise-suppressed feature encoding by injecting heatmap-aware positional encoding, and it enables high-quality query retrieval by filtering background-dominant embeddings via a gradient-based mask filter before decoding. To address feature sparsity in complex small targets, we integrate Linear-Snake Convolution to enrich retrieval-relevant representations. The gradient-based heatmap supervision is used during training only, incurring no additional gradient computation at inference. As a result, our design reduces decoder layers from eight to three and achieves a 59.4% parameter reduction (66.3M vs. 163M) while maintaining consistent accuracy gains under a reduced compute budget across benchmarks. Code Repository: https://github.com/yidimopozhibai/Noise-Suppressed-Query-Retrieval

Abstract

Object Detection

Explore similar work

Learning Where to Embed: Noise-Aware Positional Embedding for Query Retrieval in Small-Object Detection

UHR-DETR: Efficient End-to-End Small Object Detection for Ultra-High-Resolution Remote Sensing Imagery

FSDETR: Frequency-Spatial Feature Enhancement for Small Object Detection