Real-time aerial small object detection is an important visual signal and image processing problem, requiring a detector to preserve fine-grained localization while avoiding redundant computation on large background regions. This paper focuses on this deployment-oriented aerial/UAV setting rather than claiming a universal detector for all object detection scenarios. Existing Transformer-based detectors provide strong global modeling, but their dense query initialization and multi-layer decoder still spend substantial computation on background tokens, which is inefficient when small objects occupy only sparse image regions. To address this problem, this paper proposes HGSQ, a Heatmap-Guided Sparse Query Detector for real-time aerial small object detection. HGSQ uses a lightweight Heatmap Budget Predictor (HBP) to predict a foreground budget map in a single forward pass. The predicted heatmap is then used by three fixed components: Heatmap-Guided Sparse Query Selection (HSQS), which initializes decoder queries from high-confidence foreground positions; Heatmap-Gated Lite Snake Convolution (HGLSConv), which performs local shape refinement only on heatmap-activated small-object regions; and Adaptive Query-Decoder Budgeting (AQDB), which adjusts the query budget and decoder depth according to the estimated object density. Unlike post-hoc heatmap generation, HGSQ treats the heatmap as a real-time computation budget rather than a visualization map during deployment. Experiments on NWPU VHR-10 and VisDrone2019 show that HGSQ achieves 95.10 mAP50 on NWPU VHR-10 and 54.8 mAP50 on VisDrone2019, while reducing GFLOPs to 48.6 and running at 96.0 FPS on an RTX 4070 under our TensorRT FP16 deployment protocol.
Figures & tables
Figure 1: Motivation of HGSQ. Dense Transformer decoding allocates many queries to background regions. HGSQ predicts a heatmap budget map and selects sparse foreground queries for real-time detection.
Figure 2: Overview of the proposed HGSQ framework. The Heatmap Budget Predictor (HBP) produces a foreground budget map that guides foreground-oriented query construction, heatmap-gated local refinement, and adaptive query-decoder budgeting in a single forward pass. The heatmap-guided path allocates more computation to high-budget small-object regions while reducing redundant processing of background regions.
Figure 3: Illustration of Heatmap-Gated Lite Snake Convolution (HGLSConv). HGLSConv activates local refinement only in high-budget regions and uses bounded axis-wise snake sampling for the horizontal and vertical branches.
Input:
Image I ; query bounds Kmin,Kmax ; decoder depths Dmin,Dmid,Dmax
Output:
Detections D={bk,ck,sk}k=1N
Algorithm 1 Single-forward inference of HGSQ
Method
Backbone / Scale
Datasets
Efficiency
NWPU VHR-10
VisDrone2019
Params(M)
GFLOPs
FPS
mAP50
mAP50:95
mAP50
mAP50:95
AP-small
YOLOv8s
CSPDarknet-s
91.80
64.70
43.0
26.0
14.6
11.1
28.5
123.0
YOLOv11s
C3k2/CSP-s
92.30
66.00
41.6
25.2
13.9
9.4
21.3
128.0
YOLOv11m
C3k2/CSP-m
92.68
68.89
47.3
29.3
18.4
20.0
67.7
108.0
SOD-YOLO
YOLOv8-m based
93.80
67.80
52.6
35.1
23.7
22.6
94.9
72.5
Table 1: Main comparison on NWPU VHR-10 and VisDrone2019. Methods marked with † are re-evaluated under our RTX 4070 TensorRT FP16 protocol. Other published detectors are included as reference points for the accuracy-speed trade-off when identical TensorRT reproduction is unavailable.
Variant
Backbone
Decoder
Modules
Accuracy
Efficiency
HBP
HSQS
HGLSConv
AQDB
mAP50
mAP50:95
GFLOPs
FPS
RT-DETR-HGNetv2 baseline
HGNetv2
6
–
–
–
–
92.90
62.10
–
80.0
Baseline
HGNetv2
original
–
–
–
–
92.60
60.20
62.4
82.0
Baseline, 3 decoder
HGNetv2
3
–
–
–
–
91.80
58.90
–
112.0
+ HBP + HSQS
HGNetv2
3
✓
✓
–
–
93.80
64.20
51.0
108.0
+ HBP + HSQS + HGLSConv
HGNetv2
3
✓
✓
✓
–
94.70
66.70
52.4
100.0
Table 2: Ablation and same-backbone fairness study under the HGNetv2 backbone and identical training setting.
Query strategy
Budget
mAP50:95
FPS
K
Layers
Learned dense query
fixed
6
61.50
80.0
Encoder top- K query
300
6
62.10
83.3
Heatmap top- K query
100
3
64.00
128.0
Heatmap top- K query
300
3
67.50
99.0
Heatmap top- K query
500
3
68.00
85.0
Table 3: HSQS query-budget ablation with different query selection strategies.
Method
Query quality
Avg. queries
FG ratio
GT recall
Duplicate ratio
Learned dense query
18.4
82.1
31.5
300
Encoder top- K query
24.7
86.8
27.2
300
Heatmap top- K query
41.3
91.6
18.5
300
Adaptive heatmap query
45.8
92.4
15.9
224
Table 4: Query quality analysis for dense, encoder-selected, and heatmap-selected queries.
Module
Gate
mAP50
mAP50:95
AP-s
GFLOPs
FPS
Std. Conv
–
93.70
64.80
21.8
50.1
104.0
Dilated
–
94.00
65.50
22.6
51.0
101.0
DCNv2/3
–
94.80
67.70
24.4
61.4
72.0
DSConv
–
94.20
66.10
23.1
54.8
88.0
LSConv
–
93.91
63.69
24.0
57.0
84.0
Lite Snake
–
94.50
66.80
24.8
52.9
95.0
Table 5: Comparison of local refinement modules.
Component
Latency (ms)
RT-DETR-HGNetv2-X
HMPE
HGSQ
Preprocessing
0.6
0.6
0.6
Backbone
4.8
4.2
4.2
Neck
1.5
1.2
1.2
Heatmap head
–
0.4
0.3
LSConv / HGLSConv
–
1.8
0.8
Table 6: End-to-end latency breakdown measured with batch size 1 and TensorRT FP16 deployment.
Group
HBP
K
D
AP50
Lat.
Worst
FPS
Rec.
(ms)
(ms)
Sparse
92.8
152
1.8
57.0
9.1
13.2
109.9
Medium
91.1
242
2.3
55.0
10.5
15.4
95.2
Dense
89.3
403
3.0
47.5
14.2
22.7
70.4
Overall
90.6
224
2.2
54.8
10.4
22.7
96.0
Table 7: Density-aware AQDB behavior and latency on VisDrone2019.
Transformer-based detectors have advanced small-object detection, but they often remain inefficient and vulnerable to background-induced query noise, which motivates deep decoders to refine low-quality queries. We present HELP (Heatmap-guided Embedding Learning Paradigm), a noise-aware positional-semantic fusion framework that studies where to embed positional information by selectively preserving positional encodings in foreground-salient regions while suppressing background clutter. Within HELP, we introduce Heatmap-guided Positional Embedding (HPE) as the core embedding mechanism and visualize it with a heatbar for interpretable diagnosis and fine-tuning. HPE is integrated into both the encoder and decoder: it guides noise-suppressed feature encoding by injecting heatmap-aware positional encoding, and it enables high-quality query retrieval by filtering background-dominant embeddings via a gradient-based mask filter before decoding. To address feature sparsity in complex small targets, we integrate Linear-Snake Convolution to enrich retrieval-relevant representations. The gradient-based heatmap supervision is used during training only, incurring no additional gradient computation at inference. As a result, our design reduces decoder layers from eight to three and achieves a 59.4% parameter reduction (66.3M vs. 163M) while maintaining consistent accuracy gains under a reduced compute budget across benchmarks. Code Repository: https://github.com/yidimopozhibai/Noise-Suppressed-Query-Retrieval
Yangchen Zeng, Zhenyu Yu, Dongming Jiang +5
Southeast University · Fudan University · The University of Texas at Dallas +6
Ultra-High-Resolution (UHR) imagery has become essential for modern remote sensing, offering unprecedented spatial coverage. However, detecting small objects in such vast scenes presents a critical dilemma: retaining the original resolution for small objects causes prohibitive memory bottlenecks. Conversely, conventional compromises like image downsampling or patch cropping either erase small objects or destroy context. To break this dilemma, we propose UHR-DETR, an efficient end-to-end transformer-based detector designed for UHR imagery. First, we introduce a Coverage-Maximizing Sparse Encoder that dynamically allocates finite computational resources to informative high-resolution regions, ensuring maximum object coverage with minimal spatial redundancy. Second, we design a Global-Local Decoupled Decoder. By integrating macroscopic scene awareness with microscopic object details, this module resolves semantic ambiguities and prevents scene fragmentation. Extensive experiments on the UHR imagery datasets (e.g., STAR and SODA-A) demonstrate the superiority of UHR-DETR under strict hardware constraints (e.g., a single 24GB RTX 3090). It achieves a 2.8% mAP improvement while delivering a 10× inference speedup compared to standard sliding-window baselines on the STAR dataset. Our codes and models will be available at https://github.com/Li-JingFang/UHR-DETR.
Jingfang Li, Haoran Zhu, Wen Yang +4
School of Electronic Information, Wuhan University, Wuhan, 430072 China
Small object detection remains a significant challenge due to feature degradation from downsampling, mutual occlusion in dense clusters, and complex background interference. To address these issues, this paper proposes FSDETR, a frequency-spatial feature enhancement framework built upon the RT-DETR baseline. By establishing a collaborative modeling mechanism, the method effectively leverages complementary structural information. Specifically, a Spatial Hierarchical Attention Block (SHAB) captures both local details and global dependencies to strengthen semantic representation. Furthermore, to mitigate occlusion in dense scenes, the Deformable Attention-based Intra-scale Feature Interaction (DA-AIFI) focuses on informative regions via dynamic sampling. Finally, the Frequency-Spatial Feature Pyramid Network (FSFPN) integrates frequency filtering with spatial edge extraction via the Cross-domain Frequency-Spatial Block (CFSB) to preserve fine-grained details. Experimental results show that with only 14.7M parameters, FSDETR achieves 13.9% APS on VisDrone 2019 and 48.95% AP50 tiny on TinyPerson, showing strong performance on small-object benchmarks. The code and models are available at https://github.com/YT3DVision/FSDETR.
Jianchao Huang, Fengming Zhang, Haibo Zhu +1
School of Artificial Intelligence and Computer Science, Jiangnan University, Wuxi, China