Small object detection (SOD) remains challenging due to extremely limited pixels and ambiguous object boundaries. These characteristics lead to challenging annotation, limited availability of large-scale high-quality datasets, and inherently weak semantic representations for small objects. In this work, we first address the data limitation by introducing TinySet-9M, the first large-scale, multi-domain dataset for small object detection. Beyond filling the gap in large-scale datasets, we establish a benchmark to evaluate the effectiveness of existing label-efficient detection methods for small objects. Our evaluation reveals that weak visual cues further exacerbate the performance degradation of label-efficient methods in small object detection, highlighting a critical challenge in label-efficient SOD. Secondly, to tackle the limitation of insufficient semantic representation, we move beyond training-time feature enhancement and propose a new paradigm termed Point-Prompt Small Object Detection (P2SOD). This paradigm introduces sparse point prompts at inference time as an efficient information bridge for category-level localization, enabling semantic augmentation. Building upon the P2SOD paradigm and the large-scale TinySet-9M dataset, we further develop DEAL (DEtect Any smalL object), a scalable and transferable point-prompted detection framework that learns robust, prompt-conditioned representations from large-scale data. With only a single click at inference time, DEAL achieves a 31.4% relative improvement over fully supervised baselines under strict localization metrics (e.g., AP75) on TinySet-9M, while generalizing effectively to unseen categories and unseen datasets. Our project is available at https://zhuhaoraneis.github.io/TinySet-9M/.
Figures & tables
Fig. 2: Statistical analysis of the TinySet-9M dataset. (a) shows the composition of TinySet-9M, where different source datasets form distinct sub-domains; (b) presents the distribution of object scales, in which the green line indicates the average object scale of the dataset (20.4) and the red line denotes the scale threshold for small objects; (c) illustrates the distribution of object density across images; (d) shows the comparison of TinySet-9M and Object365 in terms of the proportion of small objects, object density, average object scale, number of instances, and number of images.
Fig. 3: Visualization of the TinySet-9M dataset. The dataset contains a large number of densely distributed objects with extremely small spatial scales, which results in blurred object boundaries and weak semantic representations.
ID
Method
Setting
Backbone
Schedule
AP0.25
AP0.5
AP0.75
APvt
APt
APs
APm
fully-supervised:
#1
RetinaNet [ 56 ]
100% labeled
ResNet-50
1 ×
56.7
37.8
10.6
6.5
25.0
49.7
83.3
#2
Faster R-CNN [ 57 ]
ResNet-50
1 ×
51.9
43.2
17.8
5.0
30.5
59.0
84.5
#3
FCOS [ 58 ]
ResNet-50
1 ×
66.3
49.5
17.1
13.8
41.1
62.3
83.7
#4
RT-DETR [ 59 ]
ResNet-50
1 ×
57.2
43.1
12.1
11.3
35.3
54.7
74.1
#5
HS-FPN (Faster R-CNN) [ 12 ]
ResNet-50
1 ×
51.8
43.0
17.5
5.9
31.2
59.1
83.0
TABLE I: Main results of fully-supervised, noise-supervised, semi-supervised, sparse-annotated, point-supervised, sparse-shot, and zero-shot methods on TinySet-9M (class-agnostic). For the training schedule, 1 × denotes 3 epochs. All experiments are run on a computer with an NVIDIA RTX 3090 (24 GB) GPU. We use FP32 with 1024 × 1024 inputs.
Fig. 4: Detection results of different detection paradigms on TinySet-9M dataset. Orange boxes denote the detection results with confidence higher than 0.2.
Fig. 5: Overview of the proposed method. (a) Illustration of the overall architecture of DEAL, which is built upon the RT-DETR framework and extended to support point-prompted small object detection. (b) Illustration of the training pipeline of the proposed Prediction-Guided Cyclic Point Prompting (PG-CPP) strategy.
Fig. 6: Inference settings for the P 2 SOD task. Setting (1) uses one point prompt per category in each image; Setting (2) assigns one point prompt to each object instance; Setting (3) randomly selects a single category and uses one point from that category as the prompt; Setting (4) randomly selects a category and uses point prompts from all its instances.
Method
Backbone
Schedule
AP0.25
AP0.5
AP0.75
APvt
APt
APs
APm
fully-supervised:
RetinaNet [ 56 ]
ResNet-50
1 ×
56.7
37.8
10.6
6.5
25.0
49.7
83.3
Faster R-CNN [ 57 ]
ResNet-50
1 ×
51.9
43.2
17.8
5.0
30.5
59.0
84.5
FCOS [ 58 ]
ResNet-50
1 ×
66.3
49.5
17.1
13.8
41.1
62.3
83.7
RT-DETR [ 59 ]
ResNet-50
1 ×
57.2
43.1
12.1
11.3
35.3
54.7
74.1
HS-FPN (Faster R-CNN) [ 12 ]
ResNet-50
1 ×
51.8
43.0
17.5
5.9
31.2
59.1
83.0
TABLE II: Main results of fully-supervised, point-supervised, zero-shot (box-prompt), zero-shot (text-prompt) methods, and our proposed DEAL on TinySet-9M (class-agnostic). For the training schedule, 1 × denotes 3 epochs. All experiments are run on a computer with an NVIDIA RTX 4090 (24 GB) GPU. We use FP32 with 1024 × 1024 inputs. Our proposed DEAL is conducted under setting (1).
Method
Backbone
Pre-Training Data
w/o fine-tuning
w/ fine-tuning
DIOR
DOTA-v2.0
DIOR
DOTA-v2.0
Generic Object Detection
RetinaNet [ 56 ]
ResNet-50
-
-
-
66.1
51.6
Faster R-CNN [ 57 ]
ResNet-50
-
-
-
52.1
54.8
FCOS [ 64 ]
ResNet-50
-
-
-
71.9
54.3
GASSL [ 65 ]
ResNet-50
-
-
-
67.4
-
TABLE III: The cross-dataset evaluation results on DIOR, DOTA-v2.0 test set. The results of DOTA-v2.0 are all based on horizontal detection boxes. GeoImageNet, Sentinel-2, TOV-NI, TOV-R, FMoW, SatlasPretrain, MillionAID, RingMoPretrain, and multi-modal RSI are remote sensing datasets. Our proposed DEAL is conducted under setting (1). ( ⋅ ) denotes APsmall .
TABLE IV: Ablations. We train on TinySet-9M train set , test on test set . HFE, PE, and DGQA represent Hybrid Feature Enhancement, Point Embedding, and Point-Guided Density Activation, respectively. Gray row means the default setting.
Fig. 7: Different point prompt position during inference under setting (3).
Methods
Training
Training
Inference
Inference
time (ms)
memory (MB)
time (ms)
memory (MB)
RT-DETR
98.8
9430.0
28.1
490.2
Rex-Omni
–
–
4026.9
20264.0
SAM3
–
–
169.3
4354.8
DEAL
320.2
11036.0
40.8
581.1
TABLE V: Training and inference memory consumption and runtime overhead with batch_size=1.
Dataset
Pre-training Data
AP0.5
APvt
APt
APs
APm
COCO
ImageNet
50.8
7.6
25.5
37.1
60.0
TinySet-9M
51.0
7.9
25.3
37.1
60.5
TinyPerson
ImageNet
8.9
0.5
2.0
9.7
28.2
TinySet-9M
9.1
0.5
2.0
9.8
29.8
xView
ImageNet
30.6
2.2
9.7
23.6
32.1
TinySet-9M
31.0
4.0
11.8
25.0
32.7
TABLE VI: Performance of the trained feature extractors when transferred to other object detection tasks. In these experiments, FCOS is adopted as the detection framework, and all models are trained for 3 epochs.
Fig. 8: Detection results of zero-shot methods SAM3 and our proposed DEAL on DOTA-v2.0 dataset. Green boxes, red boxes, red points, and orange boxes denote the gt , box visual prompts, point visual prompts, and detection results, respectively.
Fig. 9: Detection results on images collected from the Internet. All images are resized to a resolution of 1024 × 1024. None of these images belong to the training datasets, and they exhibit unseen categories or data distributions, such as cartoon images and surveillance camera footage. Red points and orange boxes denote the point visual prompts and the corresponding detection results, respectively.
Fig. 10: Unseen sceneries feature association visualization. In this figure, green bounding boxes denote ground-truth annotations, and red stars indicate the point prompts. Each feature map visualizes the cosine similarity between the feature at every spatial location and the feature at the prompted point location.
Small object detection remains a significant challenge due to feature degradation from downsampling, mutual occlusion in dense clusters, and complex background interference. To address these issues, this paper proposes FSDETR, a frequency-spatial feature enhancement framework built upon the RT-DETR baseline. By establishing a collaborative modeling mechanism, the method effectively leverages complementary structural information. Specifically, a Spatial Hierarchical Attention Block (SHAB) captures both local details and global dependencies to strengthen semantic representation. Furthermore, to mitigate occlusion in dense scenes, the Deformable Attention-based Intra-scale Feature Interaction (DA-AIFI) focuses on informative regions via dynamic sampling. Finally, the Frequency-Spatial Feature Pyramid Network (FSFPN) integrates frequency filtering with spatial edge extraction via the Cross-domain Frequency-Spatial Block (CFSB) to preserve fine-grained details. Experimental results show that with only 14.7M parameters, FSDETR achieves 13.9% APS on VisDrone 2019 and 48.95% AP50 tiny on TinyPerson, showing strong performance on small-object benchmarks. The code and models are available at https://github.com/YT3DVision/FSDETR.
Jianchao Huang, Fengming Zhang, Haibo Zhu +1
School of Artificial Intelligence and Computer Science, Jiangnan University, Wuxi, China
Despite the remarkable progress over the past decades, accurately identifying small objects remains challenging because of their insufficient visual cues. Previous works typically attempt to construct discriminative representation of the small objects. However, the wide range frequency domain noises and label ambiguities have been greatly overlooked, which significantly hinders the accurate localization. To address these issues, we propose a novel small object detection (SOD) detector termed DyFrDet, which is able to precisely localize the small object by dynamically suppressing the background distractions in frequency domain. Specifically, we propose a Dynamic Frequency-aware Feature Pyramid Network (DyFrFPN) to adaptively suppress low-frequency redundancy and excessive high-frequency noises. The DyFrFPN transforms the hierarchical features into frequency domain representation, and introduces a Dynamic Band Predictor (DBP) to preserve the discriminative components for small object identification. Afterwards, we present a novel Label Disambiguation Module (LDM), which leverages probabilistic distributions to explicitly model and alleviate the inherent ambiguity of target labels, yielding efficient improvement in localization precision of the small objects with low-resolution. Extensive experiments demonstrate that DyFrDet achieves state-of-the-art performance across multiple benchmarks, indicating its effectiveness and robustness in various challenging scenarios. Our code is available at https://github.com/ManOfStory/DyFrDet.
Zihan Yang, Yang Guo, Hongxing Zhang +2
Beijing University of Posts and Telecommunications · Shenzhen Campus of Sun Yat-sen University
Small Object Detection (SOD) is a fundamental yet challenging problem in computer vision due to its limited spatial resolution and weak visual cues. Although recent approaches have achieved remarkable advances, the background distractors in different frequency spectra still degrade the performance. In this paper, we propose a novel small object detection framework termed SFDNet, which is capable of detecting small objects via efficient spectrum-aware feature disentanglement. Specifically, we propose an Adaptive Spectrum Disentanglement (ASD) module that decomposes backbone features into multiple complementary spectral components, aiming to construct discriminative object-relevant representations by discarding the background distractors for each component. Afterwards, to strengthen the semantic consistency of the similar objects in the same class, we propose a Class-Wise Prototype Distillation (CPD) procedure, which establishes class prototypes for the object instances and enforces the compact representation by efficient prototype distillation. Extensive experiments on multiple challenging benchmarks show that SFDNet outperforms existing state-of-the-art methods by a large margin. Our code is available at https://github.com/ManOfStory/SFDNet.
Yang Guo, Zihan Yang, Feifei Kou +3
Shenzhen Campus of Sun Yat-sen University · Beijing University of Posts and Telecommunications, Beijing, China · Hangzhou International Innovation Institute, Beihang University, Hangzhou, China +2