Small object detection (SOD) remains challenging due to extremely limited pixels and ambiguous object boundaries. These characteristics lead to challenging annotation, limited availability of large-scale high-quality datasets, and inherently weak semantic representations for small objects. In this work, we first address the data limitation by introducing TinySet-9M, the first large-scale, multi-domain dataset for small object detection. Beyond filling the gap in large-scale datasets, we establish a benchmark to evaluate the effectiveness of existing label-efficient detection methods for small objects. Our evaluation reveals that weak visual cues further exacerbate the performance degradation of label-efficient methods in small object detection, highlighting a critical challenge in label-efficient SOD. Secondly, to tackle the limitation of insufficient semantic representation, we move beyond training-time feature enhancement and propose a new paradigm termed Point-Prompt Small Object Detection (P2SOD). This paradigm introduces sparse point prompts at inference time as an efficient information bridge for category-level localization, enabling semantic augmentation. Building upon the P2SOD paradigm and the large-scale TinySet-9M dataset, we further develop DEAL (DEtect Any smalL object), a scalable and transferable point-prompted detection framework that learns robust, prompt-conditioned representations from large-scale data. With only a single click at inference time, DEAL achieves a 31.4% relative improvement over fully supervised baselines under strict localization metrics (e.g., AP75) on TinySet-9M, while generalizing effectively to unseen categories and unseen datasets. Our project is available at https://zhuhaoraneis.github.io/TinySet-9M/.
Figures & tables
Fig. 2: Statistical analysis of the TinySet-9M dataset. (a) shows the composition of TinySet-9M, where different source datasets form distinct sub-domains; (b) presents the distribution of object scales, in which the green line indicates the average object scale of the dataset (20.4) and the red line denotes the scale threshold for small objects; (c) illustrates the distribution of object density across images; (d) shows the comparison of TinySet-9M and Object365 in terms of the proportion of small objects, object density, average object scale, number of instances, and number of images.
Fig. 3: Visualization of the TinySet-9M dataset. The dataset contains a large number of densely distributed objects with extremely small spatial scales, which results in blurred object boundaries and weak semantic representations.
ID
Method
Setting
Backbone
Schedule
AP0.25
AP0.5
AP0.75
APvt
APt
APs
APm
fully-supervised:
#1
RetinaNet [ 56 ]
100% labeled
ResNet-50
1 ×
56.7
37.8
10.6
6.5
25.0
49.7
83.3
#2
Faster R-CNN [ 57 ]
ResNet-50
1 ×
51.9
43.2
17.8
5.0
30.5
59.0
84.5
#3
FCOS [ 58 ]
ResNet-50
1 ×
66.3
49.5
17.1
13.8
41.1
62.3
83.7
#4
RT-DETR [ 59 ]
ResNet-50
1 ×
57.2
43.1
12.1
11.3
35.3
54.7
74.1
#5
HS-FPN (Faster R-CNN) [ 12 ]
ResNet-50
1 ×
51.8
43.0
17.5
5.9
31.2
59.1
83.0
TABLE I: Main results of fully-supervised, noise-supervised, semi-supervised, sparse-annotated, point-supervised, sparse-shot, and zero-shot methods on TinySet-9M (class-agnostic). For the training schedule, 1 × denotes 3 epochs. All experiments are run on a computer with an NVIDIA RTX 3090 (24 GB) GPU. We use FP32 with 1024 × 1024 inputs.
Fig. 4: Detection results of different detection paradigms on TinySet-9M dataset. Orange boxes denote the detection results with confidence higher than 0.2.
Fig. 5: Overview of the proposed method. (a) Illustration of the overall architecture of DEAL, which is built upon the RT-DETR framework and extended to support point-prompted small object detection. (b) Illustration of the training pipeline of the proposed Prediction-Guided Cyclic Point Prompting (PG-CPP) strategy.
Fig. 6: Inference settings for the P 2 SOD task. Setting (1) uses one point prompt per category in each image; Setting (2) assigns one point prompt to each object instance; Setting (3) randomly selects a single category and uses one point from that category as the prompt; Setting (4) randomly selects a category and uses point prompts from all its instances.
Method
Backbone
Schedule
AP0.25
AP0.5
AP0.75
APvt
APt
APs
APm
fully-supervised:
RetinaNet [ 56 ]
ResNet-50
1 ×
56.7
37.8
10.6
6.5
25.0
49.7
83.3
Faster R-CNN [ 57 ]
ResNet-50
1 ×
51.9
43.2
17.8
5.0
30.5
59.0
84.5
FCOS [ 58 ]
ResNet-50
1 ×
66.3
49.5
17.1
13.8
41.1
62.3
83.7
RT-DETR [ 59 ]
ResNet-50
1 ×
57.2
43.1
12.1
11.3
35.3
54.7
74.1
HS-FPN (Faster R-CNN) [ 12 ]
ResNet-50
1 ×
51.8
43.0
17.5
5.9
31.2
59.1
83.0
TABLE II: Main results of fully-supervised, point-supervised, zero-shot (box-prompt), zero-shot (text-prompt) methods, and our proposed DEAL on TinySet-9M (class-agnostic). For the training schedule, 1 × denotes 3 epochs. All experiments are run on a computer with an NVIDIA RTX 4090 (24 GB) GPU. We use FP32 with 1024 × 1024 inputs. Our proposed DEAL is conducted under setting (1).
Method
Backbone
Pre-Training Data
w/o fine-tuning
w/ fine-tuning
DIOR
DOTA-v2.0
DIOR
DOTA-v2.0
Generic Object Detection
RetinaNet [ 56 ]
ResNet-50
-
-
-
66.1
51.6
Faster R-CNN [ 57 ]
ResNet-50
-
-
-
52.1
54.8
FCOS [ 64 ]
ResNet-50
-
-
-
71.9
54.3
GASSL [ 65 ]
ResNet-50
-
-
-
67.4
-
TABLE III: The cross-dataset evaluation results on DIOR, DOTA-v2.0 test set. The results of DOTA-v2.0 are all based on horizontal detection boxes. GeoImageNet, Sentinel-2, TOV-NI, TOV-R, FMoW, SatlasPretrain, MillionAID, RingMoPretrain, and multi-modal RSI are remote sensing datasets. Our proposed DEAL is conducted under setting (1). ( ⋅ ) denotes APsmall .
TABLE IV: Ablations. We train on TinySet-9M train set , test on test set . HFE, PE, and DGQA represent Hybrid Feature Enhancement, Point Embedding, and Point-Guided Density Activation, respectively. Gray row means the default setting.
Fig. 7: Different point prompt position during inference under setting (3).
Methods
Training
Training
Inference
Inference
time (ms)
memory (MB)
time (ms)
memory (MB)
RT-DETR
98.8
9430.0
28.1
490.2
Rex-Omni
–
–
4026.9
20264.0
SAM3
–
–
169.3
4354.8
DEAL
320.2
11036.0
40.8
581.1
TABLE V: Training and inference memory consumption and runtime overhead with batch_size=1.
Dataset
Pre-training Data
AP0.5
APvt
APt
APs
APm
COCO
ImageNet
50.8
7.6
25.5
37.1
60.0
TinySet-9M
51.0
7.9
25.3
37.1
60.5
TinyPerson
ImageNet
8.9
0.5
2.0
9.7
28.2
TinySet-9M
9.1
0.5
2.0
9.8
29.8
xView
ImageNet
30.6
2.2
9.7
23.6
32.1
TinySet-9M
31.0
4.0
11.8
25.0
32.7
TABLE VI: Performance of the trained feature extractors when transferred to other object detection tasks. In these experiments, FCOS is adopted as the detection framework, and all models are trained for 3 epochs.
Fig. 8: Detection results of zero-shot methods SAM3 and our proposed DEAL on DOTA-v2.0 dataset. Green boxes, red boxes, red points, and orange boxes denote the gt , box visual prompts, point visual prompts, and detection results, respectively.
Fig. 9: Detection results on images collected from the Internet. All images are resized to a resolution of 1024 × 1024. None of these images belong to the training datasets, and they exhibit unseen categories or data distributions, such as cartoon images and surveillance camera footage. Red points and orange boxes denote the point visual prompts and the corresponding detection results, respectively.
Fig. 10: Unseen sceneries feature association visualization. In this figure, green bounding boxes denote ground-truth annotations, and red stars indicate the point prompts. Each feature map visualizes the cosine similarity between the feature at every spatial location and the feature at the prompted point location.
Shenzhen Campus of Sun Yat-sen University · Beijing University of Posts and Telecommunications, Beijing, China · Hangzhou International Innovation Institute, Beihang University, Hangzhou, China +2