Prompt-Free Universal Region Proposal Network

Qihong Tang; Changhan Liu; Shaofeng Zhang; Wenbin Li; Qi Fan; Yang Gao

cs.CVMar 18, 2026

Prompt-Free Universal Region Proposal Network

Authors: Qihong Tang, Changhan Liu, Shaofeng Zhang, Wenbin Li, Qi Fan, Yang Gao

Organizations: Nanjing University · University of Science and Technology of China

Abstract

Identifying potential objects is critical for object recognition and analysis across various computer vision applications. Existing methods typically localize potential objects by relying on exemplar images, predefined categories, or textual descriptions. However, their reliance on image and text prompts often limits flexibility, restricting adaptability in real-world scenarios. In this paper, we introduce a novel Prompt-Free Universal Region Proposal Network (PF-RPN), which identifies potential objects without relying on external prompts. First, the Sparse Image-Aware Adapter (SIA) module performs initial localization of potential objects using a learnable query embedding dynamically updated with visual features. Next, the Cascade Self-Prompt (CSP) module identifies the remaining potential objects by leveraging the self-prompted learnable embedding, autonomously aggregating informative visual features in a cascading manner. Finally, the Centerness-Guided Query Selection (CG-QS) module facilitates the selection of high-quality query embeddings using a centerness scoring network. Our method can be optimized with limited data (e.g., 5% of MS COCO data) and applied directly to various object detection application domains for identifying potential objects without fine-tuning, such as underwater object detection, industrial defect detection, and remote sensing image object detection. Experimental results across 19 datasets validate the effectiveness of our method. Code is available at https://github.com/tangqh03/PF-RPN.

Figures & tables

Figure 1: Existing visual/text prompt based OVD methods typically rely on predefined categories or exemplar images to propose potential objects for the target image. Recent prompt-free OVD methods often leverage VLMs to generate textual descriptions for the target image to localize potential objects which introduce considerable latency costs. In contrast, our PF-RPN doesn’t require any external prompts and only utilizes visual features to generate high-quality proposals. Experimental results show the effectiveness of our PF-RPN in localizing potential objects.

Figure 2: Overall architecture of our PF-RPN. It comprises three core components: (1) the Sparse Image-Aware Adapter (SIA) module, which adaptively integrates multi-level feature maps $F^{I}_{i}$ with a learnable embedding $F^{T}$ via a routing mechanism and cross-attention; (2) the Cascade Self-Prompt (CSP) module, which iteratively refines the embedding through masked average pooling across multiple visual levels; and (3) the Centerness-Guided Query Selection (CG-QS) module, which decodes the features into final predictions optimized by contrastive, regression, and centerness losses.

Datasets	Methods	Prompt Free	$AR_{100}$	$AR_{300}$	$AR_{900}$	$AR_{s}$	$AR_{m}$	$AR_{l}$
	GDINO † [ 26 ]	✗	52.9	53.5	54.7	31.1	41.6	63.9
	GDINO ‡ [ 26 ]	✓	54.7	57.8	61.6	34.1	49.3	67.0
	YOLOE-v8-L † [ 40 ]	✗	44.4	46.2	47.1	21.6	36.6	54.9
	YWorldv8-L † [ 6 ]	✗	49.6	51.1	51.6	25.1	42.7	60.6
	Qwen-VL † [ 2 ]	✗	20.1	20.1	20.1	1.0	3.0	26.5
	GLIP † [ 21 ]	✗	47.6	47.6	47.6	21.2	34.6	56.0

Table 1: Comparison with OVD models, MLLMs, and RPNs. The best results are highlighted in bold .

AR_{100/300/900}

denotes the average recall under 100/300/900 candidate boxes, respectively, and

AR_{s/m/l}

denotes the average recall for small/medium/large objects.

\dagger

indicates using the original class names from the corresponding dataset as text input, and

\ddagger

indicates replacing the original class names with “object” to obtain a prompt-free setting. Our method achieves the best performance on both the CD-FSOD and ODinW13 benchmarks.

Figure 3: Effect of iterations in the Cascade Self-Prompt module. Visualization of region selection across different Cascade Self-Prompt iterations. Green points indicate the object regions selected by the model in the current iteration. As the number of iterations increases, the model progressively selects more object regions in the image, demonstrating the effectiveness of our cascade self-prompt mechanism.

Figure 4: Effect of the Sparse Image-Aware Adapter. Visualization of similarity heatmaps between the learnable embedding and image features before and after the module update. Each pair of heatmaps (top: before, bottom: after) corresponds to the same image. After the update, the learnable embedding exhibits stronger responses in semantically relevant regions, indicating improved alignment between visual and learned representations and providing a stronger prior for the cascade self-prompt module.

SIA	CSP	CG-QS	$AR_{100}$	$AR_{300}$	$AR_{900}$	$AR_{s}$	$AR_{m}$	$AR_{l}$
$\times$	$\times$	$\times$	52.9	53.5	54.7	31.1	41.6	63.9
$\checkmark$	$\times$	$\times$	57.8	63.1	66.7	47.9	58.2	77.5
$\times$	$\checkmark$	$\times$	58.1	62.9	65.8	37.7	56.7	77.0
$\times$	$\times$	$\checkmark$	54.4	56.9	60.2	21.9	50.6	79.8
$\checkmark$	$\times$	$\checkmark$	60.2	64.7	68.0	47.1	60.7	79.1
$\times$	$\checkmark$	$\checkmark$	59.6	64.3	67.3	45.2	60.6	78.1

Table 2: Results of module ablation studies. SIA denotes the Sparse Image-Aware Adapter, CSP denotes the Cascade Self-Prompt module and CG-QS denotes the Centerness-Guided Query Selection module. The best results are highlighted in bold .

COCO	IN	$AR_{100}$	$AR_{300}$	$AR_{900}$	$AR_{s}$	$AR_{m}$	$AR_{l}$
$1\%$	0	57.2	63.7	68.0	42.2	60.9	76.6
$5\%$	0	58.8	64.1	67.8	40.2	59.0	78.6
$10\%$	0	59.0	64.1	67.5	38.9	59.1	78.9
$1\%$	$5\%$	59.1	63.3	66.8	36.2	59.9	79.8
$5\%$	$5\%$	60.7	65.3	68.2	38.5	61.9	80.3
$10\%$	$5\%$	60.9	65.5	68.5	39.6	61.6	80.5

Table 3: Results of data ablation studies. COCO denotes the COCO dataset and IN denotes the ImageNet dataset, The best results are highlighted in bold .

Method	$AR_{100}$	$AR_{300}$	$AR_{900}$	$AR_{s}$	$AR_{m}$	$AR_{l}$
GDINO (SwinB)	52.9	53.5	54.7	31.1	41.6	63.9
+ ours	60.7	65.3	68.2	38.5	61.9	80.3
GDINO (ResNet50)	40.4	43.4	44.6	21.7	32.5	54.9
+ ours	45.6	51.4	58.0	28.1	50.6	73.7

Table 4: Results with different backbones. The best results for each baseline are highlighted in bold . Our model effectively improves the performance of detectors with different backbone.

Dataset	Method	$AR_{100}$	$AR_{300}$	$AR_{900}$	$AR_{s}$	$AR_{m}$	$AR_{l}$
	w/o MoE	58.6	61.2	63.5	37.1	56.5	73.5
CD-FSOD	Ours	60.7	65.3	68.2	38.5	61.9	80.3
	w/o MoE	68.7	71.7	73.8	37.9	67.0	80.7
ODinW13	Ours	76.5	78.6	79.8	45.4	71.9	85.8

Table 5: Ablation study on the effectiveness of the MoE module. The best results for each baseline are highlighted in bold .

Method	Task	$AP$	$AP_{50}$	$AR_{75}$	$AP_{s}$	$AP_{m}$	$AP_{l}$
DE-VIT	OD	33.0	52.9	33.7	17.1	35.4	47.3
+ ours	OD	36.7	56.2	38.8	23.5	40.1	50.7
CD-VITO	CD-FSOD	29.6	49.1	30.0	5.7	24.4	37.6
+ ours	CD-FSOD	35.1	59.6	35.0	9.0	22.6	41.9

Table 6: Results of integrating our model into well-trained detectors. OD denotes the conventional object detection task evaluated on the COCO dataset and CDFSOD denotes the cross-domain few-shot object detection task evaluated on the CD-FSOD benchmark. The best results are highlighted in bold .

Figure 5: Effect of the Centerness-Guided Query Selection. Visualization of query selection before and after applying the Centerness-Guided Query Selection (CG-QS) module. Each pair of heatmaps (top: before, bottom: after) corresponds to the same image. After applying the CG-QS module, the model tends to select queries near object centers, thereby generating more accurate proposals.

Figure 6: Qualitative results of PF-RPN on several object detection benchmarks. PF-RPN exhibits strong cross-domain generalization and localization ability, accurately proposing potential object regions without any domain-specific fine-tuning.

Figure 7: Prediction box comparison. The red box corresponds to the query indicated by the red star, while the blue box corresponds to the query indicated by the blue star. It can be observed that the query located near the object center produces a more accurate bounding box than queries near object boundaries.

$k$	AR 100	AR 300	AR 900	AR s	AR m	AR l
1	60.3	64.7	67.4	44.2	60.0	78.8
2	60.7	65.3	68.2	38.5	61.9	80.3
3	59.9	64.4	67.3	41.5	59.0	79.6
4	59.7	64.4	67.3	41.5	59.0	79.6

Table 7: Ablation study of parameter

k

in the Sparse Image-Aware Adapter module on the CD-FSOD benchmark. Best results are highlighted in bold .

$k$	AR 100	AR 300	AR 900	AR s	AR m	AR l
1	60.5	65.1	68.1	39.6	59.8	79.4
3	60.6	65.1	68.1	39.6	60.3	79.5
5	60.7	65.3	68.2	38.5	61.9	80.3
7	58.5	63.1	66.3	37.1	58.3	79.2
9	59.6	64.3	67.4	39.7	59.6	78.9

Table 8: Ablation study of parameter

\lambda

in the Objective Loss on CD-FSOD benchmark. The best results are highlighted in bold .

Panel of Figure 8 — Figure 8: Comparison of feature localization between object-internal features and the learnable embedding $\tilde{F}^{T}$ . The similarity map in Fig. 8(a) shows that the 4th-level feature at the red point focuses on the object region with high semantic consistency, while the learnable embedding $\tilde{F}^{T}$ in Fig. 8(b) produces a more diffused response, indicating weaker object correspondence.

Iteration	$AR_{100}$	$AR_{300}$	$AR_{900}$	$AR_{s}$	$AR_{m}$	$AR_{l}$	ms/img
Iter 1	59.6	63.1	65.4	35.7	57.8	77.7	214.3
Iter 2	59.9	63.7	65.9	36.7	57.9	78.2	216.7
Iter 3	60.7	65.3	68.2	38.5	61.9	80.3	218.9

Table 9: Latency and performance analysis of different CSP iterations on the CD-FSOD benchmark.

Metric	GLIP [ 26 ]	CasRPN [ 39 ]	RPN [ 34 ]	PF-RPN(GDINO)	PF-RPN(YWorld)
$AR_{100}$	47.6	45.8	32.0	60.7	52.3
FPS	5.5	24.8	27.8	4.6	25.1

Table 10: Efficiency comparison of PF-RPN integrated with different detectors.

	DeViT [ 49 ]		DeViT + Ours		CD-ViTO [ 12 ]		CD-ViTO + Ours
Metric	Top 100	Top 300	Top 100	Top 300	Top 100	Top 300	Top 100	Top 300
FP ( $\downarrow$ )	18.2	21.7	17.1	18.3	15.3	20.2	14.6	17.9
AP ( $\uparrow$ )	32.3	33.4	37.3 (+5.0)	37.1 (+3.7)	26.8	29.0	34.2 (+7.4)	35.1 (+6.1)

Table 11: Analysis of false positives on different baselines under varying top proposal settings.

Method	$AR_{100}$	$AR_{300}$	$AR_{900}$	$AR_{s}$	$AR_{m}$	$AR_{l}$
MMGDINO-B	72.4	73.5	73.9	46.4	66.7	78.7
+ Ours	76.3	78.9	79.8	45.7	75.2	86.3
MMGDINO-L	73.6	74.2	74.5	48.3	69.8	79.4
+ Ours	77.4	79.1	80.1	48.5	78.4	86.5

Table 12: Performance comparison of integrating PF-RPN with stronger base models (MMGrounding DINO [ 58 ] ).

		Average Recall (AR)			Scale ( $AR$ )			Efficiency
Benchmark	Method	$AR_{100}$	$AR_{300}$	$AR_{900}$	$S$	$M$	$L$	FPS	VRAM
	GDINO [ 26 ]	54.7	57.8	61.6	34.1	49.3	67.0	3.3	0.9G
	GenerateU [ 22 ]	47.7	54.1	55.7	28.1	48.3	69.4	0.22	12.2G
	Open-Det [ 3 ]	36.6	46.3	54.3	28.2	45.3	67.7	0.15	30.7G
CD-FSOD	PF-RPN (Ours)	60.7	65.3	68.2	38.5	61.9	80.3	4.6	0.5G
	GDINO [ 26 ]	69.1	70.9	72.4	40.8	64.6	78.4	3.3	0.9G

Table 13: Comparison with open-source prompt-free methods regarding performance and efficiency.

Figure 9: Detailed performance (

AR

) trends on all 19 target datasets compared to alternative methods.

Object Detection

Object detection locates and classifies objects in images or video, balancing localization accuracy, robustness, and inference speed.

See all 624 papers

DETR-ViP: Detection Transformer with Robust Discriminative Visual Prompts
Apr 16, 2026Bo Qian, Dahu Shi, Xing WeiOpen-Vocabulary Object Detection Detection Transformer
School of Software Engineering, Xi’an Jiaotong University · State Key Laboratory of Human-Machine Hybrid Augmented Intelligence, Xi’an Jiaotong University · Work done during the internship at Hikrobot Co., Ltd. +2
PET-DINO: Unifying Visual Cues into Grounding DINO with Prompt-Enriched Training
Apr 1, 2026Weifu Fu, Jinyang Li, Bin-Bin Gao +6Object Detection Open-Vocabulary Object Detection
YouTu Lab, Tencent · Huazhong University of Science and Technology · Kling Team, Kuaishou Technology
GFR-SAM: Training-Free Referring Camouflaged Object Segmentation via Cross-Image Prompting
Jul 13, 2026Yilong Yang, Jianxin Tian, Shengchuan Zhang +1Image Segmentation Promptable Image Segmentation
Key Laboratory of Multimedia Trusted Perception and Efficient Computing, Ministry of Education of China, Xiamen University

Apr 16, 2026J/K move · Enter open · S save

DETR-ViP: Detection Transformer with Robust Discriminative Visual Prompts

Bo Qian, Dahu Shi, Xing Wei

School of Software Engineering, Xi’an Jiaotong University · State Key Laboratory of Human-Machine Hybrid Augmented Intelligence, Xi’an Jiaotong University · Work done during the internship at Hikrobot Co., Ltd.+2

Open-Vocabulary Object Detection Detection Transformer Visual Prompt Learning

Open paper AskPDF arXiv

Visual prompted object detection enables interactive and flexible definition of target categories, thereby facilitating open-vocabulary detection. Since visual prompts are derived directly from image features, they often outperform text prompts in recognizing rare categories. Nevertheless, research on visual prompted detection has been largely overlooked, and it is typically treated as a byproduct of training text prompted detectors, which hinders its development. To fully unlock the potential of visual-prompted detection, we investigate the reasons why its performance is suboptimal and reveal that the underlying issue lies in the absence of global discriminability in visual prompts. Motivated by these observations, we propose DETR-ViP, a robust object detection framework that yields class-distinguishable visual prompts. On top of basic image-text contrastive learning, DETR-ViP incorporates global prompt integration and visual-textual prompt relation distillation to learn more discriminative prompt representations. In addition, DETR-ViP employs a selective fusion strategy that ensures stable and robust detection. Extensive experiments on COCO, LVIS, ODinW, and Roboflow100 demonstrate that DETR-ViP achieves substantially higher performance in visual prompt detection compared to other state-of-the-art counterparts. A series of ablation studies and analyses further validate the effectiveness of the proposed improvements and shed light on the underlying reasons for the enhanced detection capability of visual prompts.

Abstract

Object Detection

Explore similar work

DETR-ViP: Detection Transformer with Robust Discriminative Visual Prompts

PET-DINO: Unifying Visual Cues into Grounding DINO with Prompt-Enriched Training

GFR-SAM: Training-Free Referring Camouflaged Object Segmentation via Cross-Image Prompting