Dual-Granularity Semantic Prompting for Language Guidance Infrared Small Target Detection

Zixuan Wang; Haoran Sun; Jiaming Lu; Wenxuan Wang; Zhongling Huang; Dingwen Zhang; Xuelin Qian; Junwei Han

cs.CVNov 24, 2025

Dual-Granularity Semantic Prompting for Language Guidance Infrared Small Target Detection

Authors: Zixuan Wang, Haoran Sun, Jiaming Lu, Wenxuan Wang, Zhongling Huang, Dingwen Zhang, Xuelin Qian, Junwei Han

Organizations: School of Automation, Northwestern Polytechnical University Xi’an, Shaanxi 710072, China

Abstract

Infrared small target detection remains challenging due to limited feature representation and severe background interference, resulting in sub-optimal performance. While recent CLIP-inspired methods attempt to leverage textual guidance for detection, they are hindered by inaccurate text descriptions and reliance on manual annotations. To overcome these limitations, we propose DGSPNet, an end-to-end language prompt-driven framework. Our approach integrates dual-granularity semantic prompts: coarse-grained textual priors (e.g., 'infrared image', 'small target') and fine-grained personalized semantic descriptions derived through visual-to-textual mapping within the image space. This design not only facilitates learning fine-grained semantic information but also can inherently leverage language prompts during inference without relying on any annotation requirements. By fully leveraging the precision and conciseness of text descriptions, we further introduce a text-guide channel attention (TGCA) mechanism and text-guide spatial attention (TGSA) mechanism that enhances the model's sensitivity to potential targets across both low- and high-level feature spaces. Extensive experiments demonstrate that our method significantly improves detection accuracy and achieves state-of-the-art performance on three benchmark datasets.

Figures & tables

Fig. 1: Schematic of our proposed DGSPNet. Given an input infrared image, the pre-encoder first extracts multi-scale visual features, which are then converted into the textual space through an inversion net. The outputs, act as fine-grained implicit prompts, are integrated with a coarse-grained explicit template to form our dual-granularity semantic prompts. The semantic prompts are subsequently fed into the re-encoder and decoder to guide the learning and refinement of channel and spatial information, respectively. The illustration is organized around our two-stage training pipeline. The pre-training stage jointly trains all components using a reconstruction objective. In the normal-training stage, the inversion network is frozen, and the remaining modules are further fine-tuned.

Fig. 2: Illustration of TGCA and TGSA modules. Guided by text features, these modules generate channel and spatial attention weights, which are applied via multiplication and residual connections to enhance visual features during the encoding and decoding process.

Method	Backbone	Size	IRSTD-1K			NUDT-SIRST			NUAA-SIRST
Method	Backbone	Size	IoU $\uparrow$	P ${}_{d}\uparrow$	F ${}_{a}\downarrow$	IoU $\uparrow$	P ${}_{d}\uparrow$	F ${}_{a}\downarrow$	IoU $\uparrow$	P ${}_{d}\uparrow$	F ${}_{a}\downarrow$
IPI [ 40 ]	Traditional	256	27.92	81.37	16.18	17.76	74.49	41.23	25.67	84.63	16.67
PSTNN [ 41 ]	Traditional	256	24.57	71.99	35.26	14.85	66.13	44.17	30.30	72.80	48.99
MSLSTIPT [ 42 ]	Traditional	256	11.43	79.03	1524	8.342	47.40	888.1	10.30	82.13	1131
RKformer [ 43 ]	Hybird	512	64.12	93.27	18.65	92.25	96.86	6.580	77.24	99.11	1.580
IAANet [ 44 ]	Hybird	256	66.25	93.15	14.20	90.22	97.26	8.320	74.22	93.53	22.70

TABLE I: Comparison with existing IRSTD approaches on the IRSTD-1k, NUDT-SIRST, and NUAA-SIRST datasets. The evaluation metrics are IoU (

10^{-2}

P_{d}

(

10^{-2}

), and

F_{a}

(

10^{-6}

).The first and second best results are highlighted in bold and underlined , respectively. The symbol “-” denotes that the corresponding result is not reported in the original paper.

Prompt-driven Module Variant				IRSTD-1K			NUDT-SIRST
Baseline	TGCA	Cross-Attention	TGSA	IoU $\uparrow$	$P_{d}\uparrow$	$F_{a}\downarrow$	IoU $\uparrow$	$P_{d}\uparrow$	$F_{a}\downarrow$
✓				65.31	91.91	17.72	94.38	98.15	8.66
✓	✓			66.99	92.25	20.89	94.43	98.83	2.69
✓	✓	✓		67.54	92.92	26.93	94.66	98.73	2.64
✓	✓		✓	67.87	92.59	27.97	95.27	99.04	2.57
✓		✓	✓	67.81	91.91	36.00	94.81	98.94	1.49

TABLE II: Ablation study on the prompt-driven semantic guidance module to evaluate the necessity and synergy of sub-components ( i.e. , TGCA, Cross-Attention, TGSA). Experiments are conducted on IRSTD-1K and NUDT-SIRST datasets, with evaluation metrics including IoU (

10^{-2}

P_{d}

(

10^{-2}

), and

F_{a}

(

10^{-6}

#Tokens	Text Variant	IoU $\uparrow$	$P_{d}\uparrow$	$F_{a}\downarrow$
0	A photo of an infrared image, with targets in the background.	66.41	91.92	22.43
1	A photo of an infrared image, with $s_{1}^{*}$ in the background.	69.29	92.25	17.78
2	A photo of an infrared image, with $s_{1}^{}$ in $s_{2}^{}$ background.	70.87	93.26	14.23
3	A photo of an infrared image, with $s_{1}^{}$ $s_{2}^{}$ in $s_{3}^{*}$ background.	67.37	93.26	16.45
4	A photo of an infrared image, with $s_{1}^{}$ $s_{2}^{}$ in $s_{3}^{}$ $s_{4}^{}$ background.	69.22	91.58	19.91

TABLE III: Ablation study on text designs to evaluate the influence of the number of fine-grained semantic tokens (#Tokens) on model performance, implemented on the IRSTD-1K dataset. Various text variants (incorporating different quantities of fine-grained tokens such as

s_{1}^{*}

s_{2}^{*}

) are assessed, with evaluation metrics including IoU(

10^{-2}

P_{d}

(

10^{-2}

), and

F_{a}

(

10^{-6}

Method	IoU $\uparrow$	P ${}_{d}\uparrow$	F ${}_{a}\downarrow$
w/o pre-training	67.78	93.60	33.78
w/o freezing the inversion network	68.22	93.93	37.34
Ours	70.87	93.26	14.23

TABLE IV: Ablation study of training strategies on IRSTD-1K. Experiments verify the effects of whether to adopt pre-training and whether to freeze the inversion net, with evaluation metrics including IoU (

10^{-2}

P_{d}

(

10^{-2}

), and

F_{a}

(

10^{-6}

$\lambda_{rec}$	$\lambda_{contra}$	IoU $\uparrow$	$P_{d}$ $\uparrow$	$F_{a}$ $\downarrow$
1	0.5	69.91	92.78	15.64
1	1	70.87	93.26	14.23
2	1	69.75	92.51	16.89
1	2	68.92	91.83	21.47
1	0	68.83	92.25	20.26
0	1	66.60	91.91	39.01

TABLE V: Ablation study on the pre-training loss weights.

\lambda_{rec}

and

\lambda_{contra}

denote the weights of the reconstruction loss and the contrastive loss, respectively. The evaluation metrics are IoU (

10^{-2}

P_{d}

(

10^{-2}

), and

F_{a}

(

10^{-6}

). The best results are highlighted in bold .

Encoder Dimension	IRSTD-1K
Encoder Dimension	IoU $\uparrow$	P ${}_{d}\uparrow$	F ${}_{a}\downarrow$	Param.
(8, 16, 32, 64, 128)	70.49	90.57	23.64	1.16
(16, 32, 64, 128, 256)	70.02	93.60	17.97	2.23
(32, 64, 128, 256, 512)	70.87	93.26	14.23	6.39
(64, 128, 256, 512, 1024)	67.51	89.90	24.46	22.78

TABLE VI: Ablation study on dimensions of the encoder. The evaluation metrics are IoU (

10^{-2}

P_{d}

(

10^{-2}

), and

F_{a}

(

10^{-6}

). “Param.” refers to the number of parameters (M) of the U-Net architecture. The best results are highlighted in bold .

No.	Layers	IRSTD-1K
No.	Layers	IoU $\uparrow$	P ${}_{d}\uparrow$	F ${}_{a}\downarrow$
1	(1, 2)	67.70	89.90	10.68
2	(1, 2, 3)	70.87	93.26	14.23
3	(1, 2, 3, 4)	69.39	91.92	18.56
4	(1, 2, 3, 4, 5)	68.63	90.91	20.53

TABLE VII: Ablation study on the input of the inversion net. The evaluation metrics are IoU (

10^{-2}

P_{d}

(

10^{-2}

), and

F_{a}

(

10^{-6}

). The best results are highlighted in bold .

Loss coefficients Variant		IRSTD-1K
$\lambda_{1}$	$\lambda_{2}$	IoU $\uparrow$	P ${}_{d}\uparrow$	F ${}_{a}\downarrow$
1	1	70.87	93.26	14.23
1	0.5	69.36	91.92	19.64
0.5	1	69.04	94.27	21.67
1	0	66.01	91.24	19.01
0	1	68.53	90.91	20.19

TABLE VIII: Ablation study on loss coefficient.

\lambda_{1},\lambda_{2}

denote the coefficients of BCE loss and SoftIou loss, respectively. The evaluation metrics are IoU (

10^{-2}

P_{d}

(

10^{-2}

), and

F_{a}

(

10^{-6}

). The best results are highlighted in bold .

Methods	ALCNet [ 26 ]	ISNet [ 5 ]	AGPCNet [ 52 ]	DNANet [ 48 ]	UIUNet [ 49 ]	MSHNet [ 56 ]	L2SKNet [ 55 ]	DGSPNet ( Ours )
FPS	97.08	57.80	41.15	33.22	31.54	40.81	86.20	89.9

TABLE IX: FPS Comparison of Existing Infrared Target Detection Approaches on IRSTD-1k Dataset

Variant	Ablation Study	IoU $\uparrow$	P ${}_{d}\uparrow$	F ${}_{a}\downarrow$
-	Full model	70.87	93.26	14.23
w/o text (similar capacity)	Visual-only baseline	65.31	91.91	17.72
	Visual-only attention	65.75	92.26	15.42
	Visual-only attention w/ pre-training	66.12	88.89	16.26
prompt tuning-like design	Learnable query in Inversion Net	67.83	91.24	35.18
prompt tuning-like design	Learnable fine-grained tokens	69.17	89.89	35.67

TABLE X: Further ablation study on the design of fine-grained semantic tokens. Comparisons against three categories of alternative designs: (i) visual-only baselines with standard attention and comparable model capacity, (ii) prompt tuning-like designs with learnable but image-independent prompts, and (iii) special cases using random queries, random fine-grained tokens, or fixed manual text prompts. All variants are evaluated on the IRSTD-1K dataset using IoU,

P_{d}

, and

F_{a}

as metrics.

Fig. 3: t-SNE visualization of the learned fine-grained semantic token $s^{*}_{2}$ extracted from random selected samples. Images highlighted by yellow indicate samples with a mixture of background categories.

Fig. 4: Visualization of different IRSTD methods. The green boxes indicate correctly detected targets, red boxes denote false alarms, and orange circles represent missed targets. Additionally, dashed boxes show close-up views of the targets for clearer visualization.

Fig. 5: The left half presents the visualization of the TGCA module, while the right half shows that of the TGSA. The first column shows the original images, the second column presents the ground-truth masks, while the third and fourth columns are the feature maps before and after being fed into the TGCA and TGSA modules, respectively.

Infrared Small Target Detection

Infrared small target detection finds faint, often sub-pixel objects against cluttered thermal backgrounds while limiting false alarms.

See all 46 papers

DGNet: Dual-knowledge Guided Network for Infrared Small Target Detection
Sep 1, 2026Chenglong Yu, Mingzhu Xu, Jing Wang +3Infrared Small Target Detection Object Detection
Shandong University · Jinan, China · Harbin Institute of Technology, Shenzhen +1
ADGNet: Asymmetric Dual-text Guided Network for Infrared Small Target Detection
Sep 1, 2026Tongtong Wang, Mingzhu Xu, Chenglong Yu +3Prompt Learning Infrared Small Target Detection
Shandong University · Jinan, China · Harbin Institute of Technology, Shenzhen +1
FSGNet: A Frequency-Aware and Semantic Guidance Network for Infrared Small Target Detection
Mar 26, 2026Yingmei Zhang, Wangtao Bao, Yong Yang +3Infrared Small Target Detection Object Detection
School of Software and Internet of Things Engineering, Jiangxi University of Finance and Economics, Nanchang 330032, China · the postdoctoral researcher, CGN Begood Technology co., Ltd, Nanchang 330032, China · School of Computer Science and Technology, Tiangong University, Tianjin 300387, China +1

Sep 1, 2026J/K move · Enter open · S save

DGNet: Dual-knowledge Guided Network for Infrared Small Target Detection

Chenglong Yu, Mingzhu Xu, Jing Wang, Tongtong Wang+2

Shandong University · Jinan, China · Harbin Institute of Technology, Shenzhen+1

Infrared Small Target Detection Object Detection

Open paper AskPDF arXiv

InfRared Small Target Detection (IRSTD) is a prominent and challenging task in computer vision. In recent years, text-guided methods have significantly improved detection performance. However, they still suffer from two key limitations. First, a single text description simultaneously modeling both background and target leads to semantic entanglement, which contradicts the objective of background suppression and target enhancement. Second, reliance on image-specific textual prompts (requiring additional external models such as CLIP during inference) results in deployment constraints. To address these issues, we propose a novel Dual-knowledge Guided Network (DGNet) based on multiple generalizable texts. Specifically, we design a Prior-knowledge Wavelet Modulation (PWM) module, which leverages dual textual priors that separately characterize large-scale backgrounds and sparse targets to effectively disentangle and modulate entangled semantics in the frequency domain. Furthermore, we introduce a Consensus-knowledge Directional Alignment (CDA) loss, which models the initial state and the ideal target across samples as complex background' and bright target', respectively, thereby constructing a clear and unified directional optimization trajectory for the model. Extensive experiments on three public datasets demonstrate the superior performance of DGNet and the effectiveness of each component. The source code is available at https://github.com/iLearn-Lab/MM26-DGNet.

Abstract

Infrared Small Target Detection

Explore similar work

DGNet: Dual-knowledge Guided Network for Infrared Small Target Detection

ADGNet: Asymmetric Dual-text Guided Network for Infrared Small Target Detection

FSGNet: A Frequency-Aware and Semantic Guidance Network for Infrared Small Target Detection