Dual-Granularity Semantic Prompting for Language Guidance Infrared Small Target Detection

Zixuan Wang; Haoran Sun; Jiaming Lu; Wenxuan Wang; Zhongling Huang; Dingwen Zhang; Xuelin Qian; Junwei Han

cs.CVNov 24, 2025

Dual-Granularity Semantic Prompting for Language Guidance Infrared Small Target Detection

Authors: Zixuan Wang, Haoran Sun, Jiaming Lu, Wenxuan Wang, Zhongling Huang, Dingwen Zhang, Xuelin Qian, Junwei Han

Organizations: School of Automation, Northwestern Polytechnical University Xi’an, Shaanxi 710072, China

Abstract

Infrared small target detection remains challenging due to limited feature representation and severe background interference, resulting in sub-optimal performance. While recent CLIP-inspired methods attempt to leverage textual guidance for detection, they are hindered by inaccurate text descriptions and reliance on manual annotations. To overcome these limitations, we propose DGSPNet, an end-to-end language prompt-driven framework. Our approach integrates dual-granularity semantic prompts: coarse-grained textual priors (e.g., 'infrared image', 'small target') and fine-grained personalized semantic descriptions derived through visual-to-textual mapping within the image space. This design not only facilitates learning fine-grained semantic information but also can inherently leverage language prompts during inference without relying on any annotation requirements. By fully leveraging the precision and conciseness of text descriptions, we further introduce a text-guide channel attention (TGCA) mechanism and text-guide spatial attention (TGSA) mechanism that enhances the model's sensitivity to potential targets across both low- and high-level feature spaces. Extensive experiments demonstrate that our method significantly improves detection accuracy and achieves state-of-the-art performance on three benchmark datasets.

Figures & tables

Fig. 1: Schematic of our proposed DGSPNet. Given an input infrared image, the pre-encoder first extracts multi-scale visual features, which are then converted into the textual space through an inversion net. The outputs, act as fine-grained implicit prompts, are integrated with a coarse-grained explicit template to form our dual-granularity semantic prompts. The semantic prompts are subsequently fed into the re-encoder and decoder to guide the learning and refinement of channel and spatial information, respectively. The illustration is organized around our two-stage training pipeline. The pre-training stage jointly trains all components using a reconstruction objective. In the normal-training stage, the inversion network is frozen, and the remaining modules are further fine-tuned.

Fig. 2: Illustration of TGCA and TGSA modules. Guided by text features, these modules generate channel and spatial attention weights, which are applied via multiplication and residual connections to enhance visual features during the encoding and decoding process.

Method	Backbone	Size	IRSTD-1K			NUDT-SIRST			NUAA-SIRST
Method	Backbone	Size	IoU $\uparrow$	P ${}_{d}\uparrow$	F ${}_{a}\downarrow$	IoU $\uparrow$	P ${}_{d}\uparrow$	F ${}_{a}\downarrow$	IoU $\uparrow$	P ${}_{d}\uparrow$	F ${}_{a}\downarrow$
IPI [ 40 ]	Traditional	256	27.92	81.37	16.18	17.76	74.49	41.23	25.67	84.63	16.67
PSTNN [ 41 ]	Traditional	256	24.57	71.99	35.26	14.85	66.13	44.17	30.30	72.80	48.99
MSLSTIPT [ 42 ]	Traditional	256	11.43	79.03	1524	8.342	47.40	888.1	10.30	82.13	1131
RKformer [ 43 ]	Hybird	512	64.12	93.27	18.65	92.25	96.86	6.580	77.24	99.11	1.580
IAANet [ 44 ]	Hybird	256	66.25	93.15	14.20	90.22	97.26	8.320	74.22	93.53	22.70

TABLE I: Comparison with existing IRSTD approaches on the IRSTD-1k, NUDT-SIRST, and NUAA-SIRST datasets. The evaluation metrics are IoU (

10^{-2}

P_{d}

(

10^{-2}

), and

F_{a}

(

10^{-6}

).The first and second best results are highlighted in bold and underlined , respectively. The symbol “-” denotes that the corresponding result is not reported in the original paper.

Prompt-driven Module Variant				IRSTD-1K			NUDT-SIRST
Baseline	TGCA	Cross-Attention	TGSA	IoU $\uparrow$	$P_{d}\uparrow$	$F_{a}\downarrow$	IoU $\uparrow$	$P_{d}\uparrow$	$F_{a}\downarrow$
✓				65.31	91.91	17.72	94.38	98.15	8.66
✓	✓			66.99	92.25	20.89	94.43	98.83	2.69
✓	✓	✓		67.54	92.92	26.93	94.66	98.73	2.64
✓	✓		✓	67.87	92.59	27.97	95.27	99.04	2.57
✓		✓	✓	67.81	91.91	36.00	94.81	98.94	1.49

TABLE II: Ablation study on the prompt-driven semantic guidance module to evaluate the necessity and synergy of sub-components ( i.e. , TGCA, Cross-Attention, TGSA). Experiments are conducted on IRSTD-1K and NUDT-SIRST datasets, with evaluation metrics including IoU (

10^{-2}

P_{d}

(

10^{-2}

), and

F_{a}

(

10^{-6}

#Tokens	Text Variant	IoU $\uparrow$	$P_{d}\uparrow$	$F_{a}\downarrow$
0	A photo of an infrared image, with targets in the background.	66.41	91.92	22.43
1	A photo of an infrared image, with $s_{1}^{*}$ in the background.	69.29	92.25	17.78
2	A photo of an infrared image, with $s_{1}^{}$ in $s_{2}^{}$ background.	70.87	93.26	14.23
3	A photo of an infrared image, with $s_{1}^{}$ $s_{2}^{}$ in $s_{3}^{*}$ background.	67.37	93.26	16.45
4	A photo of an infrared image, with $s_{1}^{}$ $s_{2}^{}$ in $s_{3}^{}$ $s_{4}^{}$ background.	69.22	91.58	19.91

TABLE III: Ablation study on text designs to evaluate the influence of the number of fine-grained semantic tokens (#Tokens) on model performance, implemented on the IRSTD-1K dataset. Various text variants (incorporating different quantities of fine-grained tokens such as

s_{1}^{*}

s_{2}^{*}

) are assessed, with evaluation metrics including IoU(

10^{-2}

P_{d}

(

10^{-2}

), and

F_{a}

(

10^{-6}

Method	IoU $\uparrow$	P ${}_{d}\uparrow$	F ${}_{a}\downarrow$
w/o pre-training	67.78	93.60	33.78
w/o freezing the inversion network	68.22	93.93	37.34
Ours	70.87	93.26	14.23

TABLE IV: Ablation study of training strategies on IRSTD-1K. Experiments verify the effects of whether to adopt pre-training and whether to freeze the inversion net, with evaluation metrics including IoU (

10^{-2}

P_{d}

(

10^{-2}

), and

F_{a}

(

10^{-6}

$\lambda_{rec}$	$\lambda_{contra}$	IoU $\uparrow$	$P_{d}$ $\uparrow$	$F_{a}$ $\downarrow$
1	0.5	69.91	92.78	15.64
1	1	70.87	93.26	14.23
2	1	69.75	92.51	16.89
1	2	68.92	91.83	21.47
1	0	68.83	92.25	20.26
0	1	66.60	91.91	39.01

TABLE V: Ablation study on the pre-training loss weights.

\lambda_{rec}

and

\lambda_{contra}

denote the weights of the reconstruction loss and the contrastive loss, respectively. The evaluation metrics are IoU (

10^{-2}

P_{d}

(

10^{-2}

), and

F_{a}

(

10^{-6}

). The best results are highlighted in bold .

Encoder Dimension	IRSTD-1K
Encoder Dimension	IoU $\uparrow$	P ${}_{d}\uparrow$	F ${}_{a}\downarrow$	Param.
(8, 16, 32, 64, 128)	70.49	90.57	23.64	1.16
(16, 32, 64, 128, 256)	70.02	93.60	17.97	2.23
(32, 64, 128, 256, 512)	70.87	93.26	14.23	6.39
(64, 128, 256, 512, 1024)	67.51	89.90	24.46	22.78

TABLE VI: Ablation study on dimensions of the encoder. The evaluation metrics are IoU (

10^{-2}

P_{d}

(

10^{-2}

), and

F_{a}

(

10^{-6}

). “Param.” refers to the number of parameters (M) of the U-Net architecture. The best results are highlighted in bold .

No.	Layers	IRSTD-1K
No.	Layers	IoU $\uparrow$	P ${}_{d}\uparrow$	F ${}_{a}\downarrow$
1	(1, 2)	67.70	89.90	10.68
2	(1, 2, 3)	70.87	93.26	14.23
3	(1, 2, 3, 4)	69.39	91.92	18.56
4	(1, 2, 3, 4, 5)	68.63	90.91	20.53

TABLE VII: Ablation study on the input of the inversion net. The evaluation metrics are IoU (

10^{-2}

P_{d}

(

10^{-2}

), and

F_{a}

(

10^{-6}

). The best results are highlighted in bold .

Loss coefficients Variant		IRSTD-1K
$\lambda_{1}$	$\lambda_{2}$	IoU $\uparrow$	P ${}_{d}\uparrow$	F ${}_{a}\downarrow$
1	1	70.87	93.26	14.23
1	0.5	69.36	91.92	19.64
0.5	1	69.04	94.27	21.67
1	0	66.01	91.24	19.01
0	1	68.53	90.91	20.19

TABLE VIII: Ablation study on loss coefficient.

\lambda_{1},\lambda_{2}

denote the coefficients of BCE loss and SoftIou loss, respectively. The evaluation metrics are IoU (

10^{-2}

P_{d}

(

10^{-2}

), and

F_{a}

(

10^{-6}

). The best results are highlighted in bold .

Methods	ALCNet [ 26 ]	ISNet [ 5 ]	AGPCNet [ 52 ]	DNANet [ 48 ]	UIUNet [ 49 ]	MSHNet [ 56 ]	L2SKNet [ 55 ]	DGSPNet ( Ours )
FPS	97.08	57.80	41.15	33.22	31.54	40.81	86.20	89.9

TABLE IX: FPS Comparison of Existing Infrared Target Detection Approaches on IRSTD-1k Dataset

Variant	Ablation Study	IoU $\uparrow$	P ${}_{d}\uparrow$	F ${}_{a}\downarrow$
-	Full model	70.87	93.26	14.23
w/o text (similar capacity)	Visual-only baseline	65.31	91.91	17.72
	Visual-only attention	65.75	92.26	15.42
	Visual-only attention w/ pre-training	66.12	88.89	16.26
prompt tuning-like design	Learnable query in Inversion Net	67.83	91.24	35.18
prompt tuning-like design	Learnable fine-grained tokens	69.17	89.89	35.67

TABLE X: Further ablation study on the design of fine-grained semantic tokens. Comparisons against three categories of alternative designs: (i) visual-only baselines with standard attention and comparable model capacity, (ii) prompt tuning-like designs with learnable but image-independent prompts, and (iii) special cases using random queries, random fine-grained tokens, or fixed manual text prompts. All variants are evaluated on the IRSTD-1K dataset using IoU,

P_{d}

, and

F_{a}

as metrics.

Fig. 3: t-SNE visualization of the learned fine-grained semantic token $s^{*}_{2}$ extracted from random selected samples. Images highlighted by yellow indicate samples with a mixture of background categories.

Fig. 4: Visualization of different IRSTD methods. The green boxes indicate correctly detected targets, red boxes denote false alarms, and orange circles represent missed targets. Additionally, dashed boxes show close-up views of the targets for clearer visualization.

Fig. 5: The left half presents the visualization of the TGCA module, while the right half shows that of the TGSA. The first column shows the original images, the second column presents the ground-truth masks, while the third and fourth columns are the feature maps before and after being fed into the TGCA and TGSA modules, respectively.

Infrared Small Target Detection

Infrared small target detection finds faint, often sub-pixel objects against cluttered thermal backgrounds while limiting false alarms.

See all 46 papers

Sep 1, 2026

DGNet: Dual-knowledge Guided Network for Infrared Small Target Detection

InfRared Small Target Detection (IRSTD) is a prominent and challenging task in computer vision. In recent years, text-guided methods have significantly improved detection performance. However, they still suffer from two key limitations. First, a single text description simultaneously modeling both background and target leads to semantic entanglement, which contradicts the objective of background suppression and target enhancement. Second, reliance on image-specific textual prompts (requiring additional external models such as CLIP during inference) results in deployment constraints. To address these issues, we propose a novel Dual-knowledge Guided Network (DGNet) based on multiple generalizable texts. Specifically, we design a Prior-knowledge Wavelet Modulation (PWM) module, which leverages dual textual priors that separately characterize large-scale backgrounds and sparse targets to effectively disentangle and modulate entangled semantics in the frequency domain. Furthermore, we introduce a Consensus-knowledge Directional Alignment (CDA) loss, which models the initial state and the ideal target across samples as complex background' and bright target', respectively, thereby constructing a clear and unified directional optimization trajectory for the model. Extensive experiments on three public datasets demonstrate the superior performance of DGNet and the effectiveness of each component. The source code is available at https://github.com/iLearn-Lab/MM26-DGNet.

Chenglong Yu, Mingzhu Xu, Jing Wang +3

Shandong University · Jinan, China · Harbin Institute of Technology, Shenzhen +1

Infrared Small Target Detection Object Detection

Ask PDF

Sep 1, 2026

ADGNet: Asymmetric Dual-text Guided Network for Infrared Small Target Detection

InfRared Small Target Detection (IRSTD) is a challenging task. Relying solely on pixel-level information, vision-only methods struggle to distinguish targets from clutter. Current multimodal methods typically describe both targets and backgrounds with a single textual prompt. Such an approach lacks dedicated regional guidance and ignores infrared semantic asymmetry. Consequently, it provides insufficient background suppression information and introduces severe feature optimization conflicts, overwhelming small targets with noise. To address these issues, we propose a novel Asymmetric Dual-text Guided Network (ADGNet). Specifically, accounting for the infrared semantic asymmetry, we first design the Asymmetric Dual-text Prompt (ADP), comprising an image-agnostic abstract target prompt and an image-specific detailed background prompt. To leverage these prompts, we introduce an Asymmetric Dual-Branch Interaction (ADBI) module to separately guide visual features with their respective text priors, protecting targets from noise while suppressing background clutter. Subsequently, we introduce an Adaptive Feature Aggregation (AFA) module to dynamically fuse features from the two branches. Furthermore, we construct a multimodal Asymmetric Image-Text Infrared (AITIR) dataset by providing asymmetric text annotations for three public datasets (IRSTD-1K, NUDT-SIRST, and SIRST). Extensive experiments demonstrate that ADGNet outperforms 21 state-of-the-art (SOTA) methods. Code is available at https://github.com/iLearn-Lab/MM26-ADGNet.

Tongtong Wang, Mingzhu Xu, Chenglong Yu +3

Shandong University · Jinan, China · Harbin Institute of Technology, Shenzhen +1

Prompt Learning Infrared Small Target Detection Multimodal Fusion

Ask PDF

Mar 26, 2026

FSGNet: A Frequency-Aware and Semantic Guidance Network for Infrared Small Target Detection

Infrared small target detection (IRSTD) aims to identify and distinguish small targets from complex backgrounds. Leveraging the powerful multi-scale feature fusion capability of the U-Net architecture, IRSTD has achieved significant progress. However, U-Net suffers from semantic degradation when transferring high-level features from deep to shallow layers, limiting the precise localization of small targets. To address this issue, this paper proposes FSGNet, a lightweight and effective detection framework incorporating frequency-aware and semantic guidance mechanisms. Specifically, a multi-directional interactive attention module is proposed throughout the encoder to capture fine-grained and directional features, enhancing the network's sensitivity to small, low-contrast targets. To suppress background interference propagated through skip connections, a multi-scale frequency-aware module leverages Fast Fourier transform to filter out target-similar clutter while preserving salient target structures. At the deepest layer, a global pooling module captures high-level semantic information, which is subsequently upsampled and propagated to each decoder stage through the global semantic guidance flows, ensuring semantic consistency and precise localization across scales. Extensive experiments on four public IRSTD datasets demonstrate that FSGNet achieves superior detection performance and maintains high efficiency, highlighting its practical applicability and robustness. The codes will be released on https://github.com/Wangtao-Bao/FSGNet.

Yingmei Zhang, Wangtao Bao, Yong Yang +3

School of Software and Internet of Things Engineering, Jiangxi University of Finance and Economics, Nanchang 330032, China · the postdoctoral researcher, CGN Begood Technology co., Ltd, Nanchang 330032, China · School of Computer Science and Technology, Tiangong University, Tianjin 300387, China +1

Infrared Small Target Detection Object Detection Multi-Scale Attention

Ask PDF

Abstract

Infrared Small Target Detection

Explore similar work

DGNet: Dual-knowledge Guided Network for Infrared Small Target Detection

ADGNet: Asymmetric Dual-text Guided Network for Infrared Small Target Detection

FSGNet: A Frequency-Aware and Semantic Guidance Network for Infrared Small Target Detection