Rethinking Representations for Cross-Domain Infrared Small Target Detection: A Generalizable Perspective from the Frequency Domain

Yimin Fu; Songbo Wang; Feiyan Wu; Jialin Lyu; Zhunga Liu; Michael K. Ng

cs.CVApr 2, 2026

Rethinking Representations for Cross-Domain Infrared Small Target Detection: A Generalizable Perspective from the Frequency Domain

Authors: Yimin Fu, Songbo Wang, Feiyan Wu, Jialin Lyu, Zhunga Liu, Michael K. Ng

Organizations: Department of Mathematics, Hong Kong Baptist University, Hong Kong, China · School of Automation, Northwestern Polytechnical University, Xi’an, 710072, China · Wangxuan Institute of Computer Technology, Peking University, Beijing, China

Abstract

The accurate target-background separation in infrared small target detection (IRSTD) highly depends on the discriminability of extracted representations. However, most existing methods are confined to domain-consistent settings, while overlooking whether such discriminability can generalize to unseen domains. In practice, distribution shifts between training and testing data are inevitable due to variations in observational conditions and environmental factors. Meanwhile, the intrinsic indistinctiveness of infrared small targets aggravates overfitting to domain-specific patterns. Consequently, the detection performance of models trained on source domains can be severely degraded when deployed in unseen domains. To address this challenge, we propose a spatial-spectral collaborative perception network (S $^2$ CPNet) for cross-domain IRSTD. Moving beyond conventional spatial learning pipelines, we rethink IRSTD representations from a frequency perspective and reveal inconsistencies in spectral phase as the primary manifestation of domain discrepancies. Based on this insight, we develop a phase rectification module (PRM) to derive generalizable target awareness. Then, we employ an orthogonal attention mechanism (OAM) in skip connections to preserve positional information while refining informative representations. Moreover, the bias toward domain-specific patterns is further mitigated through selective style recomposition (SSR). Extensive experiments have been conducted on three IRSTD datasets, and the proposed method consistently achieves state-of-the-art performance under diverse cross-domain settings.

Figures & tables

Fig. 1: Representative image examples and overall frequency characteristic of different IRSTD datasets.

Fig. 2: The architecture overview of our proposed S 2 CPNet. Three key components are employed in the encoder-decoder framework: phase rectification module, orthogonal attention mechanism, and selective style recomposition.

Method	Venue	NUDT → IRSTD-1K			NUDT → NUAA			NUAA → IRSTD-1K			NUAA → NUDT			IRSTD-1K → NUDT			IRSTD-1K → NUAA
Method	Venue	IoU $\uparrow$	$P_{d}\uparrow$	$F_{a}\downarrow$	IoU $\uparrow$	$P_{d}\uparrow$	$F_{a}\downarrow$	IoU $\uparrow$	$P_{d}\uparrow$	$F_{a}\downarrow$	IoU $\uparrow$	$P_{d}\uparrow$	$F_{a}\downarrow$	IoU $\uparrow$	$P_{d}\uparrow$	$F_{a}\downarrow$	IoU $\uparrow$	$P_{d}\uparrow$	$F_{a}\downarrow$
Tophat [ 8 ]	$\text{OE}_{96}$	5.95	69.69	621.38	27.80	90.74	193.67	5.95	69.69	621.39	12.59	69.41	76.50	12.59	69.41	76.50	27.80	90.74	193.67
MPCM [ 9 ]	$\text{PR}_{16}$	22.90	79.12	135.53	45.18	90.74	26.25	22.90	79.12	135.53	15.60	58.62	185.54	15.60	58.62	185.54	45.18	90.74	26.25
RLCM [ 30 ]	$\text{GRSL}_{18}$	17.67	64.64	54.05	27.16	84.25	135.79	17.67	64.64	54.05	15.38	76.40	480.03	15.38	76.40	480.03	27.16	84.25	135.79
LIG [ 29 ]	$\text{IPT}_{18}$	11.67	81.14	624.59	30.10	88.88	77.39	11.67	81.14	624.59	28.29	78.41	170.65	28.29	78.41	170.65	30.10	88.88	77.39
ACMNet [ 18 ]	$\text{WACV}_{21}$	40.39	83.71	104.18	59.81	92.12	20.40	37.48	74.56	158.04	36.25	74.23	138.43	36.67	73.42	66.71	64.55	90.80	19.40

TABLE I: Additional evaluation of IRSTD performance in the single-domain generalization scenario in terms of IoU (%),

P_{d}

(%), and

F_{a}

(

10^{-6}

). The best results are highlighted in bold.

Method	Venue	NUAA-SIRST				NUDT-SIRST				IRSTD-1K
Method	Venue	IoU $\uparrow$	$F_{1}\uparrow$	$P_{d}\uparrow$	$F_{a}\downarrow$	IoU $\uparrow$	$F_{1}\uparrow$	$P_{d}\uparrow$	$F_{a}\downarrow$	IoU $\uparrow$	$F_{1}\uparrow$	$P_{d}\uparrow$	$F_{a}\downarrow$
Tophat [ 8 ]	$\text{OE}_{96}$	27.80	43.51	90.74	193.67	12.59	22.37	69.41	76.50	5.95	11.23	69.69	621.38
MPCM [ 9 ]	$\text{PR}_{16}$	45.18	62.24	90.74	26.25	15.60	26.98	58.62	185.54	22.90	37.27	79.12	135.53
RLCM [ 30 ]	$\text{GRSL}_{18}$	27.16	42.71	84.25	135.79	15.38	26.66	76.40	480.03	17.67	30.04	64.64	54.05
LIG [ 29 ]	$\text{IPT}_{18}$	30.10	46.27	88.88	77.39	28.29	36.20	78.41	170.65	11.67	20.90	81.14	624.59
TLLCM [ 31 ]	$\text{GRSL}_{19}$	17.92	30.40	84.25	20.04	11.77	21.07	72.59	151.83	11.28	27.67	81.81	212.67

TABLE II: Comparison of cross-domain ISTD performance in the multi-domain generalization scenario on NUAA-SIRST, NUDT-SIRST, and IRSTD-1K datasets in terms of IoU (%),

F_{1}

(%),

P_{d}

(%), and

F_{a}

(

10^{-6}

). The best results are highlighted in bold.

Panel of Figure 5 — Fig. 3: ROC curves of different methods when generalized to NUAA-SIRST, NUDT-SIRST, and IRSTD-1K datasets.

Fig. 4: Visualizations of cross-domain IRSTD results of different methods. Boxes in red, blue, and yellow refer to detected targets, miss detections, and false alarms. Zoomed-in regions of the detected targets are provided in the corner for clear visualization.

PRM	OAM	SSR	IoU $\uparrow$	$F_{1}\uparrow$	$P_{d}\uparrow$	$F_{a}\downarrow$
✘	✘	✘	49.42	66.14	86.73	111.67
✔	✘	✘	57.64	73.12	87.75	38.64
✘	✔	✘	51.93	68.35	89.45	92.01
✘	✘	✔	54.47	70.05	88.77	66.50
✘	✔	✔	55.62	71.47	85.71	63.69
✔	✔	✔	60.01	74.99	90.47	41.90

TABLE III: Ablation experiments about contributions of each component to cross-domain IRSTD. Best results are highlighted in bold.

Panel of Figure 8 — Fig. 5: Cross-domain IRSTD performance of SCTransNet combined with different DG approaches.

Fig. 6: Visualization of heatmaps during different decoding stages when applying CCA and OAM to UNet.

Panel of Figure 10 — Fig. 7: IRSTD performance on different datasets under domain-consistent setting.

Methods	Params	FLOPs	IoU $\uparrow$	$P_{d}\uparrow$	$F_{a}\downarrow$
MSHNet	4.1M	6.1G	54.91	86.85	55.12
S 2 CP-MSHNet	5.5M	6.7G	62.06	90.08	25.51
SCTransNet	11.2M	10.1G	53.98	84.81	56.15
S 2 CP-SCTransNet	12.3M	11.3G	62.62	90.51	24.82

TABLE IV: Analysis of computational efficiency.

Parameter	Setting	IoU $\uparrow$	$F_{1}\uparrow$	$P_{d}\uparrow$	$F_{a}\downarrow$
$\tau$	0.2	54.46	70.51	81.58	37.45
	0.3	55.35	71.26	82.75	30.21
	0.4	54.50	70.55	82.10	28.33
	0.5	53.17	69.43	80.52	38.28
$\lambda$	0.1	55.64	71.49	82.53	29.75
	0.2	56.25	72.01	84.76	29.82

TABLE V: Detection performance with different parameter settings of SSR. Best results are highlighted in bold.

Infrared Small Target Detection

Infrared small target detection finds faint, often sub-pixel objects against cluttered thermal backgrounds while limiting false alarms.

See all 43 papers

HyTBE: Hyperbolic Target-Background Expert Model for Cross-Domain Infrared Small Target Detection
Aug 6, 2026Aohua Li, Jin Kuang, Yubing Lu +1Infrared Small Target Detection Cross-Domain Generalization
College of Software, Jilin University, Changchun, Jilin 130000, China · Hunan Engineering Research Center of Advanced Embedded Computing and Intelligent Medical Systems, Xiangnan University, Chenzhou, 423300, China · College of Computer Science and Technology, Jilin University, Changchun, Jilin 130012, China +1
FSGNet: A Frequency-Aware and Semantic Guidance Network for Infrared Small Target Detection
Mar 26, 2026Yingmei Zhang, Wangtao Bao, Yong Yang +3Infrared Small Target Detection Object Detection
School of Software and Internet of Things Engineering, Jiangxi University of Finance and Economics, Nanchang 330032, China · the postdoctoral researcher, CGN Begood Technology co., Ltd, Nanchang 330032, China · School of Computer Science and Technology, Tiangong University, Tianjin 300387, China +1
Understand Before Detect: Vision--Language Learning for Omni-Domain Infrared Small Target Detection
Aug 7, 2026Haoyang Yuan, Boyang Li, Yingqian Wang +7Infrared Small Target Detection Domain Generalization
National University of Defense Technology, Changsha, China · Peking University, Beijing, China

Aug 6, 2026J/K move · Enter open · S save

HyTBE: Hyperbolic Target-Background Expert Model for Cross-Domain Infrared Small Target Detection

Aohua Li, Jin Kuang, Yubing Lu, Pingping Liu

College of Software, Jilin University, Changchun, Jilin 130000, China · Hunan Engineering Research Center of Advanced Embedded Computing and Intelligent Medical Systems, Xiangnan University, Chenzhou, 423300, China · College of Computer Science and Technology, Jilin University, Changchun, Jilin 130012, China+1

Infrared Small Target Detection Cross-Domain Generalization

Open paper AskPDF arXiv

Infrared small target detection (IRSTD) has achieved substantial progress under domain-consistent evaluation, yet detector performance often degrades markedly when generalizing to unseen infrared domains. Existing methods primarily improve detection by enhancing target responses and suppressing background interference. However, when trained on only a limited set of source domains, their learned decision rules are inevitably established from a restricted range of source-domain target-background relation patterns. We formulate this cross-domain failure as target-background relation shift: unseen domains may exhibit relation patterns that are not observed during training, thereby weakening the discriminative capability learned from the source domains. To address this problem, we propose HyTBE, a Hyperbolic Target-Background Expert model that expands source-domain relation patterns and adaptively adjusts visual representations using explicit relation cues. The Target-Background Relation Intervention selectively perturbs either targets or backgrounds, broadening the observable relation patterns during training while maintaining valid supervision. Subsequently, the Hyperbolic Relation Modeling maps multi-scale visual cues into a Poincaré ball and characterizes the target-background relation of each feature token according to its relative distances to the target and background anchors. The Hyperbolic-guided MoE Adapter further uses these hyperbolic relation representations to calibrate multi-scale visual features and aggregate expert-specific feature corrections for different relation patterns. Leave-one-domain-out experiments on NUAA-SIRST, NUDT-SIRST, and IRSTD-1K demonstrate that HyTBE achieves stronger cross-domain generalization than competitive baselines.

Abstract

Infrared Small Target Detection

Explore similar work

HyTBE: Hyperbolic Target-Background Expert Model for Cross-Domain Infrared Small Target Detection

FSGNet: A Frequency-Aware and Semantic Guidance Network for Infrared Small Target Detection

Understand Before Detect: Vision--Language Learning for Omni-Domain Infrared Small Target Detection