cs.CVMar 26, 2026

Knowledge-Guided Adversarial Training for Infrared Object Detection via Thermal Radiation Modeling

Authors: Shiji Zhao, Shukun Xiong, Maoxun Yuan, Yao Huang, Ranjie Duan, Qing Guo, Jiansheng Chen, Haibin Duan, 

Organizations: Institute of Artificial Intelligence, Beihang University, No.37 Xueyuan Road, Haidian District, Beijing 100191, P.R. China. · Security Department, Alibaba Group, Hangzhou 310056, China. · School of Computer Science, Nankai University, China. · School of Computer and Communication Engineering, University of Science and Technology Beijing, Beijing 100083, China. · 

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

In complex environments, infrared object detection exhibits broad applicability and stability across diverse scenarios. However, infrared object detection is vulnerable to both common corruptions and adversarial examples, leading to potential security risks. To improve the robustness of infrared object detection, current methods mostly adopt a data-driven ideology, which only superficially drives the network to fit the training data without specifically considering the unique characteristics of infrared images, resulting in limited robustness. In this paper, we revisit infrared physical knowledge and find that relative thermal radiation relations between different classes can be regarded as a reliable knowledge source under the complex scenarios of adversarial examples and common corruptions. Thus, we theoretically model thermal radiation relations based on the rank order of gray values for different classes, and further quantify the stability of various inter-class thermal radiation relations. Based on the above theoretical framework, we propose Knowledge-Guided Adversarial Training (KGAT) for infrared object detection, in which infrared physical knowledge is embedded into the adversarial training process, and the predicted results are optimized to be consistent with the actual physical laws. Extensive experiments on three infrared datasets and six mainstream infrared object detection models demonstrate that KGAT effectively enhances both clean accuracy and robustness against adversarial attacks and common corruptions.

Figures & tables

Figure 1: Performance of the baseline and our knowledge-guided adversarial training (KGAT) on M 3 FD dataset, using YOLO-v8 as the detector. Compared with Data-driven method, KGAT can enhance the robustness of infrared object detection towards more diverse perturbations in complex infrared environments.
Figure 2: The advantage of relative thermal radiation relations. In the M 3 FD dataset, under environmental conditions such as Clear Day, Cloudy Overcast, and Clear Dusk, although the absolute gray values of the “car” and the “people” are different, most of the relative thermal radiation relations between “car” and the “people” keep stable.
Figure 3: The Framework of our Knowledge-Guided Adversarial Training. During the knowledge extraction process , we first calculate the rank order of the gray value between different classes to extract the thermal radiation relation. We further calculate the stability of the thermal radiation relation between all the different classes. During the training process , we apply the thermal radiation relations and the corresponding variations to adjust the learning weights of each training sample so that the prediction results conform to the real-world infrared knowledge.
ModelMethodCleanA clsA locA mtdCWADAGBlurSaltGNAvg
SSDSTD71.3‾↓18.6{\underline{71.3}\phantom{{\downarrow}{18.6}}}9.8↓18.6{9.8}\phantom{{\downarrow}{18.6}}9.2↓18.6{9.2}\phantom{{\downarrow}{18.6}}8.9↓18.6{8.9}\phantom{{\downarrow}{18.6}}8.4↓18.6{8.4}\phantom{{\downarrow}{18.6}}10.5↓18.6{10.5}\phantom{{\downarrow}{18.6}}47.6↓18.6{47.6}\phantom{{\downarrow}{18.6}}50.1↓18.6{50.1}\phantom{{\downarrow}{18.6}}40.9↓18.6{40.9}\phantom{{\downarrow}{18.6}}24.9↓18.6{24.9}\phantom{{\downarrow}{18.6}}
LOC ( Zhang and Wang, 2019 )47.9↓18.6{47.9}\phantom{{\downarrow}{18.6}}29.3↓18.6{29.3}\phantom{{\downarrow}{18.6}}29.7↓18.6{29.7}\phantom{{\downarrow}{18.6}}29.0↓18.6{29.0}\phantom{{\downarrow}{18.6}}26.7↓18.6{26.7}\phantom{{\downarrow}{18.6}}38.3↓18.6{38.3}\phantom{{\downarrow}{18.6}}33.3↓18.6{33.3}\phantom{{\downarrow}{18.6}}36.8↓18.6{36.8}\phantom{{\downarrow}{18.6}}31.1↓18.6{31.1}\phantom{{\downarrow}{18.6}}33.6↓18.6{33.6}\phantom{{\downarrow}{18.6}}
CLS ( Zhang and Wang, 2019 )49.2↓18.6{49.2}\phantom{{\downarrow}{18.6}}34.9↓18.6{34.9}\phantom{{\downarrow}{18.6}}33.4↓18.6{33.4}\phantom{{\downarrow}{18.6}}35.9↓18.6{35.9}\phantom{{\downarrow}{18.6}}34.0↓18.6{34.0}\phantom{{\downarrow}{18.6}}38.9↓18.6{38.9}\phantom{{\downarrow}{18.6}}36.2↓18.6{36.2}\phantom{{\downarrow}{18.6}}38.3↓18.6{38.3}\phantom{{\downarrow}{18.6}}32.4↓18.6{32.4}\phantom{{\downarrow}{18.6}}37.0↓18.6{37.0}\phantom{{\downarrow}{18.6}}
MTD ( Zhang and Wang, 2019 )50.3↓18.6{50.3}\phantom{{\downarrow}{18.6}}35.2↓18.6{35.2}\phantom{{\downarrow}{18.6}}33.8↓18.6{33.8}\phantom{{\downarrow}{18.6}}34.9↓18.6{34.9}\phantom{{\downarrow}{18.6}}32.4↓18.6{32.4}\phantom{{\downarrow}{18.6}}39.8↓18.6{39.8}\phantom{{\downarrow}{18.6}}37.6↓18.6{37.6}\phantom{{\downarrow}{18.6}}38.8↓18.6{38.8}\phantom{{\downarrow}{18.6}}32.5↓18.6{32.5}\phantom{{\downarrow}{18.6}}37.3↓18.6{37.3}\phantom{{\downarrow}{18.6}}
CWAT ( Chen et al., 2021a )51.1↓18.6{51.1}\phantom{{\downarrow}{18.6}}36.3↓18.6{36.3}\phantom{{\downarrow}{18.6}}36.0↓18.6{36.0}\phantom{{\downarrow}{18.6}}36.7↓18.6{36.7}\phantom{{\downarrow}{18.6}}34.1↓18.6{34.1}\phantom{{\downarrow}{18.6}}46.3↓18.6{46.3}\phantom{{\downarrow}{18.6}}37.8↓18.6{37.8}\phantom{{\downarrow}{18.6}}39.9↓18.6{39.9}\phantom{{\downarrow}{18.6}}34.2↓18.6{34.2}\phantom{{\downarrow}{18.6}}39.2↓18.6{39.2}\phantom{{\downarrow}{18.6}}
Det-AdvProp ( Chen et al., 2021b )64.0↓18.6{64.0}\phantom{{\downarrow}{18.6}}34.8↓18.6{34.8}\phantom{{\downarrow}{18.6}}35.5↓18.6{35.5}\phantom{{\downarrow}{18.6}}36.1↓18.6{36.1}\phantom{{\downarrow}{18.6}}33.8↓18.6{33.8}\phantom{{\downarrow}{18.6}}44.4↓18.6{44.4}\phantom{{\downarrow}{18.6}}39.8↓18.6{39.8}\phantom{{\downarrow}{18.6}}42.4↓18.6{42.4}\phantom{{\downarrow}{18.6}}37.4↓18.6{37.4}\phantom{{\downarrow}{18.6}}40.9↓18.6{40.9}\phantom{{\downarrow}{18.6}}
Table 1: Robustness performance of different models under various adversarial attacks and common corruptions on M 3 FD. All results are based on the best checkpoints. GN denotes Gaussian Noise, and Avg denotes the average value of the overall mAP 50 . {\color[rgb]{0,0.5,0}\downarrow} and {\color[rgb]{1,0,0}\uparrow} indicate the mAP 50 decrease or increase for the current version of the KGAT method compared to baseline, respectively.
ModelMethodCleanA clsA locA mtdCWADAGBlurSaltGNAvg
SSDSTD61.9‾↓18.6\underline{61.9}\phantom{{\downarrow}{18.6}}8.8↓18.6{8.8}\phantom{{\downarrow}{18.6}}8.5↓18.6{8.5}\phantom{{\downarrow}{18.6}}7.9↓18.6{7.9}\phantom{{\downarrow}{18.6}}5.6↓18.6{5.6}\phantom{{\downarrow}{18.6}}10.3↓18.6{10.3}\phantom{{\downarrow}{18.6}}44.1↓18.6{44.1}\phantom{{\downarrow}{18.6}}47.6↓18.6{47.6}\phantom{{\downarrow}{18.6}}38.9↓18.6{38.9}\phantom{{\downarrow}{18.6}}22.7↓18.6{22.7}\phantom{{\downarrow}{18.6}}
LOC ( Zhang and Wang, 2019 )36.6↓18.6{36.6}\phantom{{\downarrow}{18.6}}26.6↓18.6{26.6}\phantom{{\downarrow}{18.6}}27.2↓18.6{27.2}\phantom{{\downarrow}{18.6}}26.9↓18.6{26.9}\phantom{{\downarrow}{18.6}}21.5↓18.6{21.5}\phantom{{\downarrow}{18.6}}29.2↓18.6{29.2}\phantom{{\downarrow}{18.6}}30.6↓18.6{30.6}\phantom{{\downarrow}{18.6}}34.4↓18.6{34.4}\phantom{{\downarrow}{18.6}}28.5↓18.6{28.5}\phantom{{\downarrow}{18.6}}29.1↓18.6{29.1}\phantom{{\downarrow}{18.6}}
CLS ( Zhang and Wang, 2019 )38.1↓18.6{38.1}\phantom{{\downarrow}{18.6}}28.3↓18.6{28.3}\phantom{{\downarrow}{18.6}}28.7↓18.6{28.7}\phantom{{\downarrow}{18.6}}28.4↓18.6{28.4}\phantom{{\downarrow}{18.6}}23.2↓18.6{23.2}\phantom{{\downarrow}{18.6}}30.6↓18.6{30.6}\phantom{{\downarrow}{18.6}}33.1↓18.6{33.1}\phantom{{\downarrow}{18.6}}35.4↓18.6{35.4}\phantom{{\downarrow}{18.6}}30.7↓18.6{30.7}\phantom{{\downarrow}{18.6}}30.8↓18.6{30.8}\phantom{{\downarrow}{18.6}}
MTD ( Zhang and Wang, 2019 )37.6↓18.6{37.6}\phantom{{\downarrow}{18.6}}28.3↓18.6{28.3}\phantom{{\downarrow}{18.6}}28.5↓18.6{28.5}\phantom{{\downarrow}{18.6}}28.4↓18.6{28.4}\phantom{{\downarrow}{18.6}}23.7↓18.6{23.7}\phantom{{\downarrow}{18.6}}27.7↓18.6{27.7}\phantom{{\downarrow}{18.6}}35.6↓18.6{35.6}\phantom{{\downarrow}{18.6}}35.3↓18.6{35.3}\phantom{{\downarrow}{18.6}}31.0↓18.6{31.0}\phantom{{\downarrow}{18.6}}30.7↓18.6{30.7}\phantom{{\downarrow}{18.6}}
CWAT ( Chen et al., 2021a )40.5↓18.6{40.5}\phantom{{\downarrow}{18.6}}33.4↓18.6{33.4}\phantom{{\downarrow}{18.6}}30.4↓18.6{30.4}\phantom{{\downarrow}{18.6}}32.7↓18.6{32.7}\phantom{{\downarrow}{18.6}}28.1↓18.6{28.1}\phantom{{\downarrow}{18.6}}34.3↓18.6{34.3}\phantom{{\downarrow}{18.6}}35.4↓18.6{35.4}\phantom{{\downarrow}{18.6}}37.6↓18.6{37.6}\phantom{{\downarrow}{18.6}}31.4↓18.6{31.4}\phantom{{\downarrow}{18.6}}33.8↓18.6{33.8}\phantom{{\downarrow}{18.6}}
Det-AdvProp ( Chen et al., 2021b )51.7↓18.6{51.7}\phantom{{\downarrow}{18.6}}32.4↓18.6{32.4}\phantom{{\downarrow}{18.6}}29.4↓18.6{29.4}\phantom{{\downarrow}{18.6}}31.1↓18.6{31.1}\phantom{{\downarrow}{18.6}}27.0↓18.6{27.0}\phantom{{\downarrow}{18.6}}34.5↓18.6{34.5}\phantom{{\downarrow}{18.6}}36.0↓18.6{36.0}\phantom{{\downarrow}{18.6}}38.4↓18.6{38.4}\phantom{{\downarrow}{18.6}}35.2↓18.6{35.2}\phantom{{\downarrow}{18.6}}35.1↓18.6{35.1}\phantom{{\downarrow}{18.6}}
Table 2: Robustness performance of different models under various adversarial attacks and common corruptions on FLIR-ADAS. All results are based on the best checkpoints. GN denotes Gaussian Noise, and Avg denotes the average value of the overall mAP 50 . {\color[rgb]{0,0.5,0}\downarrow} and {\color[rgb]{1,0,0}\uparrow} indicate the mAP 50 decrease or increase for the current version of the KGAT method compared to baseline, respectively.
ModelMethodCleanA clsA locA mtdCWADAGBlurSaltGNAvg
SSDSTD62.2‾↓18.6\underline{62.2}\phantom{{\downarrow}{18.6}}8.4↓18.6{8.4}\phantom{{\downarrow}{18.6}}8.2↓18.6{8.2}\phantom{{\downarrow}{18.6}}8.1↓18.6{8.1}\phantom{{\downarrow}{18.6}}7.7↓18.6{7.7}\phantom{{\downarrow}{18.6}}7.9↓18.6{7.9}\phantom{{\downarrow}{18.6}}43.549.5‾↓18.6\underline{49.5}\phantom{{\downarrow}{18.6}}41.1↓18.6{41.1}\phantom{{\downarrow}{18.6}}24.1↓18.6{24.1}\phantom{{\downarrow}{18.6}}
LOC ( Zhang and Wang, 2019 )39.9↓18.6{39.9}\phantom{{\downarrow}{18.6}}28.3↓18.6{28.3}\phantom{{\downarrow}{18.6}}29.8↓18.6{29.8}\phantom{{\downarrow}{18.6}}28.0↓18.6{28.0}\phantom{{\downarrow}{18.6}}27.5↓18.6{27.5}\phantom{{\downarrow}{18.6}}33.0↓18.6{33.0}\phantom{{\downarrow}{18.6}}29.2↓18.6{29.2}\phantom{{\downarrow}{18.6}}30.1↓18.6{30.1}\phantom{{\downarrow}{18.6}}27.3↓18.6{27.3}\phantom{{\downarrow}{18.6}}30.3↓18.6{30.3}\phantom{{\downarrow}{18.6}}
CLS ( Zhang and Wang, 2019 )43.6↓18.6{43.6}\phantom{{\downarrow}{18.6}}30.6↓18.6{30.6}\phantom{{\downarrow}{18.6}}31.1↓18.6{31.1}\phantom{{\downarrow}{18.6}}28.8↓18.6{28.8}\phantom{{\downarrow}{18.6}}28.7↓18.6{28.7}\phantom{{\downarrow}{18.6}}32.4↓18.6{32.4}\phantom{{\downarrow}{18.6}}30.1↓18.6{30.1}\phantom{{\downarrow}{18.6}}31.6↓18.6{31.6}\phantom{{\downarrow}{18.6}}27.4↓18.6{27.4}\phantom{{\downarrow}{18.6}}31.6↓18.6{31.6}\phantom{{\downarrow}{18.6}}
MTD ( Zhang and Wang, 2019 )44.9↓18.6{44.9}\phantom{{\downarrow}{18.6}}31.1↓18.6{31.1}\phantom{{\downarrow}{18.6}}31.3↓18.6{31.3}\phantom{{\downarrow}{18.6}}29.6↓18.6{29.6}\phantom{{\downarrow}{18.6}}29.6↓18.6{29.6}\phantom{{\downarrow}{18.6}}33.6↓18.6{33.6}\phantom{{\downarrow}{18.6}}30.5↓18.6{30.5}\phantom{{\downarrow}{18.6}}31.7↓18.6{31.7}\phantom{{\downarrow}{18.6}}28.7↓18.6{28.7}\phantom{{\downarrow}{18.6}}32.3↓18.6{32.3}\phantom{{\downarrow}{18.6}}
CWAT ( Chen et al., 2021a )44.5↓18.6{44.5}\phantom{{\downarrow}{18.6}}31.4↓18.6{31.4}\phantom{{\downarrow}{18.6}}32.0↓18.6{32.0}\phantom{{\downarrow}{18.6}}31.2↓18.6{31.2}\phantom{{\downarrow}{18.6}}30.6↓18.6{30.6}\phantom{{\downarrow}{18.6}}38.4↓18.6{38.4}\phantom{{\downarrow}{18.6}}34.8↓18.6{34.8}\phantom{{\downarrow}{18.6}}31.7↓18.6{31.7}\phantom{{\downarrow}{18.6}}28.2↓18.6{28.2}\phantom{{\downarrow}{18.6}}33.6↓18.6{33.6}\phantom{{\downarrow}{18.6}}
Det-AdvProp ( Chen et al., 2021b )47.4↓18.6{47.4}\phantom{{\downarrow}{18.6}}30.5↓18.6{30.5}\phantom{{\downarrow}{18.6}}31.6↓18.6{31.6}\phantom{{\downarrow}{18.6}}30.0↓18.6{30.0}\phantom{{\downarrow}{18.6}}29.8↓18.6{29.8}\phantom{{\downarrow}{18.6}}38.8↓18.6{38.8}\phantom{{\downarrow}{18.6}}37.4↓18.6{37.4}\phantom{{\downarrow}{18.6}}34.6↓18.6{34.6}\phantom{{\downarrow}{18.6}}30.7↓18.6{30.7}\phantom{{\downarrow}{18.6}}34.5↓18.6{34.5}\phantom{{\downarrow}{18.6}}
Table 3: Robustness performance of different models under various adversarial attacks and common corruptions on KAIST. All results are based on the best checkpoints. GN denotes Gaussian Noise, and Avg denotes the average value of the overall mAP 50 . {\color[rgb]{0,0.5,0}\downarrow} and {\color[rgb]{1,0,0}\uparrow} indicate the mAP 50 decrease or increase for the current version of the KGAT method compared to baseline, respectively.
Figure 4: Visualization of the prediction results between baseline (SALC) and our SALC+KGAT of YOLO-v8 on M 3 FD under three different attacks.
ModelMethod0 CleanA clsA locA mtdCWADAGBlurSaltGNAvg
EFLNetSTD0 88.18.9{8.9}7.3{7.3}6.4{6.4}12.7{12.7}2.1{2.1}33.0{33.0}53.5{53.5}65.930.9{30.9}
SALC0 80.2{80.2}65.2{65.2}66.8{66.8}63.6{63.6}60.3{60.3}61.6{61.6}46.3{46.3}54.3{54.3}64.3{64.3}62.5{62.5}
SALC+KGAT(ours)0 82.6{82.6}67.870.166.164.565.350.058.265.4{65.4}65.6
InfMAESTD0 84.711.3{11.3}11.0{11.0}12.7{12.7}12.4{12.4}17.5{17.5}58.8{58.8}49.8{49.8}44.8{44.8}33.7{33.7}
SALC0 76.3{76.3}68.3{68.3}66.9{66.9}67.1{67.1}61.0{61.0}70.9{70.9}69.1{69.1}50.8{50.8}46.7{46.7}64.2{64.2}
SALC+KGAT(ours)0 80.4{80.4}72.470.968.964.675.164.354.450.966.9
Table 4: Robustness performance of Specialized infrared detection architectures under various adversarial attacks and common corruptions on M 3 FD. All results are based on the best checkpoints. GN denotes Gaussian Noise, and Avg denotes the average value of the overall mAP 50 .
 MTDSALCRobustDetwiρw_{i}^{\rho}wiSw_{i}^{\mathcal{S}}CleanA clsA locA mtdCWADAGBlurSaltGN
YOLO-v8✓\checkmark    65.857.256.355.954.556.141.051.746.8
✓\checkmark  ✓\checkmark 69.467.863.867.164.266.546.457.153.3
✓\checkmark  ✓\checkmark✓\checkmark72.268.467.768.866.567.147.460.255.9
 ✓\checkmark   71.262.160.761.359.860.143.955.151.2
 ✓\checkmark ✓\checkmark 72.068.564.268.166.969.851.663.458.9
 ✓\checkmark ✓\checkmark✓\checkmark73.970.667.169.967.571.252.465.260.7
Table 5: Ablation Study towards the key components of our KGAT methods. The results are based on the M 3 FD of YOLO-v8 and SSD. The best result is highlighted in bold, while the second-best result is marked with an underline.
Figure 5: The robustness of YOLO-v8 based SALC and SALC+KGAT (ours) against A mtd attack on images with thermal radiation relations of different stability. In the x-axis labels, C, P, B, M, L, and T denote Cars, People, Bus, Motorcycle, Lamp, and Truck, respectively. The x-axis denotes the specific thermal radiation relation, and the bars indicate the stability level of that relation, using the left y-axis. The two curves represent the mAP 50 of SALC and SALC+KGAT (ours) on images containing this specific thermal radiation relation, respectively, using the right y-axis.
Figure 6: The robustness of YOLO-v8 against the AIP attack, with Average Precision (AP) plotted on the right Y-axis and Attack Success Rate (ASR) plotted on the left Y-axis. The KGAT is SALC+KGAT version.
MethodmAP 75mAP 50:95
CleanAmtdA_{mtd}GNCleanAmtdA_{mtd}GN
STD56.37.634.454.17.433.0
SALC54.141.936.448.039.033.2
SALC+KGAT57.553.047.153.248.943.7
Table 6: Detection Performance across different IoU Thresholds, including mAP 75 and mAP 50:95 . The results are based on the best checkpoints on M3FDM^{3}FD of YOLO-v8.
TpyesCleanAclsA_{cls}AlocA_{loc}AmtdA_{mtd}DAGBlurSaltGN
Baseline71.262.160.761.360.143.955.151.2
Linear71.469.468.567.569.250.564.560.1
Exp-based73.370.367.669.370.851.664.560.2
Log-based73.970.667.169.971.252.465.260.7
Table 7: Ablation study towards different mapping operations. The results of linear and nonlinear mapping are based on the best checkpoints on M3FDM^{3}FD of YOLO-v8 based on SALC. The best result is highlighted in bold.
β\betaCleanA clsA locA mtdDAGBlurSaltGN
β=0.3\beta=0.370.868.267.365.769.850.664.558.1
β=0.4\beta=0.471.769.367.666.570.450.764.659.8
β=0.5\beta=0.573.970.667.169.971.252.465.260.7
β=0.6\beta=0.672.869.467.568.870.451.965.059.8
β=0.7\beta=0.771.770.465.668.170.951.664.758.1
Table 8: Ablation Study towards the hyper-parameter β\beta . The results are based on the M 3 FD of YOLO-v8. The best result is highlighted in bold.
γ\gammaCleanA clsA locA mtdDAGBlurSaltGN
γ=0.7\gamma=0.769.567.464.867.469.747.859.558.1
γ=0.9\gamma=0.972.669.368.569.370.452.362.660.4
γ=1.1\gamma=1.173.970.667.169.971.252.465.260.7
γ=1.3\gamma=1.372.870.367.668.870.654.064.659.9
γ=1.5\gamma=1.571.269.467.069.070.151.665.059.0
Table 9: Ablation Study towards the hyper-parameter γ\gamma . The results are based on the M 3 FD of YOLO-v8. All attack settings are the same as in the previous experiments. The best result is highlighted in bold.
Noise Level10%20%30%
MethodCleanA mtdGNCleanA mtdGNCleanA mtdGN
STD75.19.151.572.98.044.369.86.135.7
SALC69.356.350.867.152.742.163.949.334.8
SALC+KGAT (ours)72.164.358.370.262.352.167.859.344.3
Table 10: Robustness comparison under imprecise bounding boxes. We report results under three noise levels (10%,20%,30%) and GN denotes Gaussian Noise.
MethodsTime(s)GFLOPsMemory(GB)
MTD306.98126.8822.30
CWAT362.36131.2622.68
Det-AdvProp342.26132.2723.58
SALC313.31129.0722.69
MTD+KGAT(ours)316.23128.0522.36
SALC+KGAT(ours)325.46130.2422.74
Table 11: Computational Overhead. Average per-epoch training time with overall GFLOPs and GPU memory.
More in this topic

Infrared Object Detection

Infrared object detection locates and classifies objects in thermal imagery, including small, low-contrast targets and visible–thermal scenes.

See all 33 papers

Explore similar work

CardsList
May 18, 2026

Unleashing the Representational Power of Fourier Shapes for Attacking Infrared Object Detection

Infrared object detection is crucial for perception in autonomous driving and surveillance but remains vulnerable to physical adversarial attacks. Unlike in the RGB domain, where attacks rely on color texture, infrared attacks must manipulate thermal signatures, making the geometry shape of heat-blocking materials the primary adversarial information carrier. Current shape-based methods suffer from a fundamental trade-off between representational capability and optimization power, limiting their attack effectiveness.In this work, we overcome this dilemma by introducing learnable Fourier shapes to the infrared domain. We utilize an end-to-end differentiable framework where a compact set of Fourier coefficients, defining the shape boundary, is analytically mapped to a pixel-space mask via the winding number theorem. This enables efficient gradient-based optimization to generate potent shapes that cause human targets to evade detection. Extensive digital and physical experiments provide a comprehensive evaluation and validate our superior performance. Our resulting physical patch achieves striking robustness, successfully evading detectors across diverse distances, angles, poses, and individuals, and achieves over 88% attack success rate at distances greater than 25m (conf.=0.5). Code is available at https://github.com/Yongyx99/Fourier-shape-attack.

Yixing Yong, Jian Wang, Ming Lei +2

School of Information and Communications Engineering, Faculty of Electronic and Information Engineering, Xi’an Jiao-tong University, Xi’an, China · School of Physics, Xi’an Jiao-tong University, Xi’an, China.

Adversarial AttacksInfrared Object DetectionPhysical Adversarial Attacks
AskPDF
Mar 23, 2026

Thermal Topology Collapse: Universal Physical Patch Attacks on Infrared Vision Systems

Infrared pedestrian detectors are increasingly deployed in all-weather perception systems, but their robustness against physical adversarial attacks remains insufficiently understood. Existing infrared physical attacks are mostly instance-specific, requiring perturbations to be optimized for particular samples, poses, or scenes, which limits scalability under changing deployment conditions. This paper proposes Universal Physical Patch Attack (UPPA), a universal cold-patch framework for infrared pedestrian detection. UPPA is built on the observation that infrared physical perturbations should exploit smooth, low-frequency thermal structures rather than visible-light texture patterns. It represents the attack carrier as topology-constrained Bézier Curved-Blocks, providing a compact and manufacturable geometric parameterization, and optimizes one shared perturbation with Particle Swarm Optimization (PSO) under Thin Plate Spline (TPS) deformation and Expectation over Transformation (EOT) imaging transformations. The optimized pattern is then deployed as wearable cold patches without sample-specific re-optimization during deployment. Experiments on five infrared datasets and nine pedestrian detectors show consistent digital attack performance, strong cross-dataset and cross-model transferability, and a 92.59% attack success rate in real-world physical experiments. Ablation, visualization, and defense analyses show that Curved-Blocks disrupt pedestrian thermal feature aggregation and remain difficult for image-restoration-style defenses to remove. These results reveal a practical universal physical vulnerability in current infrared pedestrian detection systems and provide a benchmark for robustness evaluation.

Chengyin Hu, Yikun Guo, Yuxian Dong +7

China University of Petroleum-Beijing at Karamay, No. 355, Anding Road, Karamay, 834000, Xinjiang, China. · Guangdong University of Technology, Guangzhou, Guangdong, China. · Tianjin University, Tianjin, China. +1

Adversarial Patch AttacksPhysical Adversarial AttacksAdversarial PerturbationsAdversarial RobustnessInfrared Object Detection
AskPDF
Aug 31, 2026

Physical Adversarial Examples for Person Detectors in Thermal Images Based on 3D Modeling

Thermal Infrared detection is widely used in autonomous driving, medical AI, etc., but its security has only attracted attention recently. We propose infrared adversarial clothing designed to evade thermal person detectors in real-world scenarios. The design of the adversarial clothing is based on 3D modeling, which makes it easier to simulate multiangle scenes near the real world compared to 2D modeling. We optimized the black patch layout pattern of 3D clothing based on the adversarial example technique and made physical adversarial clothing using the aerogel. The idea is to paste a set of square aerogel patches, which display black squares in thermal images, in the inner side of clothing at specific locations with specific orientations. To enhance realism, we propose a method to build infrared 3D models with real infrared photos and develop texture maps for 3D models to simulate varied infrared characteristics over time and location. In physical attacks, we achieved an attack success rate of 80.11% indoors and 76.85% outdoors against YOLOv9. In contrast, randomly placed patches yielded much lower success rates (26.53% indoors and 23.03% outdoors). The adversarial clothing also showed good transferability to unknown detectors with an ensemble attack method, demonstrating the effectiveness of our approach.

Xiaopei Zhu, Siyuan Huang, Zhanhao Hu +3

Department of Computer Science and Technology, Institute for Artificial Intelligence, BNRist, Tsinghua University, Beijing 100084, China · Chinese Institute for Brain Research (CIBR), Beijing 100010, China

Thermal ImagingAdversarial Patch AttacksInfrared Object DetectionPhysical Adversarial Attacks
AskPDF