Text-Guided Diffusion-Based Adversarial Attacks on Chest X-Ray Images
Authors: Basudha Pal, Arjun Narayanan, Neha Ajith, Vikas R Bhat, Muhammad Umair
Organizations: Department of Electrical and Computer Engineering, The Johns Hopkins University, Baltimore, 21218, MD, USA · Manipal Institute of Technology, Manipal Academy of Higher Education, Manipal, 576104, Karnataka, India · The Russell H. Morgan Department of Radiology and Radiological Sciences, The Johns Hopkins Hospital, Baltimore, 21287, MD, USA · Department of Radiology, Columbia University Irving Medical Center, New York, 10032, NY, USA
As artificial intelligence is increasingly integrated into chest X-ray (CXR) interpretation, triage, and clinical decision support, understanding its vulnerability to adversarial manipulation is critical for safe deployment. Existing robustness evaluations, however, predominantly rely on pixel-space attacks that introduce numerically constrained perturbations but may not represent plausible radiographic variation. This limitation is particularly important in multi-disease CXR classification, where models simultaneously evaluate multiple overlapping pathologies and adversarial failures may alter several diagnostic predictions. We propose a text-guided diffusion-based adversarial framework that optimizes learnable text conditioning while keeping the diffusion generator and target classifier frozen, enabling adversarial generation through a learned image prior rather than direct pixel manipulation. We evaluate the framework across multiple classifier architectures in both binary atelectasis and multi-disease CXR classification and compare it with FGSM, PGD, and Carlini-Wagner attacks. Our approach consistently produced the greatest degradation in classifier performance, reducing AUROC to 0.3885-0.5646 in binary classification and 0.4441-0.4878 in the multi-disease setting, while achieving superior image fidelity (SSIM 0.9080, LPIPS 0.1670, FID 51.23). Importantly, clinician interpretation remained unchanged for 95.9% of binary and 73.8% of multi-disease adversarial images despite substantial changes in model predictions. These findings reveal a clinically important discrepancy between human and machine interpretation and demonstrate the need to extend medical AI robustness evaluation beyond conventional pixel-space attacks toward generative threat models that can expose failures under visually and clinically plausible image variations.
Figures & tables
Figure 1: Basic architecture diagram of the proposed diffusion-guided adversarial framework. A learnable placeholder token is passed through a trainable text encoder to generate conditioning embeddings. These embeddings guide a frozen diffusion generator that reconstructs a chest X-ray image. The generated image is evaluated by a frozen disease classifier. Gradients are propagated only through the text encoder. The diagram illustrates the binary setting; the formulation extends naturally to multi-class and multi-label classification.
Task
Attack
DenseNet
ResNet
EfficientNet
Binary Atelectasis
Clean
0.7931
0.7561
0.6777
FGSM
0.4986
0.7327
0.6494
PGD
0.4531
0.7451
0.6625
CW
0.4496
0.6894
0.5337
Ours
0.3885
0.5646
0.4032
Multiclass
Clean
0.8319
0.7810
0.7989
Table 1: AUROC of chest X-ray classifiers under different adversarial attacks for binary atelectasis detection and multiclass disease classification. Lower AUROC indicates greater degradation in diagnostic performance.
Attack
SSIM ↑
LPIPS ↓
FID ↓
FGSM
0.5160
0.8054
145.55
PGD
0.8124
0.6083
86.24
CW
0.8488
0.2397
62.58
Ours
0.9080
0.1670
51.23
Table 2: Perceptual quality of adversarial chest X-ray images averaged across tasks. Higher SSIM indicates greater structural similarity to the original image, whereas lower LPIPS and FID correspond to improved perceptual realism.
Figure 2: Representative adversarial chest X-rays from the multiclass classification setting. Each row corresponds to one input radiograph and shows the original image together with adversarial examples generated using FGSM, PGD, and the proposed diffusion-based method. FGSM introduces pronounced high-frequency perturbations throughout the image, while PGD produces weaker but still perceptible perturbation patterns. In contrast, the diffusion-based method preserves the overall radiographic appearance and major anatomical structures of the source image while altering classifier predictions.
Figure 3: Representative adversarial chest X-rays from the binary atelectasis classification setting. Each row shows an original radiograph and corresponding FGSM-, PGD-, and diffusion-generated adversarial images. Conventional pixel-space attacks introduce visible image-wide perturbations, particularly for FGSM, whereas diffusion-generated images remain visually similar to the source radiographs, with preservation of major thoracic structures and substantially less apparent distortion.
Defense Strategy
FGSM
PGD
CW
Ours
None
0.5232
0.4959
0.5080
0.4441
Gaussian Smoothing
0.5360
0.5118
0.5120
0.4578
JPEG Compression
0.5246
0.5329
0.5247
0.4646
Adversarial Training (50%)
0.5681
0.5769
0.5749
0.4937
Adversarial Training (75%)
0.5813
0.5985
0.5991
0.5362
Table 3: AUROC after applying different defense strategies against adversarial attacks.
Figure 4: Clinician assessment of diffusion-generated adversarial chest radiographs. Percentage of cases exhibiting unchanged or changed clinical interpretation following adversarial perturbation. The high proportion of unchanged assessments indicates that the generated adversarial modifications largely preserved clinician-perceived diagnostic content.