PCaPaint: Prostate Cancer Inpainting by Mitigating Shortcut Learning
Organizations: GE HealthCare
Abstract
The development of AI systems for tumor-specific applications is limited by the scarcity of labeled data. Synthetic tumor inpainting offers a promising approach but faces challenges for prostate cancer MRI which contains high-resolution multi-sequence data. Although methods leveraging latent diffusion models (LDMs) enable large-volume synthesis, they are prone to shortcut learning, simply reproducing the condition image created by masking the lesion region. In this work, we introduce PCaPaint, a prostate cancer inpainting method based on LDMs that explicitly addresses this failure mode. To overcome shortcut learning that compromises synthetic tumor texture, we propose a simple yet efficient conditioning strategy in which the condition image is filled with Gaussian noise, and we provide theoretical justification. In addition, we propose a novel training objective for LDM that emphasizes the error within the lesion region. Furthermore, we introduce a multi-sequence latent design, in which T2w scans and DWI&ADC scans are compressed using two separate autoencoders to preserve their distinct frequency characteristics. Extensive experiments demonstrate that the generated synthetic data improves downstream performance in prostate lesion segmentation, patient-level classification and lesion-level detection. Furthermore, our method significantly outperforms a recent state-of-the-art LDM-based tumor inpainting method both in downstream performance and in synthetic image quality.
Figures & tables
| MRI | Metric | DiffTumor | DiffTumor NF | PCaPaint (Ours) |
|---|---|---|---|---|
| T2w | SSIM | 0.77 (0.01) | 0.79 (0.01) | 0.83 ∗ (0.02) |
| PSNR | 20.75 (0.56) | 21.58 (0.60) | 23.11 ∗ (0.62) | |
| PSNR fg | 12.38 (0.64) | 16.26 (0.80) | 20.42 ∗ (0.97) | |
| PSNR bg | 20.80 (0.55) | 21.60 (0.60) | 23.11 ∗ (0.62) | |
| DWI | SSIM | 0.51 (0.06) | 0.51 (0.06) | 0.64 ∗ (0.05) |
| PSNR | 21.39 (0.66) | 21.16 (0.67) | 25.54 ∗ (1.26) |
| Downstream Task | Real | Real+DiffTumor | Real+PCaPaint (Ours) |
|---|---|---|---|
| Segmentation, Dice | 0.47 (0.08) | 0.47 (0.09) | 0.48 ∗ (0.08) |
| Classification, AUC | 0.71 (0.09) | 0.71 (0.09) | 0.76 ∗ (0.08) |
| Detection, AP | 0.28 (0.13) | 0.29 (0.13) | 0.34 ∗ (0.15) |