PCaPaint: Prostate Cancer Inpainting by Mitigating Shortcut Learning

Levente Lippenszky; Hongxu Yang; Marcell Dömötör; Krisztian Koos; László Ruskó

cs.CVSep 29, 2026

PCaPaint: Prostate Cancer Inpainting by Mitigating Shortcut Learning

Authors: Levente Lippenszky, Hongxu Yang, Marcell Dömötör, Krisztian Koos, László Ruskó

Organizations: GE HealthCare

Abstract

The development of AI systems for tumor-specific applications is limited by the scarcity of labeled data. Synthetic tumor inpainting offers a promising approach but faces challenges for prostate cancer MRI which contains high-resolution multi-sequence data. Although methods leveraging latent diffusion models (LDMs) enable large-volume synthesis, they are prone to shortcut learning, simply reproducing the condition image created by masking the lesion region. In this work, we introduce PCaPaint, a prostate cancer inpainting method based on LDMs that explicitly addresses this failure mode. To overcome shortcut learning that compromises synthetic tumor texture, we propose a simple yet efficient conditioning strategy in which the condition image is filled with Gaussian noise, and we provide theoretical justification. In addition, we propose a novel training objective for LDM that emphasizes the error within the lesion region. Furthermore, we introduce a multi-sequence latent design, in which T2w scans and DWI&ADC scans are compressed using two separate autoencoders to preserve their distinct frequency characteristics. Extensive experiments demonstrate that the generated synthetic data improves downstream performance in prostate lesion segmentation, patient-level classification and lesion-level detection. Furthermore, our method significantly outperforms a recent state-of-the-art LDM-based tumor inpainting method both in downstream performance and in synthetic image quality.

Figures & tables

Figure 1: DiffTumor [ 3 ] samples under different conditioning strategies. Panels show the condition and sample for zero fill (gray), $-1$ fill (black) and the proposed noise fill.

Figure 2: Training the latent diffusion model (LDM). The lesion bpMRI $\bm{x}_{0}$ is encoded by the VQGAN encoders to form the latent $\bm{z}_{0}$ . LDM is conditioned on the latent healthy image $\bm{z}_{h}$ and the downsampled lesion mask $\tilde{\bm{m}}$ . The healthy image $\bm{h}$ is constructed by replacing the lesion region with Gaussian noise. LDM is trained with an objective that amplifies the error in the lesion region.

MRI	Metric $\uparrow$	DiffTumor	DiffTumor NF	PCaPaint (Ours)
T2w	SSIM	0.77 (0.01)	0.79 (0.01)	0.83 ∗ (0.02)
	PSNR	20.75 (0.56)	21.58 (0.60)	23.11 ∗ (0.62)
	PSNR fg	12.38 (0.64)	16.26 (0.80)	20.42 ∗ (0.97)
	PSNR bg	20.80 (0.55)	21.60 (0.60)	23.11 ∗ (0.62)
DWI	SSIM	0.51 (0.06)	0.51 (0.06)	0.64 ∗ (0.05)
	PSNR	21.39 (0.66)	21.16 (0.67)	25.54 ∗ (1.26)

Table 1: Synthetic image quality evaluation for DiffTumor [ 3 ] , DiffTumor with noise fill (NF) and PCaPaint. SSIM and PSNR were computed between ground truth and synthetic images for test lesion bpMRIs for the entire image, foreground (fg) and background (bg). 95% bootstrap CI widths are in brackets. Best and second-best are in bold and underline , respectively. ∗ indicates statistically significant differences between DiffTumor and PCaPaint at 5% level using two-sided Wilcoxon signed-rank tests with Holm correction.

Figure 3: Three non-lesion bpMRI examples inpainted with synthetic lesions by DiffTumor [ 3 ] and PCaPaint using generated lesion masks.

Downstream Task	Real	Real+DiffTumor	Real+PCaPaint (Ours)
Segmentation, Dice $\uparrow$	0.47 (0.08)	0.47 (0.09)	0.48 ∗ (0.08)
Classification, AUC $\uparrow$	0.71 (0.09)	0.71 (0.09)	0.76 ∗ (0.08)
Detection, AP $\uparrow$	0.28 (0.13)	0.29 (0.13)	0.34 ∗ (0.15)

Table 2: nnU-Net performance on csPCa segmentation, patient-level csPCa classification and lesion-level csPCa detection using real and synthetic data. 95% bootstrap CI widths are in brackets. Best performance is in bold . ∗ indicates statistically significant differences between Real+DiffTumor and Real+PCaPaint at 5% level.

Explore similar work

Semi-Supervised Domain Adaptation with Latent Diffusion for Pathology Image Classification
Jan 23, 2026Tengyue Zhang, Ruiwen Ding, Luoting Zhuang +3Computational Pathology Domain Adaptation
Medical & Imaging Informatics, Department of Radiological Sciences, David Geffen School of Medicine, University of California, Los Angeles (UCLA), Los Angeles, CA 90024, USA · Bioengineering Department, Henry Samueli School of Engineering, UCLA, Los Angeles, CA 90024, USA · Department of Pathology & Laboratory Sciences, David Geffen School of Medicine, UCLA, Los Angeles, CA 90024, USA
Lesion-Aware Adaptive Fourier Neural Operator for CT-to-PSMA PET Synthesis in Prostate Cancer
Aug 11, 2026Rashmi Bhaskara, Waleed M. Almutairi, Matthew Gopaulchan +5Positron Emission Tomography Radiomics
aSchool of Health Sciences, Purdue University, West Lafayette, 47907, IN, USA · bDepartment of Radiology, Indiana University School of Medicine, Indianapolis, 46202, IN, USA · cDepartment of Urology, Indiana University School of Medicine, Indianapolis, 46202, IN, USA +1
Controllable Diffusion-Based Lesion Inpainting for Scalable Histopathology Data Augmentation
Jan 13, 2026Mohamad Koohi-Moghadam, Mohammad-Ali Nikouei Mahani, Rex K. H. Au-Yeung +6Digital Pathology Computational Pathology
Department of Diagnostic Radiology, Li Ka Shing Faculty of Medicine, The University of Hong Kong, Pok Fu Lam, Hong Kong SAR, PR China · Department of Pathology, Li Ka Shing Faculty of Medicine, The University of Hong Kong, Hong Kong SAR, China · Department of Anatomical and Cellular Pathology, Prince of Wales Hospital, The Chinese University of Hong Kong, Hong Kong SAR, China. +2

Jan 23, 2026, cs.CVJ/K move · Enter open · S save

Semi-Supervised Domain Adaptation with Latent Diffusion for Pathology Image Classification

Tengyue Zhang, Ruiwen Ding, Luoting Zhuang, Yuxiao Wu+2

Medical & Imaging Informatics, Department of Radiological Sciences, David Geffen School of Medicine, University of California, Los Angeles (UCLA), Los Angeles, CA 90024, USA · Bioengineering Department, Henry Samueli School of Engineering, UCLA, Los Angeles, CA 90024, USA · Department of Pathology & Laboratory Sciences, David Geffen School of Medicine, UCLA, Los Angeles, CA 90024, USA

Computational Pathology Domain Adaptation Latent Diffusion Model Cohort

Open paper AskPDF arXiv

Deep learning models in computational pathology often fail to generalize across cohorts and institutions due to domain shift. Existing approaches either fail to leverage unlabeled data from the target domain or rely on image-to-image translation, which can distort tissue structures and compromise model accuracy. In this work, we propose a semi-supervised domain adaptation (SSDA) framework that utilizes a latent diffusion model trained on unlabeled data from both the source and target domains to generate morphology-preserving and target-aware synthetic images. By conditioning the diffusion model on foundation model features, cohort identity, and tissue preparation method, we preserve tissue structure in the source domain while introducing target-domain appearance characteristics. The target-aware synthetic images, combined with real, labeled images from the source cohort, are subsequently used to train a downstream classifier, which is then tested on the target cohort. The effectiveness of the proposed SSDA framework is demonstrated on the task of lung adenocarcinoma prognostication. The proposed augmentation yielded substantially better performance on the held-out test set from the target cohort, without degrading source-cohort performance. The approach improved the weighted F1 score on the target-cohort held-out test set from 0.611 to 0.706 and the macro F1 score from 0.641 to 0.716. Our results demonstrate that target-aware diffusion-based synthetic data augmentation provides a promising and effective approach for improving domain generalization in computational pathology.

Abstract

Explore similar work

Semi-Supervised Domain Adaptation with Latent Diffusion for Pathology Image Classification

Lesion-Aware Adaptive Fourier Neural Operator for CT-to-PSMA PET Synthesis in Prostate Cancer

Controllable Diffusion-Based Lesion Inpainting for Scalable Histopathology Data Augmentation