Domain-Adapted Latent Diffusion Models for CT-to-PET Image Translation

Yann Bourdé; Lea Bogensperger; Arnela Hadzic; Michael Krauthammer

cs.CVOct 26, 2026

Domain-Adapted Latent Diffusion Models for CT-to-PET Image Translation

Authors: Yann Bourdé, Lea Bogensperger, Arnela Hadzic, Michael Krauthammer

Organizations: Department of Quantitative Biomedicine, University of Zurich · Institute for Medical Informatics, Statistics and Documentation, Medical University of Graz

Abstract

Positron Emission Tomography (PET) provides important metabolic information for tumor detection and staging but remains significantly less accessible than Computed Tomography (CT) due to its high cost, infrastructure requirements, and reliance on injected radioactive tracers. CT-to-PET image translation using generative models aims to address this disparity, yet state-of-the-art diffusion-based methods are limited by the scarcity of paired training data. To overcome this, we propose a domain-adapted latent diffusion framework that transfers natural-image priors using a relatively small dataset of 21,930 paired CT/PET slices, incorporating a signed power transform to resolve extreme intensity sparsity in PET data. Compared to CPDM, a state-ofthe-art baseline for CT-to-PET image translation, our approach yields superior performance in pixel-wise reconstruction and localized tumor fidelity, demonstrating effective transfer of natural-image priors to the medical domain. We further provide a systematic analysis of evaluation metric reliability for sparse PET data, showing that standard full-image metrics can be misleading indicators of translated image quality, and propose background-filtered and mask-restricted metrics as more clinically meaningful alternatives.

Figures & tables

Modality	MAE $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$
CT	0.0111	33.57	0.936	0.0616
PET	0.0019	48.07	0.982	0.0147

Table 1: VAE reconstruction metrics.

Model	MAE $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$
CPDM original ckpt	$\mathmakebox[r]{0.0204}\pm 0.0079$	$\mathmakebox[r]{26.67}\pm 2.80$	$\mathmakebox[r]{0.828}\pm 0.060$	$\mathmakebox[r]{0.145}\pm 0.020$
CPDM	$\mathmakebox[r]{\underline{0.0141}}\pm 0.0101$	$\mathmakebox[r]{29.04}\pm 4.30$	$\mathmakebox[r]{0.884}\pm 0.082$	$\mathmakebox[r]{\textbf{0.109}}\pm 0.021$
base	$\mathmakebox[r]{0.0182}\pm 0.0100$	$\mathmakebox[r]{\textbf{30.62}}\pm 4.64$	$\mathmakebox[r]{0.591}\pm 0.079$	$\mathmakebox[r]{0.204}\pm 0.023$
power	$\mathmakebox[r]{\textbf{0.0129}}\pm 0.0081^{*}$	$\mathmakebox[r]{30.04}\pm 4.01$	$\mathmakebox[r]{\textbf{0.913}}\pm 0.066^{*}$	$\mathmakebox[r]{\underline{0.123}}\pm 0.024^{*}$
base frozen	$\mathmakebox[r]{0.0173}\pm 0.0080^{*}$	$\mathmakebox[r]{\underline{30.17}}\pm 3.89$	$\mathmakebox[r]{0.661}\pm 0.069^{*}$	$\mathmakebox[r]{0.238}\pm 0.021^{*}$
power frozen	$\mathmakebox[r]{0.0158}\pm 0.0072$	$\mathmakebox[r]{28.50}\pm 3.22$	$\mathmakebox[r]{\underline{0.893}}\pm 0.062$	$\mathmakebox[r]{0.137}\pm 0.023$

Table 2: Conditional CT-to-PET results — full-image metrics, mean and standard deviation. Per-column best results are denoted in bold , second-best results are underlined . ∗ indicates a statistically significant improvement (

p<0.05

, paired Wilcoxon signed-rank test) compared to the retrained CPDM baseline.

Model	Masked MAE $\downarrow$	Masked SSIM $\uparrow$	Masked LPIPS $\downarrow$
CPDM original ckpt	$\mathmakebox[r]{0.189}\pm 0.120$	$\mathmakebox[r]{0.243}\pm 0.124$	$\mathmakebox[r]{0.463}\pm 0.066$
CPDM	$\mathmakebox[r]{0.221}\pm 0.131$	$\mathmakebox[r]{0.276}\pm 0.160$	$\mathmakebox[r]{0.400}\pm 0.069$
base	$\mathmakebox[r]{0.181}\pm 0.101$	$\mathmakebox[r]{\textbf{0.485}}\pm 0.194$	$\mathmakebox[r]{\textbf{0.368}}\pm 0.098$
power	$\mathmakebox[r]{\textbf{0.165}}\pm 0.092^{*}$	$\mathmakebox[r]{\underline{0.480}}\pm 0.175^{*}$	$\mathmakebox[r]{\underline{0.369}}\pm 0.092^{*}$
base frozen	$\mathmakebox[r]{\underline{0.171}}\pm 0.098^{*}$	$\mathmakebox[r]{0.462}\pm 0.170^{*}$	$\mathmakebox[r]{0.413}\pm 0.094^{*}$
power frozen	$\mathmakebox[r]{\textbf{0.165}}\pm 0.095$	$\mathmakebox[r]{0.427}\pm 0.157$	$\mathmakebox[r]{0.416}\pm 0.090$

Table 3: Conditional CT-to-PET results — masked/tumour-box metrics, mean and standard deviation. Per-column best results are denoted in bold , second-best results are underlined . ∗ indicates a statistically significant improvement (

p<0.05

, paired Wilcoxon signed-rank test) compared to the retrained CPDM baseline.

Figure 1: Conditional CT-to-PET generation examples. Columns: input CT, then PET generations (top) and absolute error maps vs. ground truth (bottom) for each model.

	MAE $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	Masked MAE $\downarrow$
Unfiltered (mean)	0.0154	32.58	0.587	0.184	0.1719
Filtered (mean)	0.0093	32.91	0.909	0.152	—

Table 4: Conditional CT-to-PET evaluation metrics with background filter (

\tau=0.05

), averaged over 160 test samples.

Figure 2: Filtered vs unfiltered CT-to-PET generations with absolute error maps.

Medical Image Synthesis

Medical image synthesis creates, translates, reconstructs, and edits clinical images to fill missing data and support analysis.

See all 310 papers

3D CT-to-PET Translation via Latent Brownian Bridge Diffusion
Sep 11, 2026Sarita Mourya, Francesco Di Feola, Pierangelo Veltri +1PET/CT Imaging Medical Image Synthesis
Department of Medical and Surgical Sciences, Universit`a ”Magna Græcia” di Catanzaro Catanzaro, Italy · Department of Diagnostics and Intervention, Radiation Physics, Biomedical Engineering, Ume˚a University, Sweden · Department of Computer Science, Modeling, Electronics and Systems Engineering (DIMES) University of Calabria, Italy +1
Generation of Heterogeneous PET Images from Uniform Organ Activity Maps Using a Pretrained Domain-Adapted Diffusion Model
May 18, 2026Suya Li, Kaushik Dutta, Debojyoti Pal +2Diffusion Model Adaptation PET/CT Imaging
Mallinckrodt Institute of Radiology, Washington University School of Medicine, St. Louis, USA · Imaging Science Program, McKelvey School of Engineering, Washington University in St Louis, St. Louis, USA · Department of Surgery, Washington University School of Medicine, St. Louis, USA +1
TransNSNet: A Transformer-Enhanced Multi-Scale Adversarial Framework for CT-to-PET Image Synthesis
Oct 16, 2026Shaojun Li, Xiaoyu Deng, Ikuo KeshiMedical Image Synthesis Image-to-Image Translation
Fukui University of Technology, Fukui Campus, 3-6-1 Gakuen, Fukui City, Fukui Prefecture 910-8505, JAPAN · University of Fukui, Bunkyo Campus, 3-9-1 Bunkyo, Fukui City, Fukui Prefecture 910-8507, JAPAN

Sep 11, 2026J/K move · Enter open · S save

3D CT-to-PET Translation via Latent Brownian Bridge Diffusion

Sarita Mourya, Francesco Di Feola, Pierangelo Veltri, Paolo Soda

Department of Medical and Surgical Sciences, Universit`a ”Magna Græcia” di Catanzaro Catanzaro, Italy · Department of Diagnostics and Intervention, Radiation Physics, Biomedical Engineering, Ume˚a University, Sweden · Department of Computer Science, Modeling, Electronics and Systems Engineering (DIMES) University of Calabria, Italy+1

PET/CT Imaging Medical Image Synthesis 3D Diffusion Models Diffusion Models

Open paper AskPDF arXiv

Computed tomography (CT) and positron emission tomography (PET) provide complementary anatomical and functional information for cancer diagnosis and treatment planning. However, the widespread use of PET is limited by high radiation exposure, elevated costs, and restricted availability. To address these limitations, deep learning-based CT-to-PET translation has emerged as a promising approach for synthesizing PET-like information directly from CT images, although accurately modeling the large cross-modal gap remains challenging. In this work, we propose a 3D CT-to-PET translation framework based on latent Brownian Bridge Diffusion (BBDM). The method consists of two stages. First, a Variational Autoencoder (VAE) is trained on paired CT-PET patches, integrating contrastive learning to improve latent alignment between anatomical and metabolic representations. Second, a BBDM is trained in the latent space to translate CT latent representations into their corresponding PET counterparts. The translated PET latents are then decoded and stitched to reconstruct the final 3D PET volume. We evaluate the proposed approach on two publicly available datasets. Quantitative results based on image fidelity and lesion-level PET-specific metrics demonstrate improved performance compared with competing methods. In particular, the proposed approach improves PET signal fidelity, better preserves clinically relevant uptake patterns, and shows improved performance in preserving small-lesion metabolic activation, paving the way for virtual imaging applications.

Abstract

Medical Image Synthesis

Explore similar work

3D CT-to-PET Translation via Latent Brownian Bridge Diffusion

Generation of Heterogeneous PET Images from Uniform Organ Activity Maps Using a Pretrained Domain-Adapted Diffusion Model

TransNSNet: A Transformer-Enhanced Multi-Scale Adversarial Framework for CT-to-PET Image Synthesis