Translating MRI to PET through Conditional Diffusion Models with Enhanced Pathology Awareness

Yitong Li; Igor Yakushev; Dennis M. Hedderich; Christian Wachinger

cs.CVMar 19, 2026

Translating MRI to PET through Conditional Diffusion Models with Enhanced Pathology Awareness

Authors: Yitong Li, Igor Yakushev, Dennis M. Hedderich, Christian Wachinger

Organizations: Lab for Artificial Intelligence in Medical Imaging, Institute for Diagnostic and Interventional Radiology, School of Medicine and Health, TUM Klinikum, Technical University of Munich (TUM), Munich, 81675, Germany · Munich Center for Machine Learning (MCML), Munich, Germany · Department of Nuclear Medicine, School of Medicine and Health, TUM Klinikum, Munich, 81675, Germany · Department of Neuroradiology, School of Medicine and Health, TUM Klinikum, Munich, 81675, Germany

Abstract

Positron emission tomography (PET) is a widely recognized technique for diagnosing neurodegenerative diseases, offering critical functional insights. However, its high costs and radiation exposure hinder its widespread use. In contrast, magnetic resonance imaging (MRI) does not involve such limitations. While MRI also detects neurodegenerative changes, it is less sensitive for diagnosis compared to PET. To overcome such limitations, one approach is to generate synthetic PET from MRI. Recent advances in generative models have paved the way for cross-modality medical image translation; however, existing methods largely emphasize structural preservation while neglecting the critical need for pathology awareness. To address this gap, we propose PASTA, a novel image translation framework built on conditional diffusion models with enhanced pathology awareness. PASTA surpasses state-of-the-art methods by preserving both structural and pathological details through its highly interactive dual-arm architecture and multi-modal condition integration. Additionally, we introduce a novel cycle exchange consistency and volumetric generation strategy that significantly enhances PASTA's ability to produce high-quality 3D PET images. Our qualitative and quantitative results demonstrate the high quality and pathology awareness of the synthesized PET scans. For Alzheimer's diagnosis, the performance of these synthesized scans improves over MRI by 4%, almost reaching the performance of actual PET. Our code is available at https://github.com/ai-med/PASTA.

Figures & tables

Figure 1: For Alzheimer’s disease, PET reveals distinctly reduced glucose uptake in the temporoparietal lobe (bottom circles), mirroring atrophy on MRI with higher sensitivity. Compared to the ground-truth (GT) PET, state-of-the-art diffusion models (SOTA DM) fail to recover such pathology in the synthesized (Syn) PET from MRI input. In contrast, PASTA shows improvement in preserving disease-specific pathology.

Figure 2: PASTA features a symmetric dual-arm structure with a conditioner arm ( $\phi_{\omega}$ ), a denoiser arm ( $\mathbf{x}_{\theta}$ ), and adaptive conditional modules (AdaGN). Through AdaGN, PASTA conditions the feature maps $\boldsymbol{h}$ from $\mathbf{x}_{\theta}$ on timestep $t$ , clinical data $\boldsymbol{c}$ , and task representation $\boldsymbol{h}_{m}$ from $\phi_{\omega}$ . It achieves high-quality 3D PET synthesis through a volumetric generation strategy.

Figure 3: Cycle exchange consistency (CycleEx) strategy of PASTA. The two translation mappings $\boldsymbol{G}_{p}:\mathcal{M}\rightarrow\mathcal{P}$ and $\boldsymbol{G}_{m}:\mathcal{P}\rightarrow\mathcal{M}$ maintain cycle consistency. In addition, their network architectures are mirrored: both contain the same conditioner arm $\phi_{\omega}$ and denoiser arm $\mathbf{x}_{\theta}$ , but with an exchanged position. This strategy ensures information sharing between the two arms.

Figure 4: MetaROIs illustration on a brain MRI.

Figure 5: Qualitative comparison of cross-modality synthesis methods. The first row shows images from a normal control subject with no obvious pathology, and the second row shows an AD patient with obvious hypometabolism in the temporoparietal lobe (bottom left and right parts). The left temporoparietal lobe is magnified in the third row. For the normal subject, most baselines recover the structure and metabolic information well; for the AD subject, PASTA demonstrates superior generation fidelity and pathology preservation.

Data	Method	MAE ( $\times 10^{-2}$ ) $\downarrow$	MSE ( $\times 10^{-2}$ ) $\downarrow$	PSNR $\uparrow$	SSIM(%) $\uparrow$
ADNI	CycleGAN [ 36 ]	9.83 $\pm$ 0.92	2.61 $\pm$ 0.42	16.38 $\pm$ 0.66	47.48 $\pm$ 2.36
	Reg-GAN [ 52 ]	8.66 $\pm$ 1.03	2.47 $\pm$ 0.48	16.66 $\pm$ 0.83	55.70 $\pm$ 1.31
	Pix2Pix [ 43 ]	5.56 $\pm$ 0.43	1.15 $\pm$ 0.14	19.90 $\pm$ 0.49	73.23 $\pm$ 1.42
	ResVit [ 17 ]	6.72 $\pm$ 0.72	1.74 $\pm$ 0.24	19.40 $\pm$ 0.59	69.83 $\pm$ 2.08
	BBDM-LDM [ 10 ]	3.96 $\pm$ 0.69	0.54 $\pm$ 0.15	23.75 $\pm$ 1.01	84.25 $\pm$ 1.30
	BBDM [ 10 ]	3.88 $\pm$ 0.47	0.56 $\pm$ 0.11	23.37 $\pm$ 0.76	84.55 $\pm$ 1.36

Table 1: Quantitative comparison between the baselines on the ADNI dataset. We report the mean

\pm

standard deviation across the test set.

Data	Method	MAE ( $\times 10^{-2}$ ) $\downarrow$	MSE ( $\times 10^{-2}$ ) $\downarrow$	PSNR $\uparrow$	SSIM(%) $\uparrow$
In-house	CycleGAN [ 36 ]	15.10 $\pm$ 1.40	4.76 $\pm$ 1.06	15.29 $\pm$ 0.68	29.30 $\pm$ 2.50
	Reg-GAN [ 52 ]	10.83 $\pm$ 1.52	2.47 $\pm$ 0.52	16.17 $\pm$ 0.91	42.02 $\pm$ 2.25
	Pix2Pix [ 43 ]	9.27 $\pm$ 1.62	2.32 $\pm$ 0.93	18.50 $\pm$ 0.95	58.37 $\pm$ 5.68
	ResVit [ 17 ]	6.05 $\pm$ 1.48	1.31 $\pm$ 0.93	19.18 $\pm$ 1.37	65.86 $\pm$ 5.17
	BBDM-LDM [ 10 ]	4.66 $\pm$ 2.42	0.73 $\pm$ 0.50	22.71 $\pm$ 4.36	79.88 $\pm$ 7.25
	BBDM [ 10 ]	4.85 $\pm$ 1.69	0.61 $\pm$ 0.39	23.03 $\pm$ 3.11	52.07 $\pm$ 16.9

Table 2: Quantitative comparison between the baselines on the in-house dataset. We report the mean

\pm

standard deviation across one balanced test split.

Data	Method	MAE ( $\times 10^{-2}$ ) $\downarrow$	MSE ( $\times 10^{-2}$ ) $\downarrow$	PSNR $\uparrow$	SSIM(%) $\uparrow$
In-house	CycleGAN [ 36 ]	11.7 $\pm$ 1.91	2.88 $\pm$ 0.95	16.09 $\pm$ 1.14	31.8 $\pm$ 6.6
	RegGAN [ 52 ]	10.8 $\pm$ 1.41	2.53 $\pm$ 0.65	16.91 $\pm$ 0.82	41.5 $\pm$ 2.6
	Pix2Pix [ 43 ]	8.75 $\pm$ 0.40	2.03 $\pm$ 0.11	18.73 $\pm$ 0.19	58.3 $\pm$ 2.4
	ResVit [ 17 ]	7.09 $\pm$ 0.74	1.87 $\pm$ 0.34	19.93 $\pm$ 0.48	68.3 $\pm$ 2.5
	BBDM-LDM [ 10 ]	4.48 $\pm$ 0.17	0.66 $\pm$ 0.05	23.04 $\pm$ 0.37	78.9 $\pm$ 0.7
	BBDM [ 10 ]	4.70 $\pm$ 0.62	0.69 $\pm$ 0.05	22.88 $\pm$ 0.67	71.0 $\pm$ 12.0

Table 3: Quantitative comparison between the baselines on the in-house dataset with 5-fold cross-validation. We report the mean

\pm

standard deviation across the five test splits.

Input	BACC $\uparrow$	F1-Score $\uparrow$	AUC $\uparrow$
MRI	79.23 $\pm$ 4.30	74.97 $\pm$ 5.95	85.88 $\pm$ 3.79
MRI+ $\mathbf{c}$	81.51 $\pm$ 3.16	77.83 $\pm$ 3.83	89.19 $\pm$ 1.41
GT PET	87.02 $\pm$ 2.35	80.77 $\pm$ 2.62	89.04 $\pm$ 1.88
Syn PET (ResVit)	78.54 $\pm$ 4.15	74.41 $\pm$ 5.28	81.80 $\pm$ 4.95
Syn PET (BBDM)	72.93 $\pm$ 8.48	70.76 $\pm$ 6.02	83.29 $\pm$ 5.13
Syn PET (PASTA)	83.41 $\pm$ 2.67	79.98 $\pm$ 3.51	91.63 $\pm$ 2.21

Table 4: Classification results for Alzheimer’s disease classification with different input modalities.

Method	MAE ROI ( $\times 10^{-4}$ ) $\downarrow$	MSE ROI ( $\times 10^{-4}$ ) $\downarrow$	PSNR ROI $\uparrow$	SSIM ROI (%) $\uparrow$
CycleGAN [ 36 ]	18.54 $\pm$ 4.94	4.51 $\pm$ 2.23	32.73 $\pm$ 2.13	99.41 $\pm$ 0.16
Reg-GAN [ 52 ]	21.06 $\pm$ 4.03	6.69 $\pm$ 1.86	30.88 $\pm$ 0.93	99.09 $\pm$ 0.10
Pix2Pix [ 43 ]	12.59 $\pm$ 2.14	2.29 $\pm$ 0.64	35.46 $\pm$ 1.21	99.52 $\pm$ 0.06
ResVit [ 17 ]	13.12 $\pm$ 2.20	2.86 $\pm$ 0.69	34.49 $\pm$ 1.34	99.42 $\pm$ 0.11
BBDM-LDM [ 10 ]	10.79 $\pm$ 2.36	1.63 $\pm$ 0.63	37.03 $\pm$ 1.69	99.67 $\pm$ 0.08
BBDM [ 10 ]	10.54 $\pm$ 2.24	1.66 $\pm$ 0.61	36.96 $\pm$ 1.70	99.64 $\pm$ 0.09

Table 5: Pathology-localized quantitative comparison on the ADNI dataset using ROI-based metrics computed within AD-related MetaROI regions. We report the mean

\pm

standard deviation across the test set.

		Age	Gender	Edu	MMSE	ADAS	ApoE4	All
CN	MAE $\downarrow$	3.34	3.32	3.34	3.30	3.30	3.34	3.26
CN	$\text{MAE}_{ROI}$ $\downarrow$	9.82	9.61	9.68	9.10	8.46	9.58	8.42
AD	MAE $\downarrow$	3.57	3.57	3.57	3.55	3.57	3.57	3.57
AD	$\text{MAE}_{ROI}$ $\downarrow$	10.16	10.12	10.17	10.13	9.94	10.14	9.84

Table 6: Results of sensitivity analysis, keeping one clinical variable at a time while neutralizing the rest during inference on ADNI test set. The outcomes are compared against the original configuration, where all variables were preserved (All). (MAE:

\times 10^{-2}

\text{MAE}_{ROI}

\times 10^{-4}

)

Figure 6: Z-score maps produced by Neurostat 3D-SSP for a healthy (left) and an Alzheimer’s patient (right). We present global metabolic Z-score maps for the ground-truth (GT) PET and synthesized PET scans from each method in different brain cortical areas. The mean absolute errors (MAE) between the maps from synthesized PETs and the GT are displayed alongside.

Ablation	MAE ( $\times 10^{-2}$ ) $\downarrow$	MSE ( $\times 10^{-2}$ ) $\downarrow$	PSNR $\uparrow$	SSIM (%) $\uparrow$
w/o CycleEx	3.99	0.54	23.64	85.14
w/o $\boldsymbol{\lambda}_{R}$	3.67	0.48	24.19	85.95
w/o $\boldsymbol{c}$	3.57	0.46	24.39	86.12
PASTA (ConcatFeats)	3.61	0.48	24.04	85.00
PASTA (M2M)	3.78	0.50	23.78	84.77
PASTA (CondCond)	3.55	0.45	24.32	86.11

Table 7: Ablation studies on important designs in PASTA.

Figure 7: Error maps for ablation studies on important designs in PASTA, for an AD-diagnosed subject. We show its ground-truth (GT) PET and the averaged error maps of the scan along three directions respectively. Specifically, we circle the areas that are highly influenced by AD in red.

$N$	MAE ( $\times 10^{-2}$ ) $\downarrow$	MSE ( $\times 10^{-2}$ ) $\downarrow$	PSNR $\uparrow$	SSIM(%) $\uparrow$
1	4.05	0.60	23.10	83.09
5	3.64	0.50	24.07	85.47
11	3.55	0.46	24.33	85.79
15	3.45	0.43	24.59	86.29
19	3.59	0.45	24.26	84.79

Table 8: Ablation on the number

N

of input neighboring slices.

Figure 8: Ablation on input number $N$ of neighboring slices. We demonstrate the synthesized PET scans with various $N$ s in three directions (axial, coronal, and sagittal), given that axial slices are used as input to the network.

Figure 9: Ablation on the directions of input slices and the average of synthesized scans from the three directions. The influence of the input slice direction is minimal on the synthesized PET scans.

Demographics	Groups	MAE ( $\times 10^{-2}$ ) $\downarrow$
Age	$<$ 60	3.55 $\pm$ 0.55
	60 - 70	3.30 $\pm$ 0.36
	70 - 80	3.44 $\pm$ 0.50
	$>$ 80	3.66 $\pm$ 0.67
Gender	Male	3.52 $\pm$ 0.57
Gender	Female	3.35 $\pm$ 0.41

Table 9: The mean absolute error (MAE) of synthesized PET on the test set among different demographics.

Appendix figures & tables2 assets

Supplementary material from the paper’s appendix.

Appendix

Method	GPU Memory (GB)	Time (ms/step)
CycleGAN [ 36 ]	4.70	67
RegGAN [ 52 ]	3.78	214
Pix2Pix [ 43 ]	5.02	25
ResViT [ 17 ]	6.94	308
BBDM-LDM [ 10 ]	4.05	98
BBDM [ 10 ]	3.46	70

Appendix

Table A.1: Computational cost of different methods under identical setup (batch size

=6

Method	MAE ROI ( $\times 10^{-4}$ ) $\downarrow$	MSE ROI ( $\times 10^{-4}$ ) $\downarrow$	PSNR ROI $\uparrow$	SSIM ROI (%) $\uparrow$
PASTA	9.07 $\pm$ 2.21	1.24 $\pm$ 0.44	38.29 $\pm$ 1.65	99.74 $\pm$ 0.07
PASTA (CCL)	9.03 $\pm$ 3.09	1.31 $\pm$ 0.69	38.21 $\pm$ 2.16	99.72 $\pm$ 0.10
PASTA (CCL, $\lambda_{R}$ learn)	8.75 $\pm$ 3.11	1.22 $\pm$ 0.71	38.51 $\pm$ 2.15	99.75 $\pm$ 0.09
PASTA (SA-AdaGN)	8.96 $\pm$ 2.64	1.17 $\pm$ 0.50	38.64 $\pm$ 1.75	99.76 $\pm$ 0.06
PASTA (SA-AdaGN, $\lambda_{R}$ learn)	8.76 $\pm$ 3.14	1.24 $\pm$ 0.78	38.46 $\pm$ 2.09	99.74 $\pm$ 0.09

Appendix

Table B.2: Pathology-localized quantitative comparison on the ADNI dataset using ROI-based metrics computed within AD-related MetaROI regions. We report the mean

\pm

standard deviation across the test set.

Explore similar work

Precision Synthesis of Multi-Tracer PET via VLM-Modulated Rectified Flow for Stratifying Mild Cognitive Impairment
Apr 13, 2026Tuo Liu, Shuijin Lin, Shaozhen Yan +4Medical Image Synthesis Rectified Flow
School of Mathematics and Statistics, Xi’an Jiaotong University, No. 28, Xianning West Road, Xi’an, 710049, Shaanxi, China · Research Center for Intelligent Medical Equipment and Devices (IMED), Xi’an Jiaotong University, No. 28, Xianning West Road, Xi’an, 710049, Shaanxi, China · Key Laboratory of Biomedical Information Engineering of Ministry of Education, School of Life Science and Technology, Xi’an Jiaotong University, No. 28, Xianning West Road, Xi’an, 710049, Shaanxi, China +1
Bridging Modalities on the Cortex: Surface-based MRI to PET Translation with a Diffusion Bridge
Sep 17, 2026Yitong Li, Alexandra Samoylova, Fabian Bongratz +4Image-to-Image Translation Vision Transformer
Technical University of Munich (TUM), TUM University Hospital, and Munich Center for Machine Learning (MCML), Germany · Department of Neurology, TUM University Hospital, Germany · Department of Neuroradiology, TUM University Hospital, Germany +1
RelA-Diffusion: Relativistic Adversarial Diffusion for Multi-Tracer PET Synthesis from Multi-Sequence MRI
Feb 24, 2026Minhui Yu, Yongheng Sun, David S. Lalush +3Medical Image Synthesis Diffusion Models
Department of Radiology and Biomedical Research Imaging Center (BRIC), University of North Carolina at Chapel Hill, Chapel Hill, NC 27599, USA. · Joint Department of Biomedical Engineering, University of North Carolina at Chapel Hill and North Carolina State University, Chapel Hill, NC 27599, USA. · Department of Exercise and Sport Science, UNC-CH, Chapel Hill, NC 27599, USA.

Apr 13, 2026J/K move · Enter open · S save

Precision Synthesis of Multi-Tracer PET via VLM-Modulated Rectified Flow for Stratifying Mild Cognitive Impairment

Tuo Liu, Shuijin Lin, Shaozhen Yan, Haifeng Wang+3

School of Mathematics and Statistics, Xi’an Jiaotong University, No. 28, Xianning West Road, Xi’an, 710049, Shaanxi, China · Research Center for Intelligent Medical Equipment and Devices (IMED), Xi’an Jiaotong University, No. 28, Xianning West Road, Xi’an, 710049, Shaanxi, China · Key Laboratory of Biomedical Information Engineering of Ministry of Education, School of Life Science and Technology, Xi’an Jiaotong University, No. 28, Xianning West Road, Xi’an, 710049, Shaanxi, China+1

Medical Image Synthesis Rectified Flow

Open paper AskPDF arXiv

The biological definition of Alzheimer's disease (AD) relies on multi-modal neuroimaging, yet the clinical utility of positron emission tomography (PET) is limited by cost and radiation exposure, hindering early screening at preclinical or prodromal stages. While generative models offer a promising alternative by synthesizing PET from magnetic resonance imaging (MRI), achieving subject-specific precision remains a primary challenge. Here, we introduce DIReCT

++

, a Domain-Informed ReCTified flow model for synthesizing multi-tracer PET from MRI combined with fundamental clinical information. Our approach integrates a 3D rectified flow architecture to capture complex cross-modal and cross-tracer relationships with a domain-adapted vision-language model (BiomedCLIP) that provides text-guided, personalized generation using clinical scores and imaging knowledge. Extensive evaluations on multi-center datasets demonstrate that DIReCT

++

not only produces synthetic PET images (

^{18}

F-AV-45 and

^{18}

F-FDG) of superior fidelity and generalizability but also accurately recapitulates disease-specific patterns. Crucially, combining these synthesized PET images with MRI enables precise personalized stratification of mild cognitive impairment (MCI), advancing a scalable, data-efficient tool for the early diagnosis and prognostic prediction of AD. The source code will be released on https://github.com/ladderlab-xjtu/DIReCT-PLUS.

Abstract

Explore similar work

Precision Synthesis of Multi-Tracer PET via VLM-Modulated Rectified Flow for Stratifying Mild Cognitive Impairment

Bridging Modalities on the Cortex: Surface-based MRI to PET Translation with a Diffusion Bridge

RelA-Diffusion: Relativistic Adversarial Diffusion for Multi-Tracer PET Synthesis from Multi-Sequence MRI