Precision Synthesis of Multi-Tracer PET via VLM-Modulated Rectified Flow for Stratifying Mild Cognitive Impairment
Authors: Tuo Liu, Shuijin Lin, Shaozhen Yan, Haifeng Wang, Jie Lu, Jianhua Ma, Chunfeng Lian
Organizations: School of Mathematics and Statistics, Xi’an Jiaotong University, No. 28, Xianning West Road, Xi’an, 710049, Shaanxi, China · Research Center for Intelligent Medical Equipment and Devices (IMED), Xi’an Jiaotong University, No. 28, Xianning West Road, Xi’an, 710049, Shaanxi, China · Key Laboratory of Biomedical Information Engineering of Ministry of Education, School of Life Science and Technology, Xi’an Jiaotong University, No. 28, Xianning West Road, Xi’an, 710049, Shaanxi, China · Department of Radiology and Nuclear Medicine, Xuanwu Hospital, Capital Medical University, No. 45, Changchun Street, Xicheng District, Beijing, 100053, China
The biological definition of Alzheimer's disease (AD) relies on multi-modal neuroimaging, yet the clinical utility of positron emission tomography (PET) is limited by cost and radiation exposure, hindering early screening at preclinical or prodromal stages. While generative models offer a promising alternative by synthesizing PET from magnetic resonance imaging (MRI), achieving subject-specific precision remains a primary challenge. Here, we introduce DIReCT++, a Domain-Informed ReCTified flow model for synthesizing multi-tracer PET from MRI combined with fundamental clinical information. Our approach integrates a 3D rectified flow architecture to capture complex cross-modal and cross-tracer relationships with a domain-adapted vision-language model (BiomedCLIP) that provides text-guided, personalized generation using clinical scores and imaging knowledge. Extensive evaluations on multi-center datasets demonstrate that DIReCT++ not only produces synthetic PET images (18F-AV-45 and 18F-FDG) of superior fidelity and generalizability but also accurately recapitulates disease-specific patterns. Crucially, combining these synthesized PET images with MRI enables precise personalized stratification of mild cognitive impairment (MCI), advancing a scalable, data-efficient tool for the early diagnosis and prognostic prediction of AD. The source code will be released on https://github.com/ladderlab-xjtu/DIReCT-PLUS.
Figures & tables
Figure 1: Overview of the DIReCT ++ framework for multimodal PET synthesis and downstream analysis. (a) Schematic of the rectified flow (RF) process that synthesizes dual-tracer 18 F-AV-45 and 18 F-FDG PET images from MRI. (b) Fine-tuning of the pre-trained BiomedCLIP model via prompt learning to encode tracer-specific and subject-level text guidance. (c) Training pipeline of the conditional 3D RF model implemented as a multi-task U-Net that predicts velocity fields for both tracers. (d) One-step inference process that simultaneously generates synthetic AV45- and FDG-PET from MRI and textual input. (e) Downstream analytical tasks, including statistical analysis, early diagnosis, and stratification, demonstrating the clinical applicability of the synthesized PET images.
Figure 2: Comparison of PET reconstruction quality across methods and datasets. (a) Radar plots show normalized quantitative results (SSIM, MSE, MAE, PSNR) for FDG-PET and AV45-PET reconstruction on ADNI, and cross-dataset generalization on OASIS. Notably, each metric is normalized by the best value among all methods. (b) Representative PET synthesis results for cognitively normal (CN, top), mild cognitive impairment (MCI, middle), and Alzheimer’s disease (AD, bottom) subjects. Columns: MRI input, competing methods, DIReCT (i.e., ours single), ours (DIReCT ++ ), and real PET. Each sub-row depicts FDG (Up) and AV45 (Down) reconstruction. (c) Generalization to OASIS using models trained on ADNI. Rows: CN and AD subjects. Our method yields consistently higher quantitative scores and superior visual fidelity across datasets.
Figure 3: Detailed regional visualization of synthetic and real PET images. The figure displays magnified regional views of FDG-PET (left two columns) and AV45-PET (right two columns) for representative AD and CN subjects. For each modality and group, the first sub-column shows real PET, and the second sub-column shows synthetic PET. Key anatomical regions are highlighted in the bottom-right corner of each sub-column, with their names labeled at the top-center. Each row corresponds to a distinct brain region, demonstrating strong regional consistency between synthetic and real PET, and clear differences between diagnostic groups.
Figure 4: Regional comparison of mean uptake between real and MRI-synthetic PET images. (a,b) Violin plots of regional mean uptake across cortical and subcortical regions for AV45 (a) and FDG (b), with statistical comparisons. (c–f) Disease-related patterns. Violin plots show mean uptake in AD (red) and CN (green). For each region, paired half-violins denote real PET (left, solid) and synthetic PET (right, dashed). Solid and dashed lines indicate significance for real and synthetic PET, respectively. (c,d) Disease-related regions for AV45 and FDG; (e,f) non–disease-related regions. Cblm-WM: Cerebellum white matter. See Extended Data for additional analyses.
Figure 5: Diagnostic classification performance using MRI-synthetic PET images across different clinical tasks. (a–c) Bar plots illustrate the classification performance—accuracy (ACC), sensitivity (SEN), specificity (SPE), and area under the ROC curve (AUC)—of eleven input-modality combinations evaluated for three diagnostic tasks: (a) Alzheimer’s disease (AD) vs. cognitively normal (CN), (b) mild cognitive impairment (MCI) vs. CN and (c) early MCI (EMCI) vs. late MCI (LMCI). The input modalities include: MRI only, real FDG-PET, synthetic FDG-PET, real AV45-PET, synthetic AV45-PET, MRI + real FDG-PET, MRI + synthetic FDG-PET, MRI + real AV45-PET, MRI + synthetic AV45-PET, MRI + real FDG + real AV45, and MRI + synthetic FDG + synthetic AV45. For visual clarity, bars corresponding to synthetic PET inputs are shown with the same colour scheme as their real PET counterparts but with dashed (hatched) fills. The results demonstrate that MRI-synthetic PET images achieve comparable diagnostic accuracy to real PET scans, highlighting their clinical utility for downstream classification tasks.
Lab for Artificial Intelligence in Medical Imaging, Institute for Diagnostic and Interventional Radiology, School of Medicine and Health, TUM Klinikum, Technical University of Munich (TUM), Munich, 81675, Germany · Munich Center for Machine Learning (MCML), Munich, Germany · Department of Nuclear Medicine, School of Medicine and Health, TUM Klinikum, Munich, 81675, Germany +1
Technical University of Munich (TUM), TUM University Hospital, and Munich Center for Machine Learning (MCML), Germany · Department of Neurology, TUM University Hospital, Germany · Department of Neuroradiology, TUM University Hospital, Germany +1
Department of Radiology and Biomedical Research Imaging Center (BRIC), University of North Carolina at Chapel Hill, Chapel Hill, NC 27599, USA. · Joint Department of Biomedical Engineering, University of North Carolina at Chapel Hill and North Carolina State University, Chapel Hill, NC 27599, USA. · Department of Exercise and Sport Science, UNC-CH, Chapel Hill, NC 27599, USA.