Adaptive Clinical-Aware Latent Diffusion for Multimodal Brain Image Generation and Missing Modality Imputation
Authors: Rong Zhou, Houliang Zhou, Yao Su, Brian Y. Chen, Yu Zhang, Lifang He, Alzheimer's Disease Neuroimaging Initiative
Organizations: Department of Computer Science and Engineering, Lehigh University, PA, USA · Worcester Polytechnic Institute, MA, USA · Department of Psychiatry and Behavioral Sciences, Stanford University School of Medicine, CA, USA · Wu Tsai Neurosciences Institute, Stanford University, CA, USA · Stanford Institute for Human-Centered AI, Stanford, CA, USA · Alzheimer’s Disease Neuroimaging Initiative
Multimodal neuroimaging provides complementary insights for Alzheimer's disease diagnosis, yet clinical datasets frequently suffer from missing modalities. We propose ACADiff, a framework that synthesizes missing brain imaging modalities through adaptive clinical-aware diffusion. ACADiff learns mappings between incomplete multimodal observations and target modalities by progressively denoising latent representations while attending to available imaging data and clinical metadata. The framework employs adaptive fusion that dynamically reconfigures based on input availability, coupled with semantic clinical guidance via GPT-4o-encoded prompts. Three specialized generators enable bidirectional synthesis among sMRI, FDG-PET, and AV45-PET. Evaluated on ADNI subjects, ACADiff achieves superior generation quality and maintains robust diagnostic performance even under extreme 80% missing scenarios, outperforming all existing baselines. To promote reproducibility, code is available at https://github.com/rongzhou7/ACADiff
Figures & tables
Figure 1: Overview of ACADiff. (a) Cross-modal latent diffusion with adaptive multi-source image conditioning and semantic clinical guidance via GPT-4o-encoded prompts. (b) Missing modality imputation for downstream diagnosis.
Figure 2: Image generation performance across methods. Higher PSNR/SSIM/NMI and lower MAE indicate better quality.
Missing Rate
Method
ACC
AUC
SEN
SPE
0%
Oracle (Real)
0.920±0.029
0.943±0.023
0.889±0.031
0.908±0.028
20%
Drop
0.825±0.042
0.852±0.040
0.788±0.043
0.806±0.041
Mean
0.798±0.045
0.826±0.043
0.761±0.046
0.779±0.044
Pix2Pix
0.865±0.035
0.893±0.032
0.821±0.035
0.848±0.035
DS-GAN
0.851±0.039
0.882±0.037
0.805±0.039
0.786±0.038
LDM
0.885±0.036
0.902±0.031
0.817±0.033
0.861±0.038
Table 1: Classification performance for AD vs. HC under different missing data strategies.
Department of Biomedical Engineering, The Hong Kong Polytechnic University, Hong Kong SAR, China · Department of Technology Management for Innovation, The University of Tokyo,2026 Japan · Department of Data Science and Artificial Intelligence, The Hong Kong Polytechnic University, Hong Kong SAR, China