CardioDiT: Latent Diffusion Transformers for 4D Cardiac MRI Synthesis
Authors: Marvin Seyfarth, Sarah Kaye Müller, Arman Ghanaat, Isabelle Ayx, Fabian Fastenrath, Philipp Wild, Alexander Hertel, Theano Papavassiliu, +2 more
Organizations: Institute for Artificial Intelligence in Cardiovascular Medicine, Medical Faculty of Heidelberg University, Heidelberg University, Germany · Department of Cardiology, Angiology, Pneumology, Heidelberg University Hospital, Heidelberg, Germany · DZHK (German Centre for Cardiovascular Research), Partner Site Heidelberg/Mannheim, Heidelberg, Germany · Department of Radiology and Nuclear Medicine, University Medical Center Mannheim, Germany · Section for Invasive Cardiology and Electrophysiology, I. Medical Department, Cardiology, Hemostasiology and Intensive Care, University Medical Centre Mannheim, Mannheim, Germany · University Medical Center of the Johannes Gutenberg-University Mainz, Germany · Department of Cardiology, Angiology, Hemostasis, and Medical Intensive Care, University Medical Centre Mannheim, Medical Faculty Mannheim, University of Heidelberg, Germany
Latent diffusion models (LDMs) have recently achieved strong performance in 3D medical image synthesis. However, modalities like cine cardiac MRI (CMR), representing a temporally synchronized 3D volume across the cardiac cycle, add an additional dimension that most generative approaches do not model directly. Instead, they factorize space and time or enforce temporal consistency through auxiliary mechanisms such as anatomical masks. Such strategies introduce structural biases that may limit global context integration and lead to subtle spatiotemporal discontinuities or physiologically inconsistent cardiac dynamics. We investigate whether a unified 4D generative model can learn continuous cardiac dynamics without architectural factorization. We propose CardioDiT, a fully 4D latent diffusion framework for short-axis cine CMR synthesis based on diffusion transformers. A spatiotemporal VQ-VAE encodes 2D+t slices into compact latents, which a diffusion transformer then models jointly as complete 3D+t volumes, coupling space and time throughout the generative process. We evaluate CardioDiT on public CMR datasets and a larger private cohort, comparing it to baselines with progressively stronger spatiotemporal coupling. Results show improved inter-slice consistency, temporally coherent motion, and realistic cardiac function distributions, suggesting that explicit 4D modeling with a diffusion transformer provides a principled foundation for spatiotemporal cardiac image synthesis. Code and models trained on public data are available at https://github.com/Cardio-AI/cardiodit.
Figures & tables
Figure 1: CardioDiT framework for 4D CMR synthesis. A VQ-GAN encodes 2D+t slices into latents stacked along d to form a 4D representation. A 4D Diffusion Transformer denoises the noisy latent, followed by VQ-GAN decoding to reconstruct the full volume.
Figure 2: Visualized slices across time and depth for one synthetic sample per model, with corresponding LV meshes. Noteworthy are strong inter-slice discontinuities for the 2D+t LDM, blurry anatomical boundaries for the 3D+t U-Net, and anatomical consistency for CardioDiT.
Model
FID ↓
Precision ↑
Recall ↑
d -SSIM ∼
AR ED∼
EF (Std) ∼
W2↓
Public Dataset
Real
52.8
0.95
0.96
0.55 (0.10)
2.1 (1.0)
47.6 (17.1)
3.09
Fact. 2D+t LDM
100.1
0.89
0.17
0.39 (0.04)
5.1 (1.1)
45.3 (8.0) ⋆
8.90
3D+t U-Net
221.5
0.59
0.64
0.75 (0.10)
1.0 (0.3)
44.5 (10.0)
7.14
CardioDiT
71.9
0.97
0.70
0.69 (0.10)
1.3 (0.5)
49.3 (15.1)
2.67
Private Dataset
Table 1: Quantitative comparison of CMR generation models using distributional (FID, Precision, Recall), structural ( d -SSIM, AR ED ) and clinical EF distribution statistics. Lower is better ↓ , higher is better ↑ , closest to real is better ∼ .
Figure 3: Normalized left ventricular volume-time curves derived from cine CMR blood pool segmentation, temporally aligned to peak-to-peak volume and scaled to equal length T . The mean curve (dark blue), inter-quartile range (IQR, light blue), and individual subject curves (light gray) are shown for the studied generative models.
Model
FID ↓
d -SSIM ∼
EF (Std) ∼
W2↓
#Par (M)
T(s) ↓
VRAM(GB) ↓
CardioDiT-S
30.5
0.68(0.06)
41.8 (7.7)
3.56
92.23
30
1.2
CardioDiT-B
25.9
0.69(0.06)
41.4 (4.7)
4.13
134.74
45
1.3
CardioDiT-L
21.2
0.69(0.09)
42.3 (6.3)
3.05
177.24
60
1.5
Fact. 2D+t
32.9
0.50(0.04)
41.1 (6.5)
3.51
166
20
1.5
3D+t
51.3
0.73(0.03)
38.1 (5.1)
7.23
578
15
3.8
Table 2: Scalability of CardioDiT. Inference time T , VRAM and number of parameters were measured for a single sample on a NVIDIA RTX 4090 GPU.
Developing robust artificial intelligence models for 4D (3D + time) medical imaging is constrained by limited annotated data, inter-device domain shifts, and privacy restrictions. To address this, we propose a 4D controllable generative framework for anatomically consistent data augmentation. A semi-supervised variational autoencoder learns a compact latent representation of anatomical volumes while jointly predicting aligned segmentation masks in a unified framework. Anatomical structure is then disentangled from temporal dynamics through a cascaded latent diffusion model (LDM). A static LDM generates subject-specific anatomy conditioned on clinical priors (diagnosis and volumes measures) and a subsequent motion LDM estimates residual latent motions, ensuring strict temporal coherence across the 4D sequence. The proposed approach was evaluated on cine cardiac MRI as a representative 4D imaging application. Experiments across multiple datasets demonstrate high controllability of static anatomy (Pearson r > 0.8) and strong temporal coherence (FVD = 288.08). In cross-vendor generalization experiments, augmenting training sets with synthetic 4D sequences significantly improves downstream segmentation performance. Using nnU-Net, the proposed augmentation strategy improves the average Dice score by 1.4% and reduces the Hausdorff Distance by 3.0mm compared to training on real data alone, for the left ventricle, Dice improves by 2.8% with a 5.4mm reduction in boundary error. Overall, this framework provides a scalable and controllable solution for 4D medical image synthesis, supporting the development of more robust models with limited annotations and cross-vendor variability. Code available on https://github.com/cyiheng/4DCardiacMRISynthesis.
Suzhou Institute of Biomedical Engineering and Technology, Chinese Academy of Science, Suzhou, China · Systemes Complexes et Intelligence Artificielle, IMT Mines Ales, Ales, France · School of Artificial Intelligence and Advanced Computing, Xi’an Jiaotong-Liverpool University, Suzhou, China
Cine cardiac magnetic resonance is the gold standard for assessing cardiac function, but the scarcity of public datasets limits the development of advanced data-driven models. To address this limitation, we propose a generative method for synthesizing temporally coherent and anatomically consistent cardiac sequences. Our text-to-video framework decouples cardiac spatial structure from temporal motion. First, a fine-tuned diffusion model synthesizes an initial frame from a clinical text prompt, controlling anatomical features. Then, a latent flow model conditioned on a cardiac phase embedding generates the complete cardiac motion, ensuring spatial consistency and temporal control. Our model generates anatomically and pathologically diverse sequences with high temporal coherence and strong fidelity to input prompts, achieving a FID of 31.68 for image realism and a CLIP score of 31.04 for text-image alignment. These experimental results highlight its potential to produce high-fidelity, on-demand medical data, offering a scalable solution to data scarcity.
SyCoIA - IMT Mines Alès · Suzhou Institute of Biomedical Engineering and Technology, Chinese Academy of Sciences, Suzhou, China · SyCoIA, IMT Mines Ales, Ales, France
Synthetic cardiac MRI (CMRI) generation has emerged as a promising strategy to overcome the scarcity of annotated medical imaging data. Recent advances in GANs, VAEs, diffusion probabilistic models, and flow-matching techniques aim to generate anatomically accurate images while addressing challenges such as limited labeled datasets, vendor variability, and risks of privacy leakage through model memorization. Maskconditioned generation improves structural fidelity by guiding synthesis with segmentation maps, while diffusion and flowmatching models offer strong boundary preservation and efficient deterministic transformations. Cross-domain generalization is further supported through vendor-style conditioning and preprocessing steps like intensity normalization. To ensure privacy, studies increasingly incorporate membership inference attacks, nearest-neighbor analyses, and differential privacy mechanisms. Utility evaluations commonly measure downstream segmentation performance, with evidence showing that anatomically constrained synthetic data can enhance accuracy and robustness across multi-vendor settings. This review aims to compare existing CMRI generation approaches through the lenses of fidelity, utility, and privacy, highlighting current limitations and the need for integrated, evaluation-driven frameworks for reliable clinical workflows.
Department of Computer Engineering, University of Peradeniya, Peradeniya, Sri Lanka · Department of Computer Engineering, University of Maryland, College Park, USA · Department of Holistic Systems, SimulaMet, Oslo, Norway