CardioDiT: Latent Diffusion Transformers for 4D Cardiac MRI Synthesis

Marvin Seyfarth; Sarah Kaye Müller; Arman Ghanaat; Isabelle Ayx; Fabian Fastenrath; Philipp Wild; Alexander Hertel; Theano Papavassiliu

cs.CVMar 26, 2026

CardioDiT: Latent Diffusion Transformers for 4D Cardiac MRI Synthesis

Authors: Marvin Seyfarth, Sarah Kaye Müller, Arman Ghanaat, Isabelle Ayx, Fabian Fastenrath, Philipp Wild, Alexander Hertel, Theano Papavassiliu, +2 more

Organizations: Institute for Artificial Intelligence in Cardiovascular Medicine, Medical Faculty of Heidelberg University, Heidelberg University, Germany · Department of Cardiology, Angiology, Pneumology, Heidelberg University Hospital, Heidelberg, Germany · DZHK (German Centre for Cardiovascular Research), Partner Site Heidelberg/Mannheim, Heidelberg, Germany · Department of Radiology and Nuclear Medicine, University Medical Center Mannheim, Germany · Section for Invasive Cardiology and Electrophysiology, I. Medical Department, Cardiology, Hemostasiology and Intensive Care, University Medical Centre Mannheim, Mannheim, Germany · University Medical Center of the Johannes Gutenberg-University Mainz, Germany · Department of Cardiology, Angiology, Hemostasis, and Medical Intensive Care, University Medical Centre Mannheim, Medical Faculty Mannheim, University of Heidelberg, Germany

Abstract

Latent diffusion models (LDMs) have recently achieved strong performance in 3D medical image synthesis. However, modalities like cine cardiac MRI (CMR), representing a temporally synchronized 3D volume across the cardiac cycle, add an additional dimension that most generative approaches do not model directly. Instead, they factorize space and time or enforce temporal consistency through auxiliary mechanisms such as anatomical masks. Such strategies introduce structural biases that may limit global context integration and lead to subtle spatiotemporal discontinuities or physiologically inconsistent cardiac dynamics. We investigate whether a unified 4D generative model can learn continuous cardiac dynamics without architectural factorization. We propose CardioDiT, a fully 4D latent diffusion framework for short-axis cine CMR synthesis based on diffusion transformers. A spatiotemporal VQ-VAE encodes 2D+t slices into compact latents, which a diffusion transformer then models jointly as complete 3D+t volumes, coupling space and time throughout the generative process. We evaluate CardioDiT on public CMR datasets and a larger private cohort, comparing it to baselines with progressively stronger spatiotemporal coupling. Results show improved inter-slice consistency, temporally coherent motion, and realistic cardiac function distributions, suggesting that explicit 4D modeling with a diffusion transformer provides a principled foundation for spatiotemporal cardiac image synthesis. Code and models trained on public data are available at https://github.com/Cardio-AI/cardiodit.

Figures & tables

Figure 1: CardioDiT framework for 4D CMR synthesis. A VQ-GAN encodes 2D+t slices into latents stacked along $d$ to form a 4D representation. A 4D Diffusion Transformer denoises the noisy latent, followed by VQ-GAN decoding to reconstruct the full volume.

Figure 2: Visualized slices across time and depth for one synthetic sample per model, with corresponding LV meshes. Noteworthy are strong inter-slice discontinuities for the 2D+t LDM, blurry anatomical boundaries for the 3D+t U-Net, and anatomical consistency for CardioDiT.

Model	FID $\downarrow$	Precision $\uparrow$	Recall $\uparrow$	$d$ -SSIM $\sim$	AR ${}_{\text{ED}}$ $\sim$	$\overline{EF}$ (Std) $\sim$	$W_{2}$ $\downarrow$
Public Dataset
Real	52.8	0.95	0.96	0.55 (0.10)	2.1 (1.0)	47.6 (17.1)	3.09
Fact. 2D+t LDM	100.1	0.89	0.17	0.39 (0.04)	5.1 (1.1)	45.3 (8.0) ⋆	8.90
3D+t U-Net	221.5	0.59	0.64	0.75 (0.10)	1.0 (0.3)	44.5 (10.0)	7.14
CardioDiT	71.9	0.97	0.70	0.69 (0.10)	1.3 (0.5)	49.3 (15.1)	2.67
Private Dataset

Table 1: Quantitative comparison of CMR generation models using distributional (FID, Precision, Recall), structural (

d

-SSIM, AR

{}_{\text{ED}}

) and clinical EF distribution statistics. Lower is better

\downarrow

, higher is better

\uparrow

, closest to real is better

\sim

Figure 3: Normalized left ventricular volume-time curves derived from cine CMR blood pool segmentation, temporally aligned to peak-to-peak volume and scaled to equal length $T$ . The mean curve (dark blue), inter-quartile range (IQR, light blue), and individual subject curves (light gray) are shown for the studied generative models.

Model	FID $\downarrow$	$d$ -SSIM $\sim$	$\overline{EF}$ (Std) $\sim$	$W_{2}$ $\downarrow$	#Par (M)	T(s) $\downarrow$	VRAM(GB) $\downarrow$
CardioDiT-S	30.5	0.68(0.06)	41.8 (7.7)	3.56	92.23	30	1.2
CardioDiT-B	25.9	0.69(0.06)	41.4 (4.7)	4.13	134.74	45	1.3
CardioDiT-L	21.2	0.69(0.09)	42.3 (6.3)	3.05	177.24	60	1.5
Fact. 2D+t	32.9	0.50(0.04)	41.1 (6.5)	3.51	166	20	1.5
3D+t	51.3	0.73(0.03)	38.1 (5.1)	7.23	578	15	3.8

Table 2: Scalability of CardioDiT. Inference time

T

, VRAM and number of parameters were measured for a single sample on a NVIDIA RTX 4090 GPU.

Explore similar work

Anatomy-Guided Residual Motion Diffusion for Controllable 4D Cardiac MRI Synthesis
Jun 25, 2026Yiheng Cao, Gustavo Andrade-Miranda, Jiatian Zhang +2Synthetic Data Augmentation Medical Image Synthesis
Suzhou Institute of Biomedical Engineering and Technology, Chinese Academy of Science, Suzhou, China · Systemes Complexes et Intelligence Artificielle, IMT Mines Ales, Ales, France · School of Artificial Intelligence and Advanced Computing, Xi’an Jiaotong-Liverpool University, Suzhou, China
Temporally Consistent and Controllable Video Generation of 2D Cine CMR via Latent Space Motion Modeling
Jun 8, 2026Yiheng Cao, Gustavo Andrade-Miranda, Jiatian Zhang +2Latent Dynamics Modeling Text-to-Video Generation
SyCoIA - IMT Mines Alès · Suzhou Institute of Biomedical Engineering and Technology, Chinese Academy of Sciences, Suzhou, China · SyCoIA, IMT Mines Ales, Ales, France
Synthetic Cardiac MRI Image Generation using Deep Generative Models
Mar 25, 2026Ishan Kumarasinghe, Dasuni Kawya, Madhura Edirisooriya +3Synthetic Data Generation Medical Image Synthesis
Department of Computer Engineering, University of Peradeniya, Peradeniya, Sri Lanka · Department of Computer Engineering, University of Maryland, College Park, USA · Department of Holistic Systems, SimulaMet, Oslo, Norway

Jun 25, 2026J/K move · Enter open · S save

Anatomy-Guided Residual Motion Diffusion for Controllable 4D Cardiac MRI Synthesis

Yiheng Cao, Gustavo Andrade-Miranda, Jiatian Zhang, Lingxiao Zhao+1

Suzhou Institute of Biomedical Engineering and Technology, Chinese Academy of Science, Suzhou, China · Systemes Complexes et Intelligence Artificielle, IMT Mines Ales, Ales, France · School of Artificial Intelligence and Advanced Computing, Xi’an Jiaotong-Liverpool University, Suzhou, China

Synthetic Data Augmentation Medical Image Synthesis Cardiac MRI Diffusion Models

Open paper AskPDF arXiv

Developing robust artificial intelligence models for 4D (3D + time) medical imaging is constrained by limited annotated data, inter-device domain shifts, and privacy restrictions. To address this, we propose a 4D controllable generative framework for anatomically consistent data augmentation. A semi-supervised variational autoencoder learns a compact latent representation of anatomical volumes while jointly predicting aligned segmentation masks in a unified framework. Anatomical structure is then disentangled from temporal dynamics through a cascaded latent diffusion model (LDM). A static LDM generates subject-specific anatomy conditioned on clinical priors (diagnosis and volumes measures) and a subsequent motion LDM estimates residual latent motions, ensuring strict temporal coherence across the 4D sequence. The proposed approach was evaluated on cine cardiac MRI as a representative 4D imaging application. Experiments across multiple datasets demonstrate high controllability of static anatomy (Pearson r > 0.8) and strong temporal coherence (FVD = 288.08). In cross-vendor generalization experiments, augmenting training sets with synthetic 4D sequences significantly improves downstream segmentation performance. Using nnU-Net, the proposed augmentation strategy improves the average Dice score by 1.4% and reduces the Hausdorff Distance by 3.0mm compared to training on real data alone, for the left ventricle, Dice improves by 2.8% with a 5.4mm reduction in boundary error. Overall, this framework provides a scalable and controllable solution for 4D medical image synthesis, supporting the development of more robust models with limited annotations and cross-vendor variability. Code available on https://github.com/cyiheng/4DCardiacMRISynthesis.

Abstract

Explore similar work

Anatomy-Guided Residual Motion Diffusion for Controllable 4D Cardiac MRI Synthesis

Temporally Consistent and Controllable Video Generation of 2D Cine CMR via Latent Space Motion Modeling

Synthetic Cardiac MRI Image Generation using Deep Generative Models