Exploiting Completeness Perception with Diffusion Transformer for Unified 3D MRI Synthesis

Junkai Liu; Nay Aung; Theodoros N. Arvanitis; Joao A. C. Lima; Steffen E. Petersen; Le Zhang

eess.IVFeb 20, 2026

Exploiting Completeness Perception with Diffusion Transformer for Unified 3D MRI Synthesis

Authors: Junkai Liu, Nay Aung, Theodoros N. Arvanitis, Joao A. C. Lima, Steffen E. Petersen, Le Zhang

Organizations: School of Engineering, University of Birmingham, UK · William Harvey Research Institute, Queen Mary University London, UK · Barts Heart Centre, St Bartholomew’s Hospital, Barts Health NHS Trust, UK · Division of Cardiology, Johns Hopkins University School of Medicine, US

Abstract

Missing data problems, such as missing modalities in multi-modal brain MRI and missing slices in cardiac MRI, pose significant challenges in clinical practice. Existing methods rely on external guidance to supply detailed missing-state information for instructing generative models to synthesize missing MRIs. However, manual indicators are not always available or reliable in real-world scenarios due to the unpredictable nature of clinical environments. Moreover, these explicit masks are not informative enough to provide guidance for improving semantic consistency. In this work, we argue that generative models should infer and recognize missing states in a self-perceptive manner, enabling them to better capture subtle anatomical and pathological variations. Towards this goal, we propose CoPeDiT, a shared completeness-perception framework for 3D MRI synthesis, following a common conditioning strategy with task-specific instantiations for different missing-data scenarios. Specifically, we incorporate dedicated pretext tasks into our tokenizer, CoPeVAE, empowering it to learn completeness-aware discriminative prompt tokens, and design MDiT3D, a specialized diffusion transformer architecture for 3D MRI synthesis that effectively uses the completeness-aware prompt tokens as guidance to enhance semantic consistency in 3D space. Comprehensive evaluations on three large-scale MRI datasets demonstrate that CoPeDiT consistently improves upon state-of-the-art methods across diverse missing patterns, yielding high-fidelity and structurally consistent MRI synthesis. Our code is available at https://github.com/JK-Liu7/CoPeDiT.

Figures & tables

Panel of Figure 1 — Figure 1: (a) We shift the paradigm from mask-dependent guidance to autonomous completeness perception via prompt token generation. This self-perceptive mechanism offers significant advantages: (b) quantitative plug-and-play performance gains across existing baselines; (c) substantially more discriminative latent representations compared to training without pretext tasks; and (d) enhanced semantic attention alignment between correlated modalities (e.g., T1-T1ce, T2-FLAIR) compared to traditional binary mask codes.

Figure 2: The overview framework of CoPeVAE. We implement two variants, CoPeVAE-B and CoPeVAE-C, with slight architectural modifications for brain and cardiac MRI synthesis tasks, respectively.

Figure 3: Architecture of the MDiT3D framework. We implement two variants with alternating blocks (Spatial/Modal for MDiT3D-B and Planar/Spatial for MDiT3D-C), using dynamic feature reshaping to model respective anatomical dependencies. Learned prompt tokens are injected via adaLN as conditional guidance. PE: Positional Embeddings; RoPE: Rotary Position Embeddings Su et al. (2024) ; Yang et al. (2025) .

PSNR

\uparrow

SSIM

\uparrow

MAE

\downarrow

FID

\downarrow

FVD

\downarrow

PSNR

\uparrow

SSIM

\uparrow

MAE

\downarrow

FID

\downarrow

FVD

\downarrow

PSNR

\uparrow

SSIM

\uparrow

MAE

\downarrow

FID

\downarrow

FVD

\downarrow

GAN-based Methods

MMGAN Sharma and Hamarneh (2020)

24.71

\pm

1.57

0.817

\pm

0.027

0.089

\pm

0.023

27.94

489.86

24.38

\pm

1.74

0.806

\pm

0.023

0.093

\pm

0.029

32.48

726.93

24.06

\pm

1.92

0.794

\pm

0.031

0.102

\pm

0.035

39.37

898.17

MMT Liu et al. (2023)

25.19

\pm

1.41

0.824

\pm

0.017

0.089

\pm

0.027

24.53

527.06

24.50

\pm

1.55

0.811

\pm

0.020

0.092

\pm

0.025

29.57

686.52

23.92

\pm

1.53

0.801

\pm

0.021

0.099

\pm

0.038

39.66

841.46

Hyper-GAE Yang et al. (2023)

24.65

\pm

1.62

0.813

\pm

0.022

0.087

\pm

0.018

28.97

609.65

24.42

\pm

1.74

0.808

\pm

0.024

0.096

\pm

0.021

33.52

815.92

23.86

\pm

1.88

0.788

\pm

0.029

0.105

\pm

0.029

41.79

948.27

Diffusion Model-based Methods

Table 1: Quantitative results for multi-modal brain MRI synthesis on the BraTS dataset. The numbers in the first row denote the number of missing modalities.

	1					2
	PSNR $\uparrow$	SSIM $\uparrow$	MAE $\downarrow$	FID $\downarrow$	FVD $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	MAE $\downarrow$	FID $\downarrow$	FVD $\downarrow$
GAN-based Methods
MMGAN Sharma and Hamarneh (2020)	22.29 $\pm$ 1.35	0.684 $\pm$ 0.015	0.100 $\pm$ 0.027	70.91	1447.83	21.13 $\pm$ 1.49	0.668 $\pm$ 0.019	0.107 $\pm$ 0.034	93.57	1787.39
MMT Liu et al. (2023)	22.64 $\pm$ 1.49	0.698 $\pm$ 0.013	0.098 $\pm$ 0.035	53.60	1329.25	21.82 $\pm$ 1.60	0.687 $\pm$ 0.019	0.102 $\pm$ 0.029	72.24	1562.44
Hyper-GAE Yang et al. (2023)	22.12 $\pm$ 1.33	0.682 $\pm$ 0.017	0.100 $\pm$ 0.039	72.62	1520.49	20.91 $\pm$ 1.45	0.662 $\pm$ 0.021	0.109 $\pm$ 0.042	98.79	1712.90
Diffusion Model-based Methods

Table 2: Quantitative results for multi-modal brain MRI synthesis on the IXI dataset . The numbers in the first row denote the number of missing modalities.

PSNR

\uparrow

SSIM

\uparrow

MAE

\downarrow

FID

\downarrow

FVD

\downarrow

PSNR

\uparrow

SSIM

\uparrow

MAE

\downarrow

FID

\downarrow

FVD

\downarrow

PSNR

\uparrow

SSIM

\uparrow

MAE

\downarrow

FID

\downarrow

FVD

\downarrow

GAN-based Methods

MMGAN Sharma and Hamarneh (2020)

25.81

\pm

0.86

0.815

\pm

0.012

0.076

\pm

0.012

16.68

392.48

24.35

\pm

0.84

0.793

\pm

0.009

0.080

\pm

0.013

27.52

604.37

23.06

\pm

1.08

0.776

\pm

0.018

0.091

\pm

0.016

45.88

767.92

MMT Liu et al. (2023)

26.02

\pm

0.82

0.824

\pm

0.009

0.075

\pm

0.016

15.90

429.72

24.73

\pm

0.87

0.809

\pm

0.010

0.080

\pm

0.015

23.75

570.29

24.12

\pm

1.06

0.794

\pm

0.017

0.086

\pm

0.015

37.39

708.68

Hyper-GAE Yang et al. (2023)

25.23

\pm

0.89

0.810

\pm

0.012

0.078

\pm

0.014

19.02

517.55

23.66

\pm

0.94

0.789

\pm

0.013

0.086

\pm

0.015

31.84

648.35

22.70

\pm

1.14

0.771

\pm

0.019

0.094

\pm

0.019

48.27

845.14

Diffusion Model-based Methods

Table 3: Quantitative results for Cardiac MRI synthesis on the UKBB dataset. The numbers in the first row denote the length of missing slices.

Figure 4: Qualitative comparison on the BraTS dataset. Rows (a)–(d) correspond to synthesizing FLAIR, T1, T1ce, and T2, respectively, from the missing modalities shown in the first column. For each case, we show three views, zoomed-in regions marked by green boxes, and the corresponding error maps.

	BraTS															UKBB
	1					2					3					8					24
	PSNR $\uparrow$	SSIM $\uparrow$	MAE $\downarrow$	FID $\downarrow$	FVD $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	MAE $\downarrow$	FID $\downarrow$	FVD $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	MAE $\downarrow$	FID $\downarrow$	FVD $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	MAE $\downarrow$	FID $\downarrow$	FVD $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	MAE $\downarrow$	FID $\downarrow$	FVD $\downarrow$
w/o $\mathbf{p}^{d}$	27.35	0.833	0.060	16.04	335.08	27.02	0.824	0.064	17.30	475.32	26.73	0.814	0.069	19.62	556.79	25.82	0.819	0.076	16.34	427.06	24.53	0.803	0.085	33.80	610.46
w/o $\mathbf{p}^{p}$	26.92	0.829	0.063	18.46	372.36	26.43	0.819	0.067	20.23	460.45	26.06	0.810	0.073	25.13	572.03	25.17	0.812	0.078	20.08	534.80	24.39	0.798	0.086	36.54	697.51
w/o $\mathbf{p}^{s}$	27.56	0.835	0.058	15.26	341.70	27.22	0.827	0.060	16.37	395.43	26.90	0.815	0.065	17.58	468.23	26.27	0.828	0.074	16.79	388.46	25.08	0.813	0.080	29.83	578.13
w/o Prompt Tokens	25.92	0.823	0.069	25.69	418.12	25.06	0.807	0.080	32.17	556.92	24.83	0.802	0.089	37.97	748.25	24.70	0.797	0.083	23.29	721.35	23.56	0.778	0.090	42.17	830.72

Table 4: Ablation study on the contribution of completeness-aware prompt tokens .

Table 9

Table 11

Panel of Figure 6 — Figure 6: Qualitative analysis of the learned prompt tokens on the BraTS dataset. (a) t-SNE projection of the count-focused prompt token $\mathbf{p}^{d}$ , showing clear clustering by the number of missing modalities. (b) t-SNE projection of the identity-focused prompt token $\mathbf{p}^{p}$ , revealing distinct separation by missing modality types. (c) Visualization of modal block attention maps under a missing configuration of [1, 0, 0, 0].

Panel of Figure 7 — Figure 7: Quantitative analysis of completeness-aware prompt tokens on the BraTS dataset. (a) Sensitivity of CoPeDiT to increasing prompt token corruption rates. Synthesis quality degrades monotonically, particularly when perturbing the position prompt token $\mathbf{p}^{p}$ . (b) Prompt token intervention matrix illustrating the PSNR drop under structured perturbations. Misleading signals (e.g., random, type-swap) and the simultaneous perturbation of all prompt tokens cause the most severe performance degradation.

Appendix figures & tables20 assets

Supplementary material from the paper’s appendix.

Appendix

Datasets	Modality	Cases	Train	Test
Brain MRI
BraTS Baid et al. (2021)	T1, T1ce, T2, FLAIR	1251	1000	251
IXI Brain Development Project (2025)	T1, T2, PD	577	462	115
Cardiac MRI
UKBB Petersen et al. (2016)	-	31350	25080	6270
MESA Zhang et al. (2018)	-	298	238	60

Appendix

Table 10: Details of brain and cardiac MRI datasets.

Prompt token encoder	$\mathcal{F}_{1}$	$\mathcal{F}_{2}$	$\mathcal{F}_{3}$
Architecture	3D Conv (in 8, out 256) 3D BatchNorm ReLU 3D Conv (in 256, out 512) 3D BatchNorm ReLU 3D Adaptive Avg Pool Linear (512, 1024) ReLU Linear (1024, 512)	3D Conv (in 8, out 256) 3D BatchNorm ReLU 3D Conv (in 256, out 512) 3D BatchNorm ReLU 3D Adaptive Avg Pool Linear (512, 1024) ReLU Linear (1024, 512)	3D Conv (in 8, out 256) 3D BatchNorm ReLU 3D Conv (in 256, out 512) 3D BatchNorm ReLU 3D Adaptive Avg Pool Linear (512, 1024) ReLU Linear (1024, 512)
Projection Head	$\mathcal{H}_{1}$	$\mathcal{H}_{2}$	$\mathcal{H}_{3}$
Architecture	SiLU Linear (512, $m-1$ )	SiLU Linear (512, $m$ )	SiLU Linear (512, 128)

Appendix

Table 11: Detailed architecture of each prompt token encoder and projection head in the pretext task.

	CoPeVAE-B	CoPeVAE-C
Architecture
Input dim.	$m\times 192\times 192\times 64$	$192\times 192\times 32$
Num. codebook	8192	8192
Latent dim.	8	8
Channels	(256, 384, 512)	(32, 64, 128)
Compression ratio	(8, 8, 8)	(8, 8, 8)

Appendix

Table 12: Hyperparameter setup of CoPeVAE.

	MDiT3D-B	MDiT3D-C
Architecture
Input dim.	$m\times 8\times 24\times 24\times 8$	$4\times 8\times 24\times 24$
Hidden dim.	768	576
Num. blocks	16	12
Num. heads	12	12
Patch size	2	1

Appendix

Table 13: Hyperparameter setup of MDiT3D.

	1				3
	Lesion-wise Fidelity		Anatomical Consistency		Lesion-wise Fidelity		Anatomical Consistency
	MAE $\downarrow$	SSIM $\uparrow$	Dice (%) $\uparrow$	HD95 (mm) $\downarrow$	MAE $\downarrow$	SSIM $\uparrow$	Dice (%) $\uparrow$	HD95 (mm) $\downarrow$
GAN-based Methods
MMGAN Sharma and Hamarneh (2020)	0.095 $\pm$ 0.025	0.792 $\pm$ 0.036	83.91 $\pm$ 10.40	11.09 $\pm$ 4.57	0.108 $\pm$ 0.043	0.786 $\pm$ 0.035	81.67 $\pm$ 9.42	11.83 $\pm$ 5.62
MMT Liu et al. (2023)	0.092 $\pm$ 0.019	0.792 $\pm$ 0.022	83.66 $\pm$ 12.05	11.24 $\pm$ 5.08	0.098 $\pm$ 0.024	0.784 $\pm$ 0.020	81.80 $\pm$ 11.59	12.08 $\pm$ 4.38
Hyper-GAE Yang et al. (2023)	0.093 $\pm$ 0.031	0.787 $\pm$ 0.034	84.18 $\pm$ 12.43	10.74 $\pm$ 4.26	0.105 $\pm$ 0.034	0.783 $\pm$ 0.032	81.29 $\pm$ 8.96	12.27 $\pm$ 5.40

Appendix

Table 14: Lesion-wise fidelity and segmentation-based anatomical consistency on the BraTS dataset. The numbers in the first row denote the number of missing modalities. All metrics are averaged over three tumor regions, including WT, TC and ET.

Method	BraTS: 3 Missing Modalities					IXI: 2 Missing Modalities
Method	PSNR $\uparrow$	SSIM $\uparrow$	MAE $\downarrow$	FID $\downarrow$	FVD $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	MAE $\downarrow$	FID $\downarrow$	FVD $\downarrow$
M2DN Meng et al. (2024)	25.15 $\pm$ 0.39	0.808 $\pm$ 0.005	0.091 $\pm$ 0.006	32.04 $\pm$ 5.11	540.42 $\pm$ 27.22	22.83 $\pm$ 0.41	0.703 $\pm$ 0.004	0.096 $\pm$ 0.003	56.11 $\pm$ 7.65	1060.50 $\pm$ 65.95
ASLDM Zhou et al. (2026)	26.16 $\pm$ 0.21	0.811 $\pm$ 0.004	0.070 $\pm$ 0.006	25.94 $\pm$ 3.30	498.32 $\pm$ 30.56	23.29 $\pm$ 0.42	0.712 $\pm$ 0.005	0.085 $\pm$ 0.006	46.34 $\pm$ 5.76	867.91 $\pm$ 47.79
CoPeDiT	27.78 $\pm$ 0.28	0.821 $\pm$ 0.005	0.062 $\pm$ 0.004	15.07 $\pm$ 2.63	316.08 $\pm$ 23.46	23.88 $\pm$ 0.36	0.721 $\pm$ 0.006	0.073 $\pm$ 0.005	32.97 $\pm$ 4.56	729.28 $\pm$ 56.44

Appendix

Table 15: Multi-seed robustness analysis on challenging brain MRI synthesis settings. We report the mean and standard deviation over three random seeds.

Figure 8: Qualitative comparison on the IXI dataset. Rows (a)–(c) correspond to synthesizing T1, T2, and PD, respectively, from the missing modalities shown in the first column.

Figure 9: Qualitative results on the UKBB cardiac MRI dataset. The top and bottom results correspond to the first and last missing slices within a given volume, respectively. For each example, we compare synthesized results from different methods with the ground truth, together with zoomed-in regions and the corresponding error maps.

Figure 10: Qualitative results of CoPeDiT on the BraTS dataset under all missing-modality combinations. Each row corresponds to one target modality. The leftmost image in each row is the ground truth. The remaining columns show all missing scenarios for the target modality, denoted by a binary vector in the order of FLAIR, T1, T1ce, and T2, where 1/0 indicates missing/available modality, respectively.

	IXI
	1					2
	PSNR $\uparrow$	SSIM $\uparrow$	MAE $\downarrow$	FID $\downarrow$	FVD $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	MAE $\downarrow$	FID $\downarrow$	FVD $\downarrow$
w/o $\mathbf{p}^{d}$	23.56	0.720	0.089	35.93	724.18	23.04	0.708	0.095	48.71	917.76
w/o $\mathbf{p}^{p}$	22.81	0.697	0.099	50.70	1267.70	21.99	0.692	0.101	63.16	1281.31
w/o $\mathbf{p}^{s}$	24.17	0.724	0.082	31.89	685.13	23.56	0.710	0.087	39.87	876.94
w/o Prompt Tokens	22.48	0.696	0.102	57.64	1315.59	21.67	0.683	0.106	81.49	1542.24

Appendix

Table 16: Ablation study on the contribution of completeness perception prompt tokens on the IXI dataset.

Figure 11: Qualitative results of CoPeDiT on the IXI dataset under all missing-modality combinations. Each row corresponds to one target modality. The leftmost image in each row is the ground truth. The remaining columns show all missing scenarios for the target modality, denoted by a binary vector in the order of T1, T2, and PD, where 1/0 indicates missing/available modality, respectively.

	PSNR $\uparrow$	SSIM $\uparrow$	MAE $\downarrow$	FID $\downarrow$	FVD $\downarrow$
MMT Liu et al. (2023)	22.64	0.698	0.098	53.60	1329.25
+ Prompt Tokens (ours)	23.19 (+0.55)	0.707 (+0.009)	0.092 (-0.006)	46.13 (–7.47)	1096.06 (–233.19)
Hyper-GAE Yang et al. (2023)	22.12	0.682	0.100	72.62	1520.49
+ Prompt Tokens (ours)	22.46 (+0.34)	0.694 (+0.012)	0.093 (-0.007)	59.43 (–13.19)	1261.53 (–258.96)
M2DN Meng et al. (2024)	23.47	0.715	0.093	42.52	845.29
+ Prompt Tokens (ours)	23.84 (+0.37)	0.726 (+0.011)	0.083 (-0.010)	33.79 (–8.73)	684.62 (–160.67)

Appendix

Table 17: Quantitative results on the IXI dataset by incorporating our completeness perception prompt tokens into baseline methods instead of mask codes.

	BraTS															UKBB
	1					2					3					8					24
	PSNR $\uparrow$	SSIM $\uparrow$	MAE $\downarrow$	FID $\downarrow$	FVD $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	MAE $\downarrow$	FID $\downarrow$	FVD $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	MAE $\downarrow$	FID $\downarrow$	FVD $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	MAE $\downarrow$	FID $\downarrow$	FVD $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	MAE $\downarrow$	FID $\downarrow$	FVD $\downarrow$
Spatial / Planar	27.47	0.834	0.060	15.41	317.62	27.20	0.825	0.062	16.83	484.39	27.03	0.817	0.068	17.45	592.17	26.29	0.827	0.075	16.82	405.72	25.22	0.814	0.080	29.24	604.32
Both	27.76	0.836	0.056	13.94	294.37	27.54	0.827	0.059	15.48	367.20	27.29	0.818	0.064	16.77	434.83	26.33	0.827	0.073	16.39	414.37	25.28	0.815	0.080	28.45	587.13
Modal / Spatial	28.26	0.842	0.055	12.67	254.71	28.13	0.831	0.058	13.25	287.58	27.91	0.822	0.063	14.89	323.19	26.42	0.831	0.072	15.53	318.62	25.39	0.817	0.078	25.84	490.57

Appendix

Table 18: Ablation study on the choice of prompt token injection positions within the MDiT3D blocks. The labels denote the target blocks for brain and cardiac tasks, respectively (e.g., "Modal / Spatial" means injecting prompt tokens exclusively into the modal blocks for brain MRIs and spatial blocks for cardiac MRIs). "Both" indicates injection into both types of blocks for the respective tasks.

	BraTS															UKBB
	1					2					3					8					24
	PSNR $\uparrow$	SSIM $\uparrow$	MAE $\downarrow$	FID $\downarrow$	FVD $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	MAE $\downarrow$	FID $\downarrow$	FVD $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	MAE $\downarrow$	FID $\downarrow$	FVD $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	MAE $\downarrow$	FID $\downarrow$	FVD $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	MAE $\downarrow$	FID $\downarrow$	FVD $\downarrow$
UNet Rombach et al. (2022)	25.61	0.825	0.077	23.56	433.55	25.20	0.816	0.080	28.37	545.89	24.78	0.805	0.086	36.70	707.83	24.46	0.800	0.083	22.34	689.53	23.01	0.778	0.091	41.59	871.64
DiT Peebles and Xie (2023)	26.78	0.835	0.070	19.72	392.09	26.14	0.823	0.073	21.84	498.13	25.74	0.808	0.075	24.47	611.26	25.89	0.825	0.074	16.28	403.83	24.78	0.799	0.081	36.10	592.72
DiT-3D Mo et al. (2023)	26.84	0.835	0.069	19.23	359.83	26.23	0.822	0.073	21.95	487.12	25.90	0.810	0.078	26.08	593.42	26.08	0.825	0.075	16.37	397.71	24.83	0.806	0.081	33.73	610.89
MDiT3D	28.26	0.842	0.055	12.67	254.71	28.13	0.831	0.058	13.25	287.58	27.91	0.822	0.063	14.89	323.19	26.42	0.831	0.072	15.53	318.62	25.39	0.817	0.078	25.84	490.57

Appendix

Table 19: Ablation study on the contribution of 3D MRI Diffusion Transformer.

# Missing	Available Modalities				Method	Results
# Missing	T1	T1ce	T2	FLAIR	Method	PSNR $\uparrow$	SSIM $\uparrow$	MAE $\downarrow$	FID $\downarrow$	FVD $\downarrow$
1	$\checkmark$	$\checkmark$	$\checkmark$		w/o Prompt Tokens	25.49	0.820	0.070	29.56	466.08
					w/ Mask Codes	26.80	0.827	0.064	22.19	429.57
					CoPeDiT	28.10	0.839	0.057	14.19	302.16
2	$\checkmark$			$\checkmark$	w/o Prompt Tokens	24.39	0.804	0.093	35.59	724.51
					w/ Mask Codes	26.13	0.813	0.072	26.92	587.45

Appendix

Table 20: Robustness to unseen missing modality combinations on the BraTS dataset. During training, the listed missing modality combinations are excluded from both prompt token learning and diffusion training. Evaluation is conducted only on these held-out combinations.

Figure 12: Evaluation of completeness perception in CoPeVAE. We report the accuracy and macro-F1 of Task 1 and Task 2 on BraTS, IXI, and UKBB. For BraTS and IXI, the numbers denote missing modality numbers; for UKBB, they denote missing slice lengths. Task 1 evaluates missing number/length detection, while Task 2 evaluates incompleteness positioning.

Figure 13: Relationship between completeness-perception correctness and synthesis quality on the BraTS dataset . We group test samples by the correctness of the CoPeVAE count and position predictions under different missing-modality settings, and report the corresponding synthesis metrics. For each heatmap, rows denote whether the predicted missing count is correct, columns denote whether the predicted missing position is correct. Cells with no samples are marked as N/A.

Figure 14: Qualitative analysis of failure cases on the BraTS dataset. We visualize representative cases grouped by the correctness of Count prediction (Task 1) and Position prediction (Task 2). (a) When both predictions are correct, CoPeDiT produces realistic synthesis with accurate local details. (b) Incorrect position prediction causes modality-specific semantic drift despite the correct missing count. (c) Incorrect count prediction introduces additional modality bias while still covering the true missing target. (d) When both predictions are incorrect, the synthesized MRI shows the largest degradation, including contrast mismatch, blurred boundaries, and stronger reconstruction errors.

	Dice Score (%) $\uparrow$
	WT	TC	ET	AVG
Missing	83.75	82.81	83.50	83.35
GAN-based Methods
MMGAN Sharma and Hamarneh (2020)	86.90	86.27	85.26	86.14
MMT Liu et al. (2023)	87.41	85.92	84.58	85.97
Hyper-GAE Yang et al. (2023)	86.14	85.07	84.72	85.31

Appendix

Table 21: Downstream tumor segmentation results on the BraTS dataset when only one real modality is available and the remaining three modalities are synthesized. Results are averaged over four single-source settings, where T1, T1ce, T2, or FLAIR is retained as the only real input in turn.

Figure 15: Qualitative results of downstream tumor segmentation on the BraTS dataset. Rows (a)–(d) exhibit representative cases where FLAIR, T1, T1ce, and T2 are missing and replaced by synthesized modalities, respectively. We visualize the corresponding segmentation predictions and report Dice scores for WT, TC, ET, and their average.

Explore similar work

Unified Multi-plane Autoregressive Diffusion for 3D Multi-contrast MRI Synthesis
Oct 7, 2026Yejee Shin, Geonhui Son, Jinglu Wang +3Medical Image Synthesis 3D Diffusion Models
Yonsei University · Microsoft Research Asia · Korea Institute of Science and Technology
Towards Personalized Multi-Modal MRI Synthesis across Heterogeneous Datasets
Feb 23, 2026Yue Zhang, Zhizheng Zhuo, Siyao Xu +6Medical Image Synthesis Missing-Modality Learning
School of Biomedical Engineering, Division of Life Sciences and Medicine, University of Science and Technology of China, Hefei, Anhui, 230026, China · Center for Medical Imaging, Robotics, Analytic Computing & Learning (MIRACLE), Suzhou Institute for Advanced Research, University of Science and Technology of China, Suzhou, Jiangsu, 215123, China · National Key Laboratory of Intelligent Collaborative Computing, University of Electronic Science and Technology of China, Chengdu, 610097, China +6
VS-DDPM: Efficient Low-Cost Diffusion Model for Medical Modality Translation
Apr 24, 2026Nikoo Moradi, Gijs Luijten, Behrus Hinrichs-Puladi +4Image-to-Image Translation 3D Medical Imaging
Institute for Artificial Intelligence in Medicine (IKIM), Essen University Hospital (AöR), University of Duisburg-Essen, Essen, Germany · Institute of Computer Graphics and Vision, Graz University of Technology, Inffeldgasse 16, Graz, 8010, Austria · Center for Virtual and Extended Reality in Medicine (ZvRM), University Hospital Essen (AöR), Essen, Germany +7

Oct 7, 2026J/K move · Enter open · S save

Unified Multi-plane Autoregressive Diffusion for 3D Multi-contrast MRI Synthesis

Yejee Shin, Geonhui Son, Jinglu Wang, Minwoo Jung+2

Yonsei University · Microsoft Research Asia · Korea Institute of Science and Technology

Medical Image Synthesis 3D Diffusion Models Latent Diffusion Models Autoregressive Diffusion

Open paper AskPDF arXiv

Acquiring a complete set of magnetic resonance imaging (MRI) contrasts is time-intensive and uncomfortable for patients, despite the diagnostic value of multi-contrast imaging. This motivates synthesizing missing contrasts from those already acquired, which is an inherently 3D problem requiring anatomical coherence across axial, sagittal, and coronal planes. However, fully 3D generative models are often impracti- cal under computational resources that scale cubically with volume size. We propose a unified Multi-Plane Autoregressive Diffusion (MPAD), a latent diffusion framework that achieves full-volume 3D synthesis using efficient plane-wise 2D operations while preserving volumetric coherence. A 3D autoencoder first compresses MRI scans into an isotropic 3D la- tent representation. A 2D diffusion model is then trained to reconstruct masked latent slices of the target contrast, conditioned on both source- contrast slices and unmasked target-contrast slices. During inference, we introduce plane-wise autoregressive synthesis with inter-plane priors. Slices are generated autoregressively in random order within one plane orientation to maintain intra-plane continuity, then propagated as con- ditioning priors to orthogonal plane orientations to enforce inter-plane consistency. Compared to 3D latent diffusion baselines, MPAD reduces training and inference FLOPs by 7x and 3x, respectively, while also lowering inference time and peak memory consumption. Experiments on multiple datasets demonstrate that MPAD achieves superior perfor- mance, generating high-fidelity 3D volumes and supporting one-to-many translation within a single unified model.

Abstract

Explore similar work

Unified Multi-plane Autoregressive Diffusion for 3D Multi-contrast MRI Synthesis

Towards Personalized Multi-Modal MRI Synthesis across Heterogeneous Datasets

VS-DDPM: Efficient Low-Cost Diffusion Model for Medical Modality Translation