CAPS: Clinically-Verifiable Anatomy-Prioritized Synthesis for Missing MRI Imputation in Brain Tumor Segmentation

Cecilia Diana-Albelda; Arthur Longuefosse; Mario Rubén Paz Campos; Álvaro García-Martín; Jesus Bescos

eess.IVOct 20, 2026

CAPS: Clinically-Verifiable Anatomy-Prioritized Synthesis for Missing MRI Imputation in Brain Tumor Segmentation

Authors: Cecilia Diana-Albelda, Arthur Longuefosse, Mario Rubén Paz Campos, Álvaro García-Martín, Jesus Bescos

Organizations: Universidad Autónoma de Madrid, Department of Electronic Technology and Communications, Madrid, Spain · RIKEN Center for Integrative Medical Sciences, Medical Data Deep Learning Team, Tokyo, Japan · Hospital Universitario La Princesa, Diagnostic Radiology Department, Madrid, Spain

Abstract

Multi-parametric MRI is the gold standard for neuro-oncology, yet acquiring complete 4-modality protocols may be unfeasible in real-world clinical practice. When handling missing modalities, existing automated systems force a critical compromise: robust segmenters operate as uninterpretable "black boxes" that bypass image generation, while standard synthesis models often produce blurry reconstructions lacking diagnostic reliability. To address this issue, we propose CAPS, a Clinically-Verifiable Anatomy-Prioritized Synthesis framework that reconciles computational precision with clinical transparency. Our multi-modal generative architecture is driven by a hybrid objective function (L_1 + L_AFP + L_DualDice) that explicitly enforces the preservation of deep anatomical features and strict topological boundaries, rather than relying solely on pixel-wise optimization. Evaluated comprehensively on the BraTS Adult Glioma dataset, the framework achieves superior synthesis fidelity across intensity-based metrics, and demonstrates robust downstream segmentation performance (82.38% mean Dice), successfully restoring verifiable visual context. Furthermore, the framework exhibits exceptional zero-shot cross-domain generalization, maintaining high structural fidelity across completely unseen Meningioma, Pediatric, and Sub-Saharan African cohorts without domain-specific adaptation. Finally, a blinded radiological assessment confirms that our anatomy-aware approach significantly enhances both perceived image quality and clinical utility, providing clinicians with the reliable visual evidence required to trust downstream automated workflows.

Figures & tables

Figure 1: Overview of CAPS. Given three available MRI modalities ( $\mathcal{X}$ : T1n, T2w, T2f), the generator $G$ synthesizes the missing sequence ( $\hat{Y}$ , T1c in this example). The synthesized volume is then concatenated with the available modalities and fed into the clinical segmenter $S$ , which produces the predicted tumor mask ( $\hat{Z}$ ) for the three nested BraTS regions (WT, TC, ENH). Moreover, the synthesized volume is provided to a radiologist for blinded clinical validation, ensuring that the framework delivers both automated segmentation and verifiable visual context.

Figure 2: Overview of the Synthesis Framework. A trainable generator $G$ synthesizes the missing modality ( $\hat{Y}$ , T1c in this example) from the available ones ( $\mathcal{X}$ : T1n, T2w, T2f) through a hybrid objective function. First, a pixel-level $\mathcal{L}_{1}$ loss enforces global intensity consistency between the synthesized volume $\hat{Y}$ and the real one $Y$ . Then, both are processed by a frozen pre-trained auxiliary segmenter ( $S_{aux}$ ), which acts as an expert feature extractor to compute the Anatomical Feature-Prioritized ( $\mathcal{L}_{AFP}$ ) loss across multi-scale latent representations. Finally, the predicted mask ( $\hat{Z}_{aux}$ ) and the real one ( $Z$ ) are used to calculate the topological Dual Dice loss, thereby ensuring the preservation of both deep structural invariants and precise tumor boundaries.

	Dice ( $\uparrow$ )				HD95 ( $\downarrow$ )
Strategy	ENH	TC	WT	Mean	ENH	TC	WT	Mean
Reference (All Real)	81.47	86.27	92.22	86.65	21.56	9.67	4.98	12.07
Missing T1n
Zero Imputation	79.60	83.55	91.59	84.91	22.37	12.84	5.71	13.64
Trained with Dropout	79.04	86.77	92.40	86.07	30.42	8.31	4.46	14.40
Synthesis Baseline	79.89	84.97	90.87	85.24	24.48	12.60	8.06	15.05

Table 1: Quantitative Comparison of Baseline Strategies for Missing Modalities. Segmentation performance (Dice and HD95) evaluated across different missing MRI scenarios. The Reference model establishes the upper performance bound, having been trained and evaluated on the complete 4-modality protocol. For the missing scenarios, both Zero Imputation and Synthesis Baseline evaluate this exact same reference model by replacing the absent modality with zeros or a standard

\mathcal{L}_{1}

-synthesized volume, respectively. In contrast, Trained with Dropout evaluates an independent baseline model specifically trained with modality dropout augmentation to handle incomplete inputs. The best performing baseline strategy for each specific missing scenario is highlighted in bold.

	Dice ( $\uparrow$ )				HD95 ( $\downarrow$ )				IQM
Network	ENH	TC	WT	Mean	ENH	TC	WT	Mean	NMSE ( $\downarrow$ )	PSNR ( $\uparrow$ )
Reference (All Real)	81.47	86.27	92.22	86.65	21.56	9.67	4.98	12.07	-	-
nnU-net	52.27	70.27	90.03	70.86	41.44	18.83	7.37	22.55	0.0261	27.60
nnResU-net	54.47	71.50	90.01	71.99	38.44	19.22	7.47	21.71	0.0256	27.72
nnResUnet + trilinear decoder	54.54	71.46	90.26	72.09	41.13	17.42	7.47	22.01	0.0241	28.05

Table 2: Generator Architecture (

G

) Ablation Study. Quantitative performance comparison of different generative network architectures for the synthesis task in the missing T1c scenario. The study evaluates the impact of progressively upgrading the baseline nnU-Net by integrating 3D residual blocks (nnResU-Net) and replacing standard transposed convolutions with a trilinear interpolation decoder. Performance is assessed using both downstream clinical segmentation metrics (Dice and HD95) and standard Image Quality Metrics, IQM (NMSE and PSNR). The best results among the evaluated generative architectures are highlighted in bold.

	Dice ( $\uparrow$ )				HD95 ( $\downarrow$ )				IQM
Loss	ENH	TC	WT	Mean	ENH	TC	WT	Mean	NMSE ( $\downarrow$ )	PSNR ( $\uparrow$ )
Reference (All Real)	81.47	86.27	92.22	86.65	21.56	9.67	4.98	12.07	-	-
L1	54.54	71.46	90.26	72.09	41.13	17.42	7.47	22.01	0.0241	28.05
L1 + $Dice_{1}$	54.83	73.70	90.97	73.17	39.74	15.32	7.01	20.69	0.0286	26.95
L1 + $Dice_{2}$	54.93	76.99	91.17	74.36	38.21	13.07	5.94	19.07	0.0290	26.88
L1 + AFP	55.33	73.46	91.46	74.42	37.59	14.53	6.26	19.46	0.0240	27.83

Table 3: Objective Loss Function Ablation Study. Quantitative evaluation of different loss configurations for the optimized generative network

G

(nnResU-Net with trilinear decoder) in the missing T1c scenario. The baseline pixel-wise

\mathcal{L}_{1}

loss is progressively augmented with the Anatomical Feature-Prioritized (

\mathcal{L}_{AFP}

) loss and topological Dice constraints (

\text{Dice}_{1}

targeting global region overlap, and

\text{Dice}_{2}

focusing on boundary precision). The proposed hybrid objective (

\mathcal{L}_{1}+\mathcal{L}_{AFP}+\text{Dice}_{1,2}

), obtained with the combined use of Generator

G

and pretrained segmenter

S_{aux}

, achieves the best overall downstream segmentation performance, obtaining the highest mean Dice score and lowest mean HD95.

Figure 3: Visual Comparison of Loss Configurations. Generated T1c sequences under varying loss configurations for the missing T1c scenario. The purely pixel-wise $\mathcal{L}_{1}$ optimization produces over-smoothed lesions lacking structural definition. While intermediate configurations ( $\mathcal{L}_{1}+\mathcal{L}_{AFP}$ and variations with partial Dice constraints) yield incremental improvements in boundary demarcation, the proposed hybrid loss ( $\mathcal{L}_{1}+\mathcal{L}_{AFP}+\mathcal{L}_{DualDice}$ ) achieves the optimal qualitative balance. It successfully recovers both the sharp anatomical margins and the critical intra-tumoral heterogeneity and the regions of neoangiogenesis, closely mirroring the real T1c target.

	Dice ( $\uparrow$ )				HD95 ( $\downarrow$ )
Training Strategy	ENH	TC	WT	Mean	ENH	TC	WT	Mean
Reference (All Real)	81.47	86.27	92.22	86.65	21.56	9.67	4.98	12.07
25% Missing Modality Ratio
Random Dropout	57.39	77.10	90.89	75.13	40.19	13.47	6.24	19.97
Synthetic (from scratch)	56.98	77.55	92.05	75.53	35.98	10.85	4.44	17.09
Synthetic (FT)	56.76	76.40	90.13	74.43	37.56	14.89	11.44	21.30

Table 4: Segmenter (

S

) Training Strategy Ablation. Quantitative evaluation of different training paradigms for the final clinical segmenter under the missing T1c scenario. Using the optimized generator configuration, we explore the impact of varying the probability ratio (25%, 50%, and 75%) of encountering a missing modality during training. The standard Random Dropout (zero-padding) baseline is compared against our proposed synthetic data augmentation, evaluated both by training the segmenter from scratch and by fine-tuning (FT) a pre-trained model. The results demonstrate that training the segmenter from scratch with a 50% probability of synthetic imputation achieves the optimal balance, yielding the highest mean Dice score and validating the utility of generative augmentation for downstream robustness.

	Dice ( $\uparrow$ )				HD95 ( $\downarrow$ )				IQM
	ENH	TC	WT	Mean	ENH	TC	WT	Mean	NMSE ( $\downarrow$ )	PSNR ( $\uparrow$ )
Missing T1n	80.58	85.87	91.24	85.90	22.56	11.07	6.02	13.22	0.0160	28.99
Missing T1c	57.95	77.59	91.86	75.80	36.10	10.90	4.63	17.21	0.0259	27.32
Missing T2w	79.22	85.46	90.69	85.12	22.99	11.59	10.91	15.16	0.0530	26.18
Missing T2f	73.58	83.93	90.57	82.69	36.93	10.09	7.68	18.23	0.0453	25.34
Average	72.83	83.21	91.09	82.38	29.65	10.91	7.31	15.96	0.0351	26.96

Table 5: Overall Performance Across All Missing Modalities. Evaluation of the fully optimized CAPS for every single missing MRI scenario (T1n, T1c, T2w, and T2f). The table reports both clinical segmentation accuracy (Dice and HD95) and Image Quality Metrics, IQM (NMSE and PSNR). Utilizing the optimal generative architecture

G

, hybrid loss formulation through the use of

S_{aux}

, and synthetic data augmentation strategy on

S

established in the previous ablation studies, the framework demonstrates robust imputation fidelity across all sequences, achieving a global average Dice score of

82.38\%

		IQM			Synthesis Type
Method	Eval. Split	NMSE $\downarrow$	PSNR $\uparrow$	SSIM % $\uparrow$	1-to-1	3-to-1
Evaluated Scenarios: Missing T2w, T2f
Pix2Pix Isola et al. (2017)	Official Val.	0.105	24.89	87.05	$\times$
CycleGAN Zhu et al. (2017)	Official Val.	0.145	23.62	82.85	$\times$
EaGAN Yu et al. (2019)	Official Val.	0.103	24.73	86.65	$\times$
ALDM Kim and Park (2024)	Official Val.	0.089	25.45	88.55	$\times$

Table 6: State-of-the-Art Comparison of Image Reconstruction Quality. Quantitative synthesis evaluation against existing baselines using standard Image Quality Metrics (IQM), reported as the mean across all evaluated missing modality scenarios within each group. To ensure a fair comparison, methods are grouped according to their specific evaluated missing scenarios and reported data partitions (Official Validation set vs. a 20% held-out split). The results demonstrate that CAPS, our proposed 3-to-1 framework, substantially outperforms traditional 1-to-1 methods and achieves highly competitive structural fidelity (highest SSIM) compared to recent multi-modal diffusion approaches.

Figure 4: Zero-Shot Cross-Domain Generalization. Qualitative comparison of real and synthesized MRI modalities (T1n, T1c, T2w, T2f) across the in-distribution training cohort (Adult Glioma) and three completely unseen Out-of-Distribution datasets: Meningioma, Pediatric Glioma, and Sub-Saharan African Glioma. CAPS consistently reconstructs the missing sequences with high structural fidelity across all scenarios, thereby reproducing domain-specific morphological variations and demonstrating robust generalization capabilities without requiring any domain-specific adaptation.

Method	ENH $\uparrow$	TC $\uparrow$	WT $\uparrow$	Mean $\uparrow$
Robust Segmentation:
Random Dropout	74.29	84.42	91.63	83.45
U-HVED Dorent et al. (2019)	70.88	80.75	88.63	80.08
mmFormer Zhang et al. (2022)	80.23	88.38	92.20	86.93
Sfusion Liu et al. (2023)	77.95	86.60	91.05	85.20
IM-Fuse Pipoli et al. (2025)	80.73	88.95	92.35	87.34

Table 7: State-of-the-Art Comparison of Segmentation. Evaluation of tumor segmentation accuracy (Dice scores), reported as the average across all four missing modality scenarios (T1n, T1c, T2w, and T2f). Methods are categorized into pure robust segmentation models and interpretable 3-to-1 synthesis-based pipelines. Our framework outperforms existing synthesis-based approaches and remains competitive with leading purely computational segmenters.

	Meningioma		Pediatric Glioma		Sub-Saharan Glioma
Missing Modality	NMSE ( $\downarrow$ )	PSNR ( $\uparrow$ )	NMSE ( $\downarrow$ )	PSNR ( $\uparrow$ )	NMSE ( $\downarrow$ )	PSNR ( $\uparrow$ )
4MRI Real	0.0	$\infty$	0.0	$\infty$	0.0	$\infty$
Missing T1n	0.0248	27.72	0.0216	28.05	0.0215	27.56
Missing T1c	0.0505	25.41	0.0330	25.93	0.0272	25.91
Missing T2w	0.0534	26.21	0.0648	25.20	0.0507	25.67
Missing T2f	0.0360	25.63	0.0266	26.46	0.0279	26.55

Table 8: Zero-Shot Cross-Domain Generalization. Quantitative synthesis evaluation (NMSE and PSNR) on three completely unseen external datasets: Meningioma, Pediatric Glioma, and Sub-Saharan African Glioma. CAPS maintains consistently high image quality across all missing modality scenarios. This demonstrates robust out-of-the-box generalization to morphological and technical domain shifts, entirely without the need for domain-specific adaptation.

Figure 5: Blinded Radiological Evaluation Interface. Web-based tool for clinical validation of synthesized T1c volumes, organized into six panels: (1) patient list for case navigation; (2) slice selector with Real/Synth overlay toggle and windowing controls; (3) available input modalities (T1n, T2w, T2f); (4) side-by-side comparison of the real T1c against two anonymized synthetic volumes (Model A and Model B), whose assignment is randomly shuffled per patient to prevent positional bias; (5) independent scoring of Image Quality and Clinical Value on a 1–5 scale for each model; and (6) session management for exporting and resuming evaluations. Tool available at https://github.com/Phyrise/NIfTI-to-HTML-Evaluator .

Figure 6: Radiologist Evaluation Scores. Distribution of blinded radiologist scores comparing the standard

\mathcal{L}_{1}

baseline (blue) against our proposed synthesis generation (green). To facilitate a direct visual comparison, the data is presented as follows: the upper half displays the score distribution for Image Quality , while the lower half displays the scores for Clinical Utility . As observed, the proposed hybrid loss decisively shifts the distribution towards the maximum scores (4-5) in both evaluation dimensions, demonstrating its superior visual fidelity and diagnostic reliability.

Medical Image Synthesis

Medical image synthesis creates, translates, reconstructs, and edits clinical images to fill missing data and support analysis.

See all 312 papers

D3Seg: Dependency-Aware Diffusion for Brain Tumor Segmentation with Missing Modalities
May 21, 2026Danish Ali, Ajmal Mian, Naveed Akhtar +1Missing-Modality Learning BraTS
The University of Western Australia, Perth, WA 6009, Australia · The University of Melbourne, Melbourne, Parkville VIC 3010, Australia
Brain MR Image Synthesis with 3D Multi-Contrast Self-Attention GAN
Mar 31, 2026Zaid A. Abod, Furqan AzizMedical Image Synthesis 3D Medical Imaging
School of Computing and Mathematical Sciences, University of Leicester, Leicester, United Kingdom.
Segmentation-Assisted Brain MRI Synthesis with Cross-Image Multi-Contrast Feature Memory Bank Retrieval Augmentation
Jun 7, 2026Wenwei Huang, Jia Wei, Jianlong ZhouImage Generation Medical Image Synthesis
South China University of Technology, Guangzhou 510640, China · UTS Data Science Institute, University of Technology Sydney, Ultimo, NSW 2007, Australia

May 21, 2026J/K move · Enter open · S save

D3Seg: Dependency-Aware Diffusion for Brain Tumor Segmentation with Missing Modalities

Danish Ali, Ajmal Mian, Naveed Akhtar, Ghulam Mubashar Hassan

The University of Western Australia, Perth, WA 6009, Australia · The University of Melbourne, Melbourne, Parkville VIC 3010, Australia

Missing-Modality Learning BraTS Multimodal Fusion Brain Tumor Segmentation Diffusion Models

Open paper AskPDF arXiv

Accurate brain tumor segmentation using multi-parametric MRI is critical for effective treatment planning. However, in clinical settings, complete acquisition of all MRI sequences is not always possible. The absence of certain MRI modalities results in substantial performance degradation in existing segmentation methods, which typically rely on naive feature concatenation or direct fusion strategies. To address this limitation, we propose a novel segmentation model D3Seg which is designed to maintain stable performance under missing-modality settings. D3Seg introduces Multi-hop Modality Graph Fusion (MMGF) to model higher-order inter-modality dependencies, a lightweight diffusion-based imputation mechanism to compensate for missing T1ce and FLAIR feature representations in latent space, and probability-space decision refinement to mitigate dominant-class overconfidence and improve delineation of underrepresented tumor subregions. We evaluate the proposed D3Seg model on BraTS 2023 Glioma as the primary benchmark and further test it on a subset of the external BraTS 2023 Meningioma cohort to assess generalization across tumor pathologies. The results are compared with the state-of-the-art models under different missing-modality conditions. The proposed model achieves approximately 1.5-2.0% Dice improvement on enhancing tumor (ET) and around 1.0% on tumor core (TC) across multiple missing-modality configurations compared to the current state-of-the-art model on BraTS Glioma dataset. Cross-cohort evaluation on BraTS Meningioma dataset demonstrates the generalizability of the proposed model, showing consistent improvements in the challenging TC and ET regions, with approximately 1.5-3.0% and 1.5-6.5% gains respectively across several missing-modality configurations.

Abstract

Medical Image Synthesis

Explore similar work

D3Seg: Dependency-Aware Diffusion for Brain Tumor Segmentation with Missing Modalities

Brain MR Image Synthesis with 3D Multi-Contrast Self-Attention GAN

Segmentation-Assisted Brain MRI Synthesis with Cross-Image Multi-Contrast Feature Memory Bank Retrieval Augmentation