Missing Modality-Aware Calibration for Trustworthy Brain Tumor Segmentation

Sol Lee; Hyunji Kim; Sungrae Hong; Donghee Han; Mun Yi

cs.CVOct 8, 2026

Missing Modality-Aware Calibration for Trustworthy Brain Tumor Segmentation

Authors: Sol Lee, Hyunji Kim, Sungrae Hong, Donghee Han, Mun Yi

Organizations: Korea Advanced Institute of Science and Technology, Daejeon, South Korea

Abstract

Multimodal brain tumor segmentation typically leverages multiple MRI modalities, yet incomplete modality acquisition is common in clinical practice due to protocol heterogeneity and scan failures. Although recent methods maintain segmentation accuracy under missing modality conditions, they frequently overlook prediction reliability, leading to miscalibrated confidence estimates that hinder clinical adoption. Existing calibration techniques are largely modality-agnostic or assume that prediction difficulty decreases monotonically as additional modalities become available. However, in brain tumor segmentation, prediction difficulty depends primarily on which modalities are absent rather than how many, leading to combination-specific and spatially heterogeneous calibration errors. To address this, we propose Missing Modality-Aware Local Temperature Scaling (MMA-LTS), a post-hoc voxel-wise confidence calibration method. It estimates a spatially adaptive temperature field conditioned on a modality-availability learnable token and a voxel-wise difficulty score. Experiments on BraTS 2020 and FeTS 2024 show that MMA-LTS improves calibration while preserving the segmentation accuracy of state-of-the-art models across diverse missing-modality scenarios, thereby enhancing trustworthiness toward clinical deployment.

Figures & tables

Figure 1: Segmentation performance and calibration results. (a) Left: DSC and ECE of state-of-the-art models on BraTS 2020, averaged over all modality combinations. Right: DC-Seg results per modality combination for WT. (b) Two cases with the same modality count but differing in combination, showing DC-Seg confidence maps (uncalibrated and MMA-LTS), voxel-wise difficulty score (VDS), and calibration strength ( $|T-1|$ ).

Figure 2: The overall framework of MMA-LTS.

Type	FLAIR	$\circ$	$\circ$	$\circ$	$\bullet$	$\circ$	$\circ$	$\bullet$	$\circ$	$\bullet$	$\bullet$	$\bullet$	$\bullet$	$\bullet$	$\circ$	$\bullet$	Avg.
	T1	$\circ$	$\circ$	$\bullet$	$\circ$	$\circ$	$\bullet$	$\bullet$	$\bullet$	$\circ$	$\circ$	$\bullet$	$\bullet$	$\circ$	$\bullet$	$\bullet$
	T1ce	$\circ$	$\bullet$	$\circ$	$\circ$	$\bullet$	$\bullet$	$\circ$	$\circ$	$\circ$	$\bullet$	$\bullet$	$\circ$	$\bullet$	$\bullet$	$\bullet$
	T2	$\bullet$	$\circ$	$\circ$	$\circ$	$\bullet$	$\circ$	$\circ$	$\bullet$	$\bullet$	$\circ$	$\circ$	$\bullet$	$\bullet$	$\bullet$	$\bullet$
WT	Uncalibrated	$24.72$	$21.76$	$34.20$	$26.62$	$14.94$	$19.67$	$23.65$	$24.09$	$22.14$	$13.67$	$13.28$	$21.48$	$12.83$	$15.02$	$13.00$	$20.07$
	$\llcorner$ DSC	$85.09$	$78.09$	$76.18$	$86.79$	$87.27$	$81.41$	$89.49$	$86.74$	$89.52$	$89.86$	$90.60$	$90.14$	$90.47$	$87.81$	$90.81$	$86.68$
	mL1-ACE [ 1 ]	$27.04$	$25.44$	$39.41$	$26.41$	$16.37$	$23.46$	$23.69$	$25.90$	$22.08$	$13.97$	$13.80$	$21.24$	$13.00$	$16.41$	$12.98$	$21.41$

Table 1: DSC and ECE comparison of DC-Seg on BraTS 2020.

\bullet

and

\circ

denote available and missing modalities. Temperature scaling-based methods (TS, DirS, LTS, SelS, MMA-LTS) preserve the DSC of the uncalibrated model; only ECE is reported.

Type	FLAIR	$\circ$	$\circ$	$\circ$	$\bullet$	$\circ$	$\circ$	$\bullet$	$\circ$	$\bullet$	$\bullet$	$\bullet$	$\bullet$	$\bullet$	$\circ$	$\bullet$	Avg.
	T1	$\circ$	$\circ$	$\bullet$	$\circ$	$\circ$	$\bullet$	$\bullet$	$\bullet$	$\circ$	$\circ$	$\bullet$	$\bullet$	$\circ$	$\bullet$	$\bullet$
	T1ce	$\circ$	$\bullet$	$\circ$	$\circ$	$\bullet$	$\bullet$	$\circ$	$\circ$	$\circ$	$\bullet$	$\bullet$	$\circ$	$\bullet$	$\bullet$	$\bullet$
	T2	$\bullet$	$\circ$	$\circ$	$\circ$	$\bullet$	$\circ$	$\circ$	$\bullet$	$\bullet$	$\circ$	$\circ$	$\bullet$	$\bullet$	$\bullet$	$\bullet$
WT	Uncalibrated	$20.89$	$17.00$	$25.25$	$17.92$	$12.09$	$14.66$	$15.71$	$19.30$	$15.33$	$8.69$	$8.49$	$14.73$	$8.49$	$12.00$	$14.60$	$15.01$
	$\llcorner$ DSC	$87.65$	$79.33$	$80.80$	$90.31$	$88.97$	$83.44$	$91.84$	$88.53$	$91.94$	$91.64$	$92.27$	$92.22$	$92.50$	$89.20$	$88.88$	$88.63$
	mL1-ACE [ 1 ]	$20.44$	$20.70$	$27.20$	$15.92$	$12.70$	$17.43$	$14.00$	$18.53$	$13.84$	$8.12$	$8.12$	$13.18$	$7.88$	$12.46$	$7.96$	$14.57$

Table 2: DSC and ECE comparison of DC-Seg on FeTS 2024.

Dataset	Model	Type	Uncal	mL1-ACE	SDC	MC	TS	DirS	LTS	SelS	MMA-LTS ∗
BraTS 2020	RobustSeg	WT	$20.68$ / $84.09$	$19.68$ / $80.13$	$20.10$ / $83.53$	$20.73$ / $84.31$	$19.54$	$20.47$	$15.57$	$15.92$	$14.93$
		TC	$26.18$ / $75.30$	$23.96$ / $71.30$	$23.73$ / $74.83$	$25.50$ / $75.44$	$24.90$	$23.96$	$18.89$	$18.63$	$16.56$
		ET	$21.18$ / $62.96$	$19.87$ / $59.79$	$19.28$ / $62.79$	$20.76$ / $63.14$	$20.22$	$20.14$	$21.02$	$20.03$	$15.78$
	mmFormer	WT	$22.10$ / $86.32$	$20.30$ / $86.06$	$20.37$ / $86.62$	$22.17$ / $86.13$	$21.49$	$22.05$	$17.01$	$18.40$	$11.61$
		TC	$29.11$ / $79.70$	$25.27$ / $78.65$	$26.32$ / $79.65$	$29.15$ / $79.56$	$28.43$	$28.43$	$19.38$	$24.88$	$17.42$
		ET	$26.50$ / $65.35$	$22.59$ / $64.15$	$23.90$ / $65.30$	$26.60$ / $65.20$	$25.83$	$26.31$	$17.78$	$22.05$	$16.46$

Table 3: DSC and ECE comparison of RobustSeg and mmFormer on BraTS 2020 and FeTS 2024. Values are reported as ECE/DSC and averaged over modality combinations.

Figure 3: Ablation study of MMA-LTS. ECE are averaged over modality combinations.

Figure 4: Time efficiency and sensitivity analysis (DC-Seg, BraTS 2020). (a) ECE versus inference and training time. (b) Sensitivity to

C_{f}

d_{e}

, and

\mathcal{D}_{\text{val}}

size (dashed: 100%).

Explore similar work

Set-Inclusive Uncertainty Modeling for Robust Brain Tumor Segmentation
Jun 29, 2026Seunghun Baek, Jihwan Park, Jaeyoon Sim +3Representation Learning Multimodal Robustness
Pohang University of Science and Technology, Pohang, South Korea
D3Seg: Dependency-Aware Diffusion for Brain Tumor Segmentation with Missing Modalities
May 21, 2026Danish Ali, Ajmal Mian, Naveed Akhtar +1Missing-Modality Learning BraTS
The University of Western Australia, Perth, WA 6009, Australia · The University of Melbourne, Melbourne, Parkville VIC 3010, Australia
A Multimodal Feature Distillation with Mamba-Transformer Network for Brain Tumor Segmentation with Incomplete Modalities
Apr 22, 2024Ming Kang, Fung Fung Ting, Shier Nee Saw +3Cross-Modal Knowledge Distillation Cross-Modality Medical Image Segmentation
School of Information Technology, Monash University, Malaysia Campus, Subang Jaya 47500, Malaysia · Augmented Intelligence and Multimodal analytics (AIM) for Health Lab, Faculty of Information Technology, Monash University, Clayton VIC 3800, Australia · Monash Medical AI Group, Monash University, Clayton VIC 3800, Australia +1

Jun 29, 2026, cs.CVJ/K move · Enter open · S save

Set-Inclusive Uncertainty Modeling for Robust Brain Tumor Segmentation

Seunghun Baek, Jihwan Park, Jaeyoon Sim, Hoseok Lee+2

Pohang University of Science and Technology, Pohang, South Korea

Representation Learning Multimodal Robustness BraTS Medical Image Segmentation Segmentation Uncertainty Estimation Brain Tumor Segmentation

Open paper AskPDF arXiv

Multimodal MRI is essential for accurate brain tumor segmentation. However, acquiring all modalities at inference is often challenging in practice, which causes intrinsic uncertainty due to unavoidable information loss. Without modeling this uncertainty, existing methods encode incomplete evidence into deterministic representations that appear plausible but lack reliability. In this regime, we propose a probabilistic representation framework that models representations as Gaussian distributions, where their mean captures task information and their variance measures uncertainty from missing evidence. To make variance reflect information deficiency, we regularize the mean from each partial configuration toward its full-modality counterpart, while scaling the variance with the discrepancy between their aligned means. We further introduce a set-inclusive strategy that exploits the hierarchical structure of modality subsets and enforces an ordering constraint to maintain their consistent uncertainty relationships. Extensive experiments on BraTS 2018 and 2020 demonstrate that our approach offers superior performance over baselines across diverse missing-modality scenarios. Code and model checkpoint are available at https://github.com/atlas-sky/SIUM.

Abstract

Explore similar work

Set-Inclusive Uncertainty Modeling for Robust Brain Tumor Segmentation

D3Seg: Dependency-Aware Diffusion for Brain Tumor Segmentation with Missing Modalities

A Multimodal Feature Distillation with Mamba-Transformer Network for Brain Tumor Segmentation with Incomplete Modalities