Mixture of Multi-Sequence Feature Difference for Contrast-Enhanced MRI Synthesis

Jonghun Kim; Junmo Kwon; Hyunjin Park

cs.CVOct 15, 2026

Mixture of Multi-Sequence Feature Difference for Contrast-Enhanced MRI Synthesis

Authors: Jonghun Kim, Junmo Kwon, Hyunjin Park

Organizations: Department of Electrical and Computer Engineering, Sungkyunkwan University, Suwon, Korea

Abstract

Contrast-enhanced magnetic resonance imaging (CE-MRI) is crucial for accurately diagnosing and segmenting brain tumors. However, the acquisition of CE-MRI requires injecting contrast agents, which poses potential health risks and increases scan costs. Here, we propose a CE-MRI synthesis method from non-contrast sequences, reducing both risk and cost. Unlike existing methods that rely on channel-wise concatenation to fuse multi-sequence information for synthesis, we introduce a Mixture of Feature Difference (MoFD) module. The proposed module uses a dynamic weighting strategy to adaptively select and aggregate informative features from both individual source sequences and explicit differences among them. Quantitative and qualitative evaluations demonstrate that our proposed method outperforms baselines in terms of image fidelity and perceptual quality on the UCSF-PDGM and UPenn-GBM datasets. Our method successfully reconstructs tumor regions and achieves segmentation performance comparable to that obtained with real CE-MRI. Our ablation studies show that although adversarial learning improves perceptual quality, a regression-based variant of our method achieves higher structural fidelity.

Figures & tables

Figure 1: Illustration of non-contrast multi-sequence MRI and contrast-enhanced MRI. Red bounding boxes show zoomed-in views of the tumor region. Unlike the non-contrast sequence images, the T1 contrast-enhanced image clearly delineates tumor boundaries.

Figure 2: Illustration of the proposed method for CE-MRI synthesis. Weight-sharing encoders extract features from each input sequence (T1, T2, and T2-FLAIR). The model uses the differences between these features and employs a softmax function to assign adaptive weights dynamically. These weighted features are then aggregated and fed into the decoder to synthesize the CE-MRI.

Figure 3: Illustration of the training objectives. (a) The structural alignment loss encourages structural information in sequence features to remain consistent across input sequences. (b) The reconstruction loss encourages the prediction to match the ground truth, while the optional GAN loss promotes realistic textures in the synthesized image.

Panel of Figure 4 — Figure 4: Qualitative results of T1ce synthesis with comparison methods on the UCSF-PDGM [ 7 , 8 ] . The bounding box (bottom left) shows a zoomed-in view of the tumor region from the T1ce. The bottom right image shows the corresponding tumor segmentation results by nnU-Net [ 22 ] . Without GAN, the results are blurry. With GAN, however, the perceptual quality improves.

Panel of Figure 5 — Figure 5: Qualitative results of T1ce synthesis with comparison methods on the UPENN-GBM [ 1 , 2 ] . The bounding box (bottom left) shows a zoomed-in view of the tumor region from the T1ce. The bottom right image shows the corresponding tumor segmentation results by nnU-Net [ 22 ] .

Panel of Figure 6 — Figure 6: Qualitative results of T1ce synthesis with various input sequence combinations on the UCSF-PDGM [ 7 , 8 ] . The bounding boxes display a zoomed-in view of the boxed area in the bottom left. The bottom right image shows the corresponding tumor segmentation results by nnU-Net [ 22 ] .

Dataset	UCSF-PDGM [ 7 , 8 ]
Tasks	T1ce Synthesis			Tumor Segmentation
Method	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	WT	ET	TC	Avg.
w/o Conv Block	23.590 $\pm$ 1.059	0.859 $\pm$ 0.026	0.028 $\pm$ 0.014	0.820 $\pm$ 0.097	0.709 $\pm$ 0.131	0.501 $\pm$ 0.185	0.677 $\pm$ 0.093
Average Feature	24.301 $\pm$ 1.198	0.875 $\pm$ 0.029	0.023 $\pm$ 0.019	0.824 $\pm$ 0.094	0.716 $\pm$ 0.146	0.512 $\pm$ 0.175	0.685 $\pm$ 0.098
Ours	24.626 $\pm$ 1.197	0.881 $\pm$ 0.029	0.022 $\pm$ 0.017	0.821 $\pm$ 0.104	0.763 $\pm$ 0.146	0.559 $\pm$ 0.174	0.714 $\pm$ 0.107

Table 4: Ablation study on the MoFD Architecture.

Dataset			UCSF-PDGM [ 7 , 8 ]
Tasks			T1ce Synthesis			Tumor Segmentation
$\mathcal{L}_{rec}$	$\mathcal{L}_{align}$	$\mathcal{L}_{gan}$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	WT	ET	TC	Avg.
✓			24.833 $\pm$ 1.263	0.891 $\pm$ 0.029	0.027 $\pm$ 0.014	0.859 $\pm$ 0.078	0.757 $\pm$ 0.117	0.474 $\pm$ 0.264	0.696 $\pm$ 0.115
✓	✓		25.009 $\pm$ 1.263	0.894 $\pm$ 0.027	0.024 $\pm$ 0.020	0.867 $\pm$ 0.080	0.769 $\pm$ 0.165	0.588 $\pm$ 0.214	0.741 $\pm$ 0.134
✓		✓	24.025 $\pm$ 1.211	0.888 $\pm$ 0.026	0.023 $\pm$ 0.013	0.824 $\pm$ 0.092	0.718 $\pm$ 0.145	0.545 $\pm$ 0.152	0.695 $\pm$ 0.098
✓	✓	✓	24.626 $\pm$ 1.197	0.881 $\pm$ 0.029	0.022 $\pm$ 0.017	0.821 $\pm$ 0.104	0.763 $\pm$ 0.146	0.559 $\pm$ 0.174	0.714 $\pm$ 0.107

Table 5: Ablation study on the Loss function.

Figure 7: Visualization of features, feature differences, and weighted sum features derived from source T1, T2, and FLAIR modalities. $f_{wsum}$ denotes the weighted sum feature. The red bounding boxes indicate zoomed-in views of the tumor region.

Figure 8: Visualization of the feature selection probability in the MoFD module . At low-level scales (scales 0 and 1), the selection probabilities are relatively evenly distributed across features and feature differences. As the network becomes deeper and approaches higher-level representations, the selection probabilities for T2 and T2-FLAIR features increase. Since image translation requires accurate pixel-level synthesis, low-level features passed through the skip connections remain important and the proposed module effectively uses all available features across scales.

Medical Image Synthesis

Medical image synthesis creates, translates, reconstructs, and edits clinical images to fill missing data and support analysis.

See all 310 papers

Segmentation-Assisted Brain MRI Synthesis with Cross-Image Multi-Contrast Feature Memory Bank Retrieval Augmentation
Jun 7, 2026Wenwei Huang, Jia Wei, Jianlong ZhouImage Generation Medical Image Synthesis
South China University of Technology, Guangzhou 510640, China · UTS Data Science Institute, University of Technology Sydney, Ultimo, NSW 2007, Australia
Dense Temporal Contrast Synthesis via Conditioned Latent Transport
Jul 31, 2026Smriti Joshi, Apostolia Tsirikoglou, Daniel M. Lang +15Magnetic Resonance Imaging Medical Image Synthesis
Departament de Matemàtiques i Informàtica, Universitat de Barcelona, Barcelona, Spain · Department of Oncology-Pathology, Karolinska Institutet, SE-171 77, Stockholm, Sweden · Institute of Machine Learning in Biomedical Imaging, Helmholtz Munich, Munich, Germany +9
TuLaBM: Tumor-Biased Latent Bridge Matching for Contrast-Enhanced MRI Synthesis
Mar 19, 2026Atharva Rege, Adinath Madhavrao Dukre, Numan Balci +2Medical Image Synthesis Magnetic Resonance Imaging
MBZUAI, Abu Dhabi, UAE · National Institute of Technology Karnataka, Surathkal, India · Cleveland Clinic Abu Dhabi, UAE +1

Jun 7, 2026J/K move · Enter open · S save

Segmentation-Assisted Brain MRI Synthesis with Cross-Image Multi-Contrast Feature Memory Bank Retrieval Augmentation

Wenwei Huang, Jia Wei, Jianlong Zhou

South China University of Technology, Guangzhou 510640, China · UTS Data Science Institute, University of Technology Sydney, Ultimo, NSW 2007, Australia

Image Generation Medical Image Synthesis Generative Adversarial Networks

Open paper AskPDF arXiv

Multi-contrast brain MRI provide complementary soft-tissue characteristics that aid in the screening and diagnosis of diseases. However, limited scanning time, image corruption and various imaging protocols often result in incomplete multi-contrast images. While current approaches excel in image synthesis, they often struggle to synthesize critical tumor regions and exploit contextual information in multi-contrast brain MRI effectively. To address this issue, we propose a synthesis-centric, segmentation-assisted closed-loop framework with retrieval augmentation synthesis. Our method overall takes a generative adversarial architecture, which aims to synthesize missing contrasts from any combination of available ones with a single model. To explicitly capture tumor semantics and focus synthesis on tumor regions, we add an auxiliary segmentation branch that predicts tumor masks and feeds them back as semantic conditioning in synthesis branch, thereby learning tumor-aware representations in the model and improving synthesis fidelity. Furthermore, we propose a dual-bank retrieval augmentation strategy. It dynamically queries two external knowledge bases, namely a tumor masks memory bank for crucial tumor context and cross-image contrast feature memory bank for global style information, to augment synthesis. Verified on two public multi-contrast magnetic resonance brain datasets: BraTs2020 and UCSF-BMSR, the proposed method is effective in handling medical brain images synthesis tasks and shows superior performance compared to previous methods. Code is available at:https://github.com/iBizzard/SSCF.git

Abstract

Medical Image Synthesis

Explore similar work

Segmentation-Assisted Brain MRI Synthesis with Cross-Image Multi-Contrast Feature Memory Bank Retrieval Augmentation

Dense Temporal Contrast Synthesis via Conditioned Latent Transport

TuLaBM: Tumor-Biased Latent Bridge Matching for Contrast-Enhanced MRI Synthesis