eess.IVMay 12, 2025

FermatSyn: SAM2-Enhanced Bidirectional Mamba with Isotropic Spiral Scanning for Multi-Modal Medical Image Synthesis

Authors: Feng Yuan, Yifan Gao, Haoyue Li, Xin Gao

Organizations: School of Biomedical Engineering (Suzhou), Division of Life Science and Medicine, University of Science and Technology of China, Hefei, China · Suzhou Institute of Biomedical Engineering and Technology, Chinese Academy of Sciences, Suzhou, China · Shanghai Innovation Institute, Shanghai, China

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Multi-modal medical image synthesis is pivotal for alleviating clinical data scarcity, yet existing methods fail to reconcile global anatomical consistency with high-fidelity local detail. We propose FermatSyn, which addresses three persistent limitations: (1) SAM2-based Prior Encoder that injects domain-aware anatomical knowledge via LoRA+^{+} efficient fine-tuning of a frozen SAM2 Vision Transformer; (2) Hierarchical Residual Downsampling Module (HRDM) coupled with a Cross-scale Integration Network (CIN) that preserves high-frequency lesion details and adaptively fuses global--local representations; and (3) continuity constrained Fermat Spiral Scanning strategy within a Bidirectional Fermat Scan Mamba (BFS-Mamba), constructing an approximately isotropic receptive field that substantially reduces the directional bias of raster or spiral serialization. Experiments on SynthRAD2023, BraTS2019, BraTS-MEN, and BraTS-MET show FermatSyn surpasses state-of-the-art methods in PSNR, SSIM, FID, and 3D structural consistency. Downstream segmentation on synthesized images yields no significant difference from real-image training (p>0.05p{>}0.05), confirming clinical utility. Code is available at https://github.com/exsinger-hub/ FermatSyn.

Figures & tables

Figure 1: Overall architecture of FermatSyn. The Hybrid Encoder fuses SAM2 global priors with HRDM local details via CIN. BFS-Mamba processes the Fermat-serialised features bidirectionally and reconstructs the target modality through a residual decoder.
Figure 2: SSM operator footprints (Jacobian sensitivity maps; μ\mu / σ\sigma : mean/std of normalised values) for three scanning strategies. (a) Raster ( μ=0.891\mu{=}0.891 , σ=0.115\sigma{=}0.115 ): stripe-directional bias. (b) Rectangular-spiral ( μ=0.890\mu{=}0.890 , σ=0.124\sigma{=}0.124 ): X-shaped corner hot-spots. (c) Proposed Fermat Spiral ( μ=0.621\mu{=}0.621 , σ=0.088\sigma{=}0.088 ): near-isotropic coverage ( σ\sigma reduced 29% vs. (b)). Colour scale: 0 (dark blue) →\to 1 (yellow).
MethodT1n →\to T1cT2w →\to T1cT2f →\to T1c
SSIMPSNRFIDSSIMPSNRFIDSSIMPSNRFID
MM-GAN [ 18 ]0.801∗0.801^{*}22.14∗22.14^{*}112.7∗112.7^{*}0.813∗0.813^{*}23.05∗23.05^{*}118.4∗118.4^{*}0.825∗0.825^{*}22.89∗22.89^{*}103.6∗103.6^{*}
MC-cGAN [ 5 ]0.814∗0.814^{*}22.83∗22.83^{*}108.3∗108.3^{*}0.821∗0.821^{*}23.77∗23.77^{*}113.1∗113.1^{*}0.836∗0.836^{*}23.41∗23.41^{*}98.2∗98.2^{*}
ResViT [ 4 ]0.832∗0.832^{*}25.58∗25.58^{*}84.3∗84.3^{*}0.840∗0.840^{*}26.63∗26.63^{*}91.2∗91.2^{*}0.855∗0.855^{*}26.27∗26.27^{*}79.6∗79.6^{*}
TransUNet [ 3 ]0.844∗0.844^{*}25.47∗25.47^{*}78.1∗78.1^{*}0.854∗0.854^{*}27.32∗27.32^{*}83.7∗83.7^{*}0.873∗0.873^{*}26.68∗26.68^{*}74.2∗74.2^{*}
SA-LDM [ 22 ]0.851∗0.851^{*}26.93∗26.93^{*}68.5∗68.5^{*}0.862∗0.862^{*}27.11∗27.11^{*}72.8∗72.8^{*}0.879∗0.879^{*}27.34∗27.34^{*}61.3∗61.3^{*}
Table 1: Intra-modal synthesis on the merged brain tumor dataset (T1n/T2w/T2f →\to T1c). ∗ : p<0.05p{<}0.05 vs. FermatSyn. Bold : best; uline : second-best.
Table 4
Figure 3: Qualitative synthesis comparison. Rows 1–3 : Intra-modal—T1n →\to T1c (metastasis), T2f →\to T1c (meningioma), T2w →\to T1c (glioma); orange boxes: tumour core ROIs. Row 4 : Cross-modal CT →\to MRI (SynthRAD2023); orange arrows: ventricular wall zoom-in. Numbers: per-case SSIM.
TaskReg.RealOursI2I-MambaSA-LDMVMambaTransU.ResViT
T1n →\to T2wWT0.8630.847†\mathbf{0.847}^{\dagger}0.826∗0.826^{*}0.831∗‾\underline{0.831^{*}}0.815∗0.815^{*}0.812∗0.812^{*}0.809∗0.809^{*}
ET0.7850.762†\mathbf{0.762}^{\dagger}0.746∗0.746^{*}0.751∗‾\underline{0.751^{*}}0.745∗0.745^{*}0.740∗0.740^{*}0.738∗0.738^{*}
TC0.8120.785†‾\underline{0.785^{\dagger}}0.793†\mathbf{0.793}^{\dagger}0.781†0.781^{\dagger}0.768∗0.768^{*}0.764∗0.764^{*}0.762∗0.762^{*}
T1n →\to T2fWT0.8690.851†\mathbf{0.851}^{\dagger}0.832∗0.832^{*}0.838∗‾\underline{0.838^{*}}0.823∗0.823^{*}0.819∗0.819^{*}0.817∗0.817^{*}
ET0.7910.775†\mathbf{0.775}^{\dagger}0.757∗0.757^{*}0.762∗‾\underline{0.762^{*}}0.748∗0.748^{*}0.743∗0.743^{*}0.741∗0.741^{*}
TC0.8180.798†\mathbf{0.798}^{\dagger}0.780†0.780^{\dagger}0.785†‾\underline{0.785^{\dagger}}0.773∗0.773^{*}0.769∗0.769^{*}0.767∗0.767^{*}
Table 4: Downstream brain tumour segmentation Dice on synthetic images from T1n ( real trained / synthetic evaluated protocol). “Real”: topline trained and evaluated on real images. † : p>0.05p{>}0.05 vs. Real. ∗ : p<0.05p{<}0.05 vs. Real.
(a) Step-wiseSynthRADBraTS(b) ScanningSynthRADBraTS
SSIMPSNRSSIMPSNRSSIMPSNRSSIMPSNR
GAN baseline0.823∗0.823^{*}23.67∗23.67^{*}0.798∗0.798^{*}24.35∗24.35^{*}Raster-Scan0.876∗0.876^{*}26.95∗26.95^{*}0.868∗0.868^{*}27.71∗27.71^{*}
+SAM2-VTE0.865∗0.865^{*}27.35∗27.35^{*}0.851∗0.851^{*}27.18∗27.18^{*}Rect. Spiral0.885∗0.885^{*}27.38∗27.38^{*}0.877∗0.877^{*}28.13∗28.13^{*}
+HRDM+CIN0.887∗0.887^{*}27.74∗27.74^{*}0.878∗0.878^{*}28.45∗28.45^{*}Fermat0.90528.670.90129.53
Full0.90528.670.90129.53     
Table 5: Ablation on SynthRAD2023(CT →\to MRI) and merged BraTS(T2f →\to T1c). ∗ : p<0.05p{<}0.05 vs. FermatSyn.
More in this topic

Medical Image Synthesis

Medical image synthesis creates, translates, reconstructs, and edits clinical images to fill missing data and support analysis.

See all 310 papers

Explore similar work

CardsList

  1. Segmentation-Assisted Brain MRI Synthesis with Cross-Image Multi-Contrast Feature Memory Bank Retrieval Augmentation

    Jun 7, 2026Wenwei Huang, Jia Wei, Jianlong ZhouImage GenerationMedical Image Synthesis

    South China University of Technology, Guangzhou 510640, China · UTS Data Science Institute, University of Technology Sydney, Ultimo, NSW 2007, Australia

  2. WFM: 3D Wavelet Flow Matching for Ultrafast Multi-Modal MRI Synthesis

    Apr 22, 2026Yalcin Tur, Mihajlo Stojkovic, Ulas BagciFlow MatchingBraTS

    Department of Computer Science, Stanford University, Stanford, CA, USA · Machine and Hybrid Intelligence Lab, Feinberg School of Medicine, Northwestern University, Chicago, IL, USA

  3. Prob-BBDM: a Probabilistic Brownian Bridge Diffusion Model for MRI sequence image-to-image translation

    Jun 23, 2026Martin Valls, Pascal Bourdon, Christine Fernandez-Maloigne +2Image-to-Image TranslationMagnetic Resonance Imaging

    UFR SFA · University of Poitiers, CNRS, XLIM, France · I3M common laboratory CNRS-Siemens Healthinners, Poitiers University Hospital and University of Poitiers, France +4