SA-CycleGAN-2.5D: Self-Attention CycleGAN with Tri-Planar Context for Multi-Site MRI Harmonization

Ishrith Gowda; Chunwei Liu

cs.CVMar 17, 2026

SA-CycleGAN-2.5D: Self-Attention CycleGAN with Tri-Planar Context for Multi-Site MRI Harmonization

Authors: Ishrith Gowda, Chunwei Liu

Organizations: Department of Electrical Engineering and Computer Sciences University of California, Berkeley Berkeley, CA 94720 · Department of Computer Science Purdue University West Lafayette, IN 47907

Abstract

Multi-site neuroimaging analysis is fundamentally confounded by scanner-induced covariate shifts, where the marginal distribution of voxel intensities $P(\mathbf{x})$ varies non-linearly across acquisition protocols while the conditional anatomy $P(\mathbf{y}|\mathbf{x})$ remains constant. This is particularly detrimental to radiomic reproducibility, where acquisition variance often exceeds biological pathology variance. Existing statistical harmonization methods (e.g., ComBat) operate in feature space, precluding spatial downstream tasks, while standard deep learning approaches are theoretically bounded by local effective receptive fields (ERF), failing to model the global intensity correlations characteristic of field-strength bias. We propose SA-CycleGAN-2.5D, a domain adaptation framework motivated by the $HΔH$ -divergence bound of Ben-David et al., integrating three architectural innovations: (1) A 2.5D tri-planar manifold injection preserving through-plane gradients $\nabla_z$ at $O(HW)$ complexity; (2) A U-ResNet generator with dense voxel-to-voxel self-attention, surpassing the $O(\sqrt{L})$ receptive field limit of CNNs to model global scanner field biases; and (3) A spectrally-normalized discriminator constraining the Lipschitz constant ( $K_D \le 1$ ) for stable adversarial optimization. Evaluated on 654 glioma patients across two institutional domains (BraTS and UPenn-GBM), our method reduces Maximum Mean Discrepancy (MMD) by 99.1% ( $1.729 \to 0.015$ ) and degrades domain classifier accuracy to near-chance (59.7%). Ablation confirms that global attention is statistically essential (Cohen's $d = 1.32$ , $p < 0.001$ ) for the harder heterogeneous-to-homogeneous translation direction. By bridging 2D efficiency and 3D consistency, our framework yields voxel-level harmonized images that preserve tumor pathophysiology, enabling reproducible multi-center radiomic analysis.

Figures & tables

Panel of Figure 1 — Figure 1: SA-CycleGAN-2.5D generator architecture. The 2.5D tri-planar input ( $12$ channels) passes through a convolutional stem, three encoder stages with CBAM modules, nine residual bottleneck blocks (three groups: CBAM, self-attention, CBAM, plus a global self-attention module), and three decoder stages with skip connections. Orange blocks denote self-attention; green blocks denote CBAM. The discriminator (not shown) is a spectrally-normalized multi-scale PatchGAN.

Hyperparameter	Value
Input channels (2.5D)	$12$ ( $4$ modalities $\times$ $3$ slices)
Encoder stages	$3$ ( $64\to 128\to 256$ channels)
Residual bottleneck blocks	$9$ at $256$ channels
Self-attention locations	Blocks 4, 5, 6 + global
CBAM modules	$11$
Generator parameters (each)	$11.9\text{M}$

Table 1: Implementation hyperparameters.

Figure 2: Harmonization results ( $A{\to}B{\to}A$ ) across T1, T1CE, T2, FLAIR (rows). Columns: input, baseline translation, +Attention translation, baseline reconstruction, +Attention reconstruction, attention difference map. Structural features are preserved; changes concentrate on global intensity (not anatomy).

Figure 3: t-SNE visualization of ResNet-18 features (

n{=}318

). (a) Raw: clear domain separation (98.4% classifier accuracy, MMD = 1.729). (b) Harmonized: domains thoroughly interleaved (59.7% accuracy, MMD = 0.015).

Type	Direction	SSIM $\uparrow$	PSNR $\uparrow$	MAE $\downarrow$	LPIPS $\downarrow$
Forward	$A{\to}B$	$.713\pm.055$	$19.35\pm 1.48$	$.053\pm.014$	$.233\pm.058$
Forward	$B{\to}A$	$.680\pm.041$	$19.65\pm 1.32$	$.074\pm.017$	$.419\pm.068$
Cycle	$A{\to}B{\to}A$	$.923\pm.016$	$27.49\pm 1.10$	$.014\pm.003$	N/A
Cycle	$B{\to}A{\to}B$	$.928\pm.015$	$27.73\pm 0.99$	$.014\pm.003$	N/A

Table 2: Translation quality. Forward: direct translation quality. Cycle: round-trip fidelity (mean

\pm

std,

n{=}7{,}897

test slices).

Domain Classification
Metric	Raw	Ours	ComBat ∗	Relative Improv.
Accuracy $\downarrow$	0.984	0.597	0.750	39.3%
AUC-ROC $\downarrow$	0.995	0.613	0.800	38.4%
F1-Score $\downarrow$	0.984	0.689	N/A	30.0%
Feature Distribution
MMD (RBF) $\downarrow$	1.729	0.015	0.003 ∗	99.1%

Table 3: Harmonization comparison.

\downarrow

\uparrow

: preferred direction. Bold : best among image-producing methods. ∗ Feature-space only (cannot produce harmonized images).

	Baseline		+Attention
Direction / Metric	Mean	Std	Mean	Std	$\Delta$	Cohen’s $d$
$A\to B\to A$ (easy direction: heterogeneous target)
Cycle SSIM	.9325	.0143	.9261	.0148	$-$ 0.64%	$-$ 1.96
Cycle PSNR	28.35	1.37	27.90	1.38	$-$ 0.45 dB	$-$ 1.44
$B\to A\to B$ (hard direction: homogeneous target)
Cycle SSIM	.9282	.0171	.9392	.0123	$+$ 1.10%	$+$ 1.13

Table 4: Ablation: self-attention contribution on foreground slices (

n{=}5{,}265

; mean intensity

\geq 0.05

). Baseline: 33.9M parameters (no attention). All

p{<}0.001

(paired

t

-test, Bonferroni-corrected

\alpha_{\mathrm{adj}}{=}0.00625

Figure 4: Ablation study: Cohen’s

d

effect size across modalities and translation directions. Positive

d

(blue) indicates attention benefit; negative

d

(red) indicates capacity rebalancing toward the harder direction. All

|d|{>}1.0

indicates large effects.

Category	$n$	CCC	ICC	Pearson $r$
First Order	168	$.007\pm.033$	$.017\pm.021$	$.007\pm.034$
GLCM	172	$.004\pm.032$	$.015\pm.021$	$.004\pm.033$
Shape	172	$.004\pm.034$	$.016\pm.021$	$.004\pm.035$
Overall	512	$\mathbf{.005{\pm}.033}$	$\mathbf{.016{\pm}.021}$	$\mathbf{.005{\pm}.034}$

Table 5: Radiomics concordance across 512 IBSI features. Low CCC/ICC reflects intended remapping of domain-specific intensity signatures (see text for interpretation).

Figure 5: Radiomics feature scatter: pre- vs. post-harmonization values across 512 IBSI features (first-order, GLCM, shape). Systematic scatter confirms intended intensity-distribution remapping; spatial structural features are preserved (cycle SSIM

>0.92

Explore similar work

Mar 5, 2026

HARP: HARmonizing in-vivo diffusion MRI using Phantom-only training

Purpose: Combining multi-site diffusion MRI (dMRI) data is hindered by inter-scanner variability, which confounds subsequent analysis. Previous harmonization methods require large, matched or traveling human subjects from multiple sites, which are impractical to acquire in many situations. This study aims to develop a deep learning-based dMRI harmonization framework that eliminates the reliance on multi-site in-vivo traveling human data for training. Methods: HARP employs a voxel-wise 1D neural network trained on an easily transportable diffusion phantom. The model learns relationships between spherical harmonics coefficients of different sites without memorizing spatial structures. Results: HARP reduced inter-scanner variability levels significantly in various measures. Quantitatively, it decreased inter-scanner variability as measured by standard error in FA (12%), MD (10%), and GFA (30%) with scan-rescan standard error as the baseline, while preserving fiber orientations and tractography after harmonization. Conclusion: We believe that HARP represents an important first step toward dMRI harmonization using only phantom data, thereby obviating the need for complex, matched in vivo multi-site cohorts. This phantom-only strategy substantially enhances the feasibility and scalability of quantitative dMRI for large-scale clinical studies.

Hwihun Jeong, Qiang Liu, Kathryn E. Keenan +7

Department of Psychiatry, Brigham and Women’s Hospital, Harvard Medical School, Boston, Massachusetts, United States of America · College of Engineering, Northeastern University, Boston, Massachusetts, United States of America · Physical Measurement Laboratory, National Institute of Standards and Technology, Boulder, Colorado, United States of America +6

Diffusion MRI Medical Image Analysis

Ask PDF

Feb 25, 2026

IHF-Harmony: Multi-Modality Magnetic Resonance Images Harmonization using Invertible Hierarchy Flow Model

Retrospective MRI harmonization is limited by poor scalability across modalities and reliance on traveling subject datasets. To address these challenges, we introduce IHF-Harmony, a unified invertible hierarchy flow framework for multi-modality harmonization using unpaired data. By decomposing the translation process into reversible feature transformations, IHF-Harmony guarantees bijective mapping and lossless reconstruction to prevent anatomical distortion. Specifically, an invertible hierarchy flow (IHF) performs hierarchical subtractive coupling to progressively remove artefact-related features, while an artefact-aware normalization (AAN) employs anatomy-fixed feature modulation to accurately transfer target characteristics. Combined with anatomy and artefact consistency loss objectives, IHF-Harmony achieves high-fidelity harmonization that retains source anatomy. Experiments across multiple MRI modalities demonstrate that IHF-Harmony outperforms existing methods in both anatomical fidelity and downstream task performance, facilitating robust harmonization for large-scale multi-site imaging studies. Code is available at https://github.com/Idea89560041/IHF-Harmony.

Pengli Zhu, Yitao Zhu, Haowen Pang +1

Department of Health Technology and Informatics, The Hong Kong Polytechnic University, Hong Kong · School of Integrated Circuits and Electronics, Beijing Institute of Technology, China · Mental Health Research Center, The Hong Kong Polytechnic University, Hong Kong +1

Magnetic Resonance Imaging Unpaired Image-to-Image Translation Normalizing Flows

Ask PDF

Mar 23, 2026

Cycle Inverse-Consistent TransMorph: A Balanced Deep Learning Framework for Brain MRI Registration

Deformable image registration plays a fundamental role in medical image analysis by enabling spatial alignment of anatomical structures across subjects. While recent deep learning-based approaches have significantly improved computational efficiency, many existing methods remain limited in capturing long-range anatomical correspondence and maintaining deformation consistency. In this work, we present a cycle inverse-consistent transformer-based framework for deformable brain MRI registration. The model integrates a Swin-UNet architecture with bidirectional consistency constraints, enabling the joint estimation of forward and backward deformation fields. This design allows the framework to capture both local anatomical details and global spatial relationships while improving deformation stability. We conduct a comprehensive evaluation of the proposed framework on a large multi-center dataset consisting of 2851 T1-weighted brain MRI scans aggregated from 13 public datasets. Experimental results demonstrate that the proposed framework achieves strong and balanced performance across multiple quantitative evaluation metrics while maintaining stable and physically plausible deformation fields. Detailed quantitative comparisons with baseline methods, including ANTs, ICNet, and VoxelMorph, are provided in the appendix. Experimental results demonstrate that CICTM achieves consistently strong performance across multiple evaluation criteria while maintaining stable and physically plausible deformation fields. These properties make the proposed framework suitable for large-scale neuroimaging datasets where both accuracy and deformation stability are critical.

Jiaqi Shang, Haojin Wu, Yinyi Lai +3

Medical Image Registration Deformable Image Registration Brain MRI

Ask PDF