MAGEFormer: Learning Metric-Consistent Representations for Anisotropic CT Segmentation

Jiaying Li; Paolo Remagnino

cs.CVOct 2, 2026

MAGEFormer: Learning Metric-Consistent Representations for Anisotropic CT Segmentation

Authors: Jiaying Li, Paolo Remagnino

Organizations: Department of Computer Science, Durham University, Durham, UK

Abstract

Vision Transformers (ViTs) have shown strong performance in volumetric segmentation, but their effectiveness on clinical CT is limited by an isotropic Euclidean lattice assumption. This conflicts with anisotropic CT acquisition, leading to two key issues: (1) a metric mismatch between voxel indices and physical anatomy, and (2) accuracy degradation from isotropic resampling. To address this, we propose MAGEFormer, a geometry-calibrated framework that embeds physical metric constraints directly into representation learning. Our method introduces Metric-Adaptive Spatial Embedding (MASE) to calibrate positional frequencies using voxel spacing, Geometry-Constrained Attention (GCA) to suppress physically implausible feature correlations, and Geometric View Voting (GVV) to reduce discretization bias during inference. We evaluate MAGEFormer on two multi-organ abdominal CT benchmarks, BTCV and FLARE 22, under a unified protocol against strong CNN and Transformer-based baselines. MAGEFormer achieves the strongest boundary accuracy among the compared methods, with 10.58 mm HD95 on BTCV and 3.40 mm HD95 on FLARE 22, and shows consistent gains in Dice under the same protocol. These results show that geometry-aware internal calibration is more effective than relying on conventional isotropic preprocessing alone for anisotropic CT segmentation.

Figures & tables

Figure 1: MAGEFormer: Architecture Overview and Geometric Motivation

	Spl	RKi	LKi	Gal	Eso	Liv	Sto	Aor	IVC	Pan	AG §	V/D ¶
	BTCV
TransUNet †	87.03	78.60	81.45	57.96	66.95	92.97	75.80	86.42	71.61	57.62	47.95	60.92
nnU-Net †	90.91	90.17	87.83	70.59	78.33	95.97	88.30	91.67	85.84	81.47	71.01	76.46
UNETR †	83.70	82.41	79.51	56.59	64.89	91.70	75.06	85.01	75.35	54.43	53.14	56.86
Swin-UNETR †	83.23	78.03	81.81	56.35	69.90	93.07	78.22	86.97	81.41	72.58	61.36	69.05
U-Mamba †	90.24	91.20	89.33	68.63	79.13	95.25	87.54	92.85	86.36	81.95	71.64	77.05

Table 1: Per-organ Dice (%,

\uparrow

) on BTCV and FLARE 22 multi-organ CT segmentation. Spl: spleen, RKi: right kidney, LKi: left kidney, Gal: gallbladder, Eso: esophagus, Liv: liver, Sto: stomach, Aor: aorta, IVC: inferior vena cava, Pan: pancreas.

	FLARE 22		BTCV
Method	Dice	HD95	Dice	HD95
TransUNet †	84.51 ${\pm}$ 0.65	15.90 ${\pm}$ 1.87	70.25 ${\pm}$ 3.81	28.16 ${\pm}$ 5.15
nnU-Net †	93.55 ${\pm}$ 0.34	4.16 ${\pm}$ 1.23	83.04 ${\pm}$ 2.30	13.79 ${\pm}$ 5.44
UNETR †	85.85 ${\pm}$ 1.09	13.39 ${\pm}$ 2.54	70.14 ${\pm}$ 2.30	64.13 ${\pm}$ 29.37
Swin-UNETR †	89.22 ${\pm}$ 0.74	31.30 ${\pm}$ 14.97	74.87 ${\pm}$ 2.72	75.21 ${\pm}$ 24.51
U-Mamba †	93.48 ${\pm}$ 0.41	4.57 ${\pm}$ 1.90	83.29 ${\pm}$ 2.45	11.91 ${\pm}$ 4.16

Table 2: Comparison of segmentation performance on FLARE 22 and BTCV datasets. Results are mean

\pm

std over five folds. Best mean values are in bold ; green indicates the lowest standard deviation.

Method	MASE	GCA	GVV	Avg DSC ( $\uparrow$ )	HD95 (mm, $\downarrow$ )
Baseline	✗	✗	✗	0.7487	75.21
+ MASE	✓	✗	✗	0.7978 (+4.91%)	37.14 ( $-$ 38.07)
+ GCA	✗	✓	✗	0.7993 (+5.06%)	34.27 ( $-$ 40.94)
+ GVV	✗	✗	✓	0.7871 (+3.84%)	62.95 ( $-$ 12.26)
+ MASE + GCA	✓	✓	✗	0.8295 (+8.08%)	18.76 ( $-$ 56.45)
Ours (Full)	✓	✓	✓	0.8411 (+9.24%)	10.58 ( $-$ 64.63)

Table 3: Ablation study of proposed components (BTCV). MASE : Metric-Adaptive Spatial Embedding; GCA : Geometry-Constrained Attention; GVV : Geometric View Voting.

Panel of Figure 2 — Figure 2: Ablation and robustness analysis. (a) Performance stratified by natural anisotropy ratio, showing increasing advantage of MAGEFormer over the baseline as anisotropy grows. (b) Controlled synthetic z-degradation experiment demonstrating the robustness gap widens with increasing downsampling factor. (c) Inference speed vs. accuracy trade-off of Geometric View Voting (GVV), achieving comparable accuracy to standard TTA with $4\times$ speedup.

Figure 3: Qualitative visualization of different architectures. From left to right: (a) Ground Truth, (b) TransUNet [ 1 ] , (c) nnU-Net [ 9 ] , (d) UNETR [ 7 ] , (e) Swin-UNETR [ 6 ] , (f) U-Mamba [ 10 ] , (g) Ours. Compared with baselines, MAGEFormer yields more anatomically consistent and physically plausible predictions, with less multi-label fragmentation and fewer intra-organ voids that are anatomically implausible.

Explore similar work

ASAP: Advancing Medical Volumetric Representation Learning with Anatomy-aware Semantically-adaptive Pre-training
May 30, 2026Rongsheng Wang, Fenghe Tang, Zihang Jiang +10Medical VLMs Medical Image Grounding
School of Biomedical Engineering, Division of Life Sciences and Medicine, University of Science and Technology of China (USTC), Hefei, Anhui, China 230026 · Medical Imaging, Robotics, Analytic Computing & Learning (MIRACLE) Lab, YRD-RIGHT, USTC Suzhou Institute for Advanced Research, Suzhou, Jiangsu, China 215123 · Jiangsu Provincial Key Laboratory of Multimodal Digital Twin Technology, Suzhou, Jiangsu, China 215123 +5
Automatic Segmentation of 3D CT scans with SAM2 using a zero-shot approach
Mar 24, 2026Miquel Lopez Escoriza, Pau Amargant Alvarez3D Medical Image Segmentation Zero-Shot Medical Image Segmentation
Department of Computer Science, EPFL, Switzerland
Towards Voxel Spacing Consistency for Medical Image Segmentation
Jun 30, 2026Xin You, Runze Yang, Minghui Zhang +63D Medical Image Segmentation
the Institute of Medical Robotics, Shanghai Jiao Tong University, 200240 Shanghai, China · CAMPAR, Technical University of Munich, 80333 Munich, Germany · the Department of Ultrasound, Shanghai Chest Hospital, 200052 Shanghai, China

May 30, 2026J/K move · Enter open · S save

ASAP: Advancing Medical Volumetric Representation Learning with Anatomy-aware Semantically-adaptive Pre-training

Rongsheng Wang, Fenghe Tang, Zihang Jiang, Yingtai Li+9

School of Biomedical Engineering, Division of Life Sciences and Medicine, University of Science and Technology of China (USTC), Hefei, Anhui, China 230026 · Medical Imaging, Robotics, Analytic Computing & Learning (MIRACLE) Lab, YRD-RIGHT, USTC Suzhou Institute for Advanced Research, Suzhou, Jiangsu, China 215123 · Jiangsu Provincial Key Laboratory of Multimodal Digital Twin Technology, Suzhou, Jiangsu, China 215123+5

Medical VLMs Medical Image Grounding Medical Image Representation Learning Medical Image Benchmarks

Open paper AskPDF arXiv

Learning transferable and interpretable representations from medical volumetric scans remains challenging due to complex anatomical structures and weak, heterogeneous supervision provided by radiology reports. In this paper, we propose Anatomy-aware Semantically-Adaptive Pre-training (ASAP), a principled vision-language pre-training framework for fine-grained medical volumetric representation learning from large-scale chest CT scans and their corresponding radiology reports. ASAP integrates three key components: (1) an anatomy-aware knowledge injection module that incorporates organ-level structural priors via off-the-shelf segmentation tool to encourage anatomically coherent representations; (2) a semantically-adaptive selective alignment mechanism that dynamically associates sentence-level findings with localized volumetric regions; and (3) a semantically-adaptive fusion module for effective interaction between anatomically informed visual features and grounded textual cues under dual-modal masked modeling paradigm. Beyond methodological contributions, we establish a comprehensive benchmark for medical volumetric vision-language pre-training on chest CT, covering 15 datasets and 22 downstream tasks spanning abnormality classification, segmentation, disease prognosis prediction, report generation, vocabulary classification, cross-modal retrieval and visual question answering. This benchmark provides standardized evaluation protocols to systematically assess representation quality under diverse clinical settings and data regimes. Extensive experiments demonstrate that ASAP consistently achieves state-of-the-art performance across tasks and datasets, with particularly pronounced gains under limited supervision and distribution shift, validating its effectiveness in learning transferable and clinically meaningful volumetric representations.

Abstract

Explore similar work

ASAP: Advancing Medical Volumetric Representation Learning with Anatomy-aware Semantically-adaptive Pre-training

Automatic Segmentation of 3D CT scans with SAM2 using a zero-shot approach

Towards Voxel Spacing Consistency for Medical Image Segmentation