Axial-Centric Cross-Plane Attention for 3D Medical Image Classification

Doyoung Park; Jinsoo Kim; Lohendran Baskaran

cs.CVFeb 25, 2026

Axial-Centric Cross-Plane Attention for 3D Medical Image Classification

Authors: Doyoung Park, Jinsoo Kim, Lohendran Baskaran

Organizations: National Heart Centre Singapore, Singapore · CVS.AI, National Heart Research Institute of Singapore, Singapore · Independent Researcher, Republic of Korea

Abstract

Abridged: Clinicians commonly interpret 3D medical images by examining multiple anatomical planes rather than relying on volumetric views. In clinical CT workflows, the axial plane often serves as the primary diagnostic reference, while the auxiliary planes provide complementary spatial context. However, many existing 3D deep learning approaches either process volumetric data holistically or assign equal importance to all planes, failing to reflect this asymmetric, axial-centric interpretation strategy. To address this, we propose an axial-centric cross-plane attention architecture for 3D medical image classification that models asymmetric dependencies between anatomical planes. The architecture employs large-scale axial CT images pretrained MedDINOv3 as a frozen feature extractor for axial, coronal, and sagittal planes. RICA blocks and intra-plane transformer encoders capture plane-specific positional and contextual information, while axial-centric cross-plane transformer encoders selectively condition axial representations on complementary auxiliary representations. Experiments on six datasets from the MedMNIST3D benchmark show that the proposed method consistently outperforms existing 3D and multi-plane models in ACC and AUC. A lightweight variant, AC-Tiny, achieves competitive performance with substantially fewer trainable parameters, suggesting that architectural design contributes more to performance gains than increased model scale. Ablation studies further validate the importance of axial-centric querying, QKV allocation, directional cross-plane fusion, residual-free cross-attention, and classification head design. Slice-level Grad-CAM visualizations demonstrate that the model identifies diagnostically relevant regions across all planes. These findings highlight the value of aligning architectural design with clinical interpretation workflows for robust 3D medical image analysis.

Figures & tables

Figure 1: Overview of the proposed axial-centric cross-plane attention architecture. Multi-planar inputs pass through a frozen MedDINOv3 backbone for plane-wise feature extraction, followed by RICA blocks for local positional context modeling. Intra-plane transformer encoders independently capture contextual self-attention within each feature sequence. Axial-centric cross-plane transformer encoders condition the primary axial features (queries) on complementary coronal and sagittal information (keys and values). The fused axial-centric representations are passed to two multilayer perceptron (MLP) heads for final classification.

	# of trainable	Fracture3D		Adrenal3D		Nodule3D		Vessel3D		Organ3D		Synapse3D
Methods	params	ACC	AUC	ACC	AUC	ACC	AUC	ACC	AUC	ACC	AUC	ACC	AUC
Official [ Yang et al.(2023)Yang, Shi, Wei, Liu, Zhao, Ke, Pfister, and Ni ]	33M	50.8	71.2	72.1	82.7	84.4	86.3	87.7	87.4	90.7	99.6	74.5	82.0
BSDA [ Zhu et al.(2024b)Zhu, Cai, Wang, Chen, Fu, and Yao ]	33M	56.9	73.1	83.8	89.2	86.1	89.2	93.2	91.7	88.7	99.4	-	-
ViViT-M + R-LLM [ Lai et al.(2024)Lai, Wu, Chen, Zhou, and Hovakimyan ]	295M	56.3	66.9	83.2	84.9	87.4	88.5	90.6	87.0	-	-	-	-
LMTTM-VMI [ Wei et al.(2025)Wei, Yang, Sun, Feng, Wang, and Han ]	21M	56.2	69.1	84.2	84.7	88.3	89.1	93.1	91.7	94.8	99.7	77.7	73.7
Medformer [ Simionescu(2025) ]	-	42.7	53.6	-	-	84.2	87.7	-	-	93.7	99.1	-	-

Table 1: Quantitative comparison of the proposed axial-centric cross-plane attention model variants with six existing 3D medical image classification models on the MedMNIST3D benchmark. The proposed variants include Axial-Centric-Tiny (AC-Tiny,

N=4

H=8

E=192

), Axial-Centric-Small (AC-Small,

N=6

H=4

E=768

), and Axial-Centric-Base (AC-Base,

N=12

H=12

E=768

). The best results are shown in bold . For conciseness, the suffix MNIST is omitted from dataset names.

Figure 2: Slice-level Grad-CAM heatmaps across three MedMNIST3D benchmarks. The slices correspond to the highest attributions from each intra-plane transformer encoder, with the selected indices indicated in parentheses (warmer colors represent stronger attribution). Correctly classified cases exhibit anatomically coherent localization across all three planes, validating effective slice-level selection before cross-plane fusion. In contrast, the misclassified malignant lung nodule shows comparatively scattered and poorly localized heatmaps. Notably, although MedDINOv3 was exclusively pretrained on axial CT slices, anatomically coherent localization is consistently observed across the coronal and sagittal planes, suggesting that feature representations are transferable across auxiliary planes.

Figure 3: Comparison of ablation study designs for axial-centric fusion and architectural modifications. The figure illustrates the Sequential Fusion, Coronal-Centric, Sagittal-Centric, and Reverse query-key-value (QKV) variants, as well as the residual connection in cross-attention design. The proposed architecture employs axial-centric querying and intentionally omits the residual connection following cross-attention, whereas the residual cross-attention variant incorporates the residual pathway, highlighted by the red dotted box.

	Fracture3D		Adrenal3D		Nodule3D		Vessel3D		Organ3D		Synapse3D
Methods	ACC	AUC	ACC	AUC	ACC	AUC	ACC	AUC	ACC	AUC	ACC	AUC
Sequential Fusion	60.4	73.9	88.3	91.6	86.8	84.8	94.0	93.7	94.4	99.8	83.5	84.8
Coronal-Centric	57.5	72.3	87.6	88.6	87.4	87.6	94.8	93.9	95.1	99.8	84.1	85.9
Sagittal-Centric	57.5	72.6	86.2	86.7	86.1	78.3	94.5	95.6	94.6	99.7	84.1	87.6
AC-Base	60.8	71.9	88.6	92.0	89.0	90.2	95.0	93.3	95.7	99.9	86.1	88.0

Table 2: Ablation studies evaluating the axial-centric fusion design on the MedMNIST3D benchmark. All variants employ

N=H=12

and

E=768

. The best results are shown in bold . For conciseness, the suffix MNIST is omitted from dataset names. Figure 3 illustrates the sequential, coronal-centric, and sagittal-centric fusion variants. The sequential variant performs a two-stage axial–coronal–sagittal fusion, while the coronal-centric and sagittal-centric variants use the corresponding plane as the query. The results highlight the importance of axial-centric and directional cross-plane conditioning.

	Fracture3D		Adrenal3D		Nodule3D		Vessel3D		Organ3D		Synapse3D
Methods	ACC	AUC	ACC	AUC	ACC	AUC	ACC	AUC	ACC	AUC	ACC	AUC
Cross-Attn + Residual	57.1	66.4	87.6	91.9	85.2	87.7	94.8	93.7	95.6	99.9	84.9	86.6
GeLU Head	56.7	68.4	86.9	89.9	87.1	86.0	94.8	91.0	95.2	99.8	83.5	86.7
Reverse QKV	57.1	69.9	84.9	86.1	86.5	87.1	94.2	93.2	89.5	99.3	83.0	84.9
AC-Base	60.8	71.9	88.6	92.0	89.0	90.2	95.0	93.3	95.7	99.9	86.1	88.0

Table 3: Ablation studies evaluating the residual connection in cross-attention, the activation function of the classification head, and query-key-value (QKV) allocation on the MedMNIST3D benchmark. All variants employ

N=H=12

and

E=768

. The best results are shown in bold . For conciseness, the suffix MNIST is omitted from dataset names. The results highlight the importance of omitting the residual connection after cross-attention, employing axial-centric QKV allocation, and using the

tanh

activation function in the classification head.

Explore similar work

Apr 7, 2026

Geometrical Cross-Attention and Nonvoid Voxelization for Efficient 3D Medical Image Segmentation

Accurate segmentation of 3D medical scans is crucial for clinical diagnostics and treatment planning, yet existing methods often fail to achieve both high accuracy and computational efficiency across diverse anatomies and imaging modalities. To address these challenges, we propose GCNV-Net, a novel 3D medical segmentation framework that integrates a Tri-directional Dynamic Nonvoid Voxel Transformer (3DNVT), a Geometrical Cross-Attention module (GCA), and Nonvoid Voxelization. The 3DNVT dynamically partitions relevant voxels along the three orthogonal anatomical planes, namely the transverse, sagittal, and coronal planes, enabling effective modeling of complex 3D spatial dependencies. The GCA mechanism explicitly incorporates geometric positional information during multi-scale feature fusion, significantly enhancing fine-grained anatomical segmentation accuracy. Meanwhile, Nonvoid Voxelization processes only informative regions, greatly reducing redundant computation without compromising segmentation quality, and achieves a 56.13% reduction in FLOPs and a 68.49% reduction in inference latency compared to conventional voxelization. We evaluate GCNV-Net on multiple widely used benchmarks: BraTS2021, ACDC, MSD Prostate, MSD Pancreas, and AMOS2022. Our method achieves state-of-the-art segmentation performance across all datasets, outperforming the best existing methods by 0.65% on Dice, 0.63% on IoU, 1% on NSD, and relatively 14.5% on HD95. All results demonstrate that GCNV-Net effectively balances accuracy and efficiency, and its robustness across diverse organs, disease conditions, and imaging modalities highlights strong potential for clinical deployment.

Chenxin Yuan, Shoupeng Chen, Haojiang Ye +3

Shenzhen Institute for Advanced Study, University of Electronic Science and Technology of China, Shenzhen, 518100, Guangdong, China · School of Electrical and Electronic Engineering, Nanyang Technological University, Singapore, 639798, Singapore · School of Science and Engineering, The Chinese University of Hong Kong, Shenzhen, Shenzhen, 518172, Guangdong, China +2

3D Medical Image Segmentation 3D ViTs Cross-Attention

Ask PDF

Jul 15, 2026

TCAM-Diff: Triplane-Aware Cross-Attention Medical Diffusion Model

We introduce TCAM-Diff, a novel 3D medical image generation model that reduces the memory requirements to encode and generate high-resolution 3D data. This model utilizes a decoder-only autoencoder method to learn triplane representation from dense volume and leverages generalization operations to prevent overfitting. Subsequently, it uses a triplane-aware cross-attention diffusion model to learn and integrate these features effectively. Furthermore, the features generated by the diffusion model can be rapidly transformed into 3D volumes using a pre-trained decoder module. Our experiments on three different scales of medical datasets, BrainTumour 128 x 128 x 128, Pancreas 256 x 256 x 256, and Colon 512 x 512 x 512, demonstrate outstanding results. We utilized MSE and SSIM to assess reconstruction quality and leveraged the Wasserstein Generative Adversarial Network (W-GAN) critic to assess generative quality. Comparisons with existing approaches show that our method gives better reconstruction and generation results than other encoder-decoder methods with similar-sized latent spaces.

Zhenkai Zhang, Krista A. Ehinger, Tom Drummond

School of Computing and Information Systems, The University of Melbourne · The University of Melbourne

3D Medical Imaging Medical Image Synthesis 3D Representation Learning 3D Diffusion Models

Ask PDF

Feb 18, 2026

RefineFormer3D: Efficient 3D Medical Image Segmentation via Adaptive Multi-Scale Transformer with Cross Attention Fusion

Accurate and computationally efficient 3D medical image segmentation remains a critical challenge in clinical workflows. Transformer-based architectures often demonstrate superior global contextual modeling but at the expense of excessive parameter counts and memory demands, restricting their clinical deployment. We propose RefineFormer3D, a lightweight hierarchical transformer architecture that balances segmentation accuracy and computational efficiency for volumetric medical imaging. The architecture integrates three key components: (i) GhostConv3D-based patch embedding for efficient feature extraction with minimal redundancy, (ii) MixFFN3D module with low-rank projections and depthwise convolutions for parameter-efficient feature extraction, and (iii) a cross-attention fusion decoder enabling adaptive multi-scale skip connection integration. RefineFormer3D contains only 2.94M parameters, substantially fewer than contemporary transformer-based methods. Extensive experiments on ACDC and BraTS benchmarks demonstrate that RefineFormer3D achieves 93.44% and 85.9% average Dice scores respectively, outperforming or matching state-of-the-art methods while requiring significantly fewer parameters. Furthermore, the model achieves fast inference (8.35 ms per volume on GPU) with low memory requirements, supporting deployment in resource-constrained clinical environments. These results establish RefineFormer3D as an effective and scalable solution for practical 3D medical image segmentation.

Kavyansh Tyagi, Vishwas Rathi, Puneet Goyal

National Institute of Technology Kurukshetra Department of Electronics and Communication Engineering · National Institute of Technology Kurukshetra Department of Computer Engineering · Indian Institute of Technology Ropar Department of Computer Science and Engineering

3D Medical Image Segmentation 3D ViTs Efficient ViTs Hybrid CNN-Transformer Architectures

Ask PDF