NEMESIS: Noise-suppressed Efficient MAE with Enhanced Superpatch Integration Strategy

Kyeonghun Kim; Hyeonseok Jung; Youngung Han; Hyunsu Go; Eunseob Choi; Seongbin Park; Junsu Lim; Jiwon Yang

cs.CVApr 2, 2026

NEMESIS: Noise-suppressed Efficient MAE with Enhanced Superpatch Integration Strategy

Authors: Kyeonghun Kim, Hyeonseok Jung, Youngung Han, Hyunsu Go, Eunseob Choi, Seongbin Park, Junsu Lim, Jiwon Yang, +4 more

Organizations: OUTTA · Chung-Ang University · Seoul National University · GIST · Sangmyung University · NVIDIA

Abstract

Volumetric CT imaging is essential for clinical diagnosis, yet annotating 3D volumes is expensive and time-consuming, motivating self-supervised learning (SSL) from unlabeled data. However, applying SSL to 3D CT remains challenging due to the high memory cost of full-volume transformers and the anisotropic spatial structure of CT data, which is not well captured by conventional masking strategies. We propose NEMESIS, a masked autoencoder (MAE) framework that operates on local 128x128x128 superpatches, enabling memory-efficient training while preserving anatomical detail. NEMESIS introduces three key components: (i) noise-enhanced reconstruction as a pretext task, (ii) Masked Anatomical Transformer Blocks (MATB) that perform dual-masking through parallel plane-wise and axis-wise token removal, and (iii) NEMESIS Tokens (NT) for cross-scale context aggregation. On the BTCV multi-organ classification benchmark, NEMESIS with a frozen backbone and a linear classifier achieves a mean AUROC of 0.9633, surpassing fully fine-tuned SuPreM (0.9493) and VoCo (0.9387). Under a low-label regime with only 10% of available annotations, it retains an AUROC of 0.9075, demonstrating strong label efficiency. Furthermore, the superpatch-based design reduces computational cost to 31.0 GFLOPs per forward pass, compared to 985.8 GFLOPs for the full-volume baseline, providing a scalable and robust foundation for 3D medical imaging.

Figures & tables

Fig. 1: The overall architecture of the NEMESIS backbone. The input superpatch is divided into volume patches and processed through the patch embedding module. The framework utilizes dual-masking and MATB-based encoders and decoders to learn high-level anatomical representations in the latent space.

Fig. 2: Overview of the superpatch-based pretraining pipeline. NEMESIS randomly crops $128^{3}$ superpatches from the original 3D CT volume to ensure memory efficiency and robust feature extraction.

Fig. 3: Detailed architecture of the 3D Adaptive Patch Embedding module, integrating Linear Projection (LP) and Superpatch-Embedder (SE) pathways.

Fig. 4: Detailed architecture of the MATB, featuring parallel axis-wise and plane-wise attention streams to capture comprehensive 3D context.

Dataset	Modality	# Scans	Task	# Classes
Pretraining (Unlabeled)
TotalSegmentator	CT	28	MAE	N/A
Mixed public CT datasets	CT	140	MAE	N/A
Downstream (Labeled)
BTCV	CT	30	Organ-presence cls.	8

TABLE I: Summary of datasets and tasks.

Method	Protocol	AUROC	F1
ResNet3D-50	Fine-tune	0.9720	0.782
SuPreM	Fine-tune	0.9493	0.678
VoCo	Fine-tune	0.9387	0.701
Random ViT	Fine-tune	0.5602	0.000 †
NEMESIS (Ours)	Frozen backbone + linear classifier	0.9633	0.753
NEMESIS (Ours)	Fine-tune	0.9702	0.791

TABLE II: Comparison of NEMESIS with baseline methods on BTCV organ-presence classification.

Method	10%	25%	50%	100%
NEMESIS (Linear)	0.9075	0.9347	0.9570	0.9633
ResNet3D-50	0.9191	0.9473	0.9612	0.9720
SuPreM	0.8826	0.9252	0.9409	0.9493
VoCo	0.8664	0.8940	0.9265	0.9387

TABLE III: Performance across different label ratios (AUROC).

Strategy	PSNR (dB) $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$
Axis-only masking	24.87	0.294	0.392
Plane-only masking	27.75	0.529	0.164

TABLE IV: Reconstruction quality under different masking strategies at a masking ratio of 0.75.

Embedding dim.	Mask ratio	AUROC	F1
384	0.50	0.9660	0.755
576	0.50	0.9605	0.760
768	0.50	0.9633	0.753
768	0.75	0.9643	0.781

TABLE V: Hyperparameter sensitivity analysis.

Fig. 5: Computational efficiency analysis. (a) Efficiency-quality trade-off. (b) GFLOPs per forward pass. NEMESIS ( $SP=128^{3}$ ) achieves a $32\times$ reduction in GFLOPs compared to the VAE baseline while maintaining superior reconstruction quality.

Explore similar work

MAESIL: Masked Autoencoder for Enhanced Self-supervised Medical Image Learning
Apr 1, 2026Kyeonghun Kim, Hyeonseok Jung, Youngung Han +14Self-Supervised Pre-Training Medical Image Representation Learning
NVIDIA ATC, Taiwan
MAE-Based Self-Supervised Pretraining for Data-Efficient Medical Image Segmentation Using nnFormer
Apr 22, 2026R. M. Krishna Sureddi, T. Satyanarayana Murthy, Nomula Varsha Reddy +2Self-Supervised Pre-Training Medical Image Segmentation
Information Technology, Chaitanya Bharathi Institute of Technology, Hyderabad, India
DALE-CT: Depth-Aware Foundation Models for Computed Tomography
Jun 5, 2026Evan W. Damron, Mahmut S. Gokmen, Mitchell A. Klusty +3Medical Imaging Foundation Models Self-Supervised Pre-Training
College of Medicine Office of Research, University of Kentucky, Lexington, KY 40506 USA

Apr 1, 2026J/K move · Enter open · S save

MAESIL: Masked Autoencoder for Enhanced Self-supervised Medical Image Learning

Kyeonghun Kim, Hyeonseok Jung, Youngung Han, Junsu Lim+13

NVIDIA ATC, Taiwan

Self-Supervised Pre-Training Medical Image Representation Learning 3D Medical Imaging Masked Autoencoders

Open paper AskPDF arXiv

Training deep learning models for three-dimensional (3D) medical imaging, such as Computed Tomography (CT), is fundamentally challenged by the scarcity of labeled data. While pre-training on natural images is common, it results in a significant domain shift, limiting performance. Self-Supervised Learning (SSL) on unlabeled medical data has emerged as a powerful solution, but prominent frameworks often fail to exploit the inherent 3D nature of CT scans. These methods typically process 3D scans as a collection of independent 2D slices, an approach that fundamentally discards critical axial coherence and the 3D structural context. To address this limitation, we propose the autoencoder for enhanced self-supervised medical image learning(MAESIL), a novel self-supervised learning framework designed to capture 3D structural information efficiently. The core innovation is the 'superpatch', a 3D chunk-based input unit that balances 3D context preservation with computational efficiency. Our framework partitions the volume into superpatches and employs a 3D masked autoencoder strategy with a dual-masking strategy to learn comprehensive spatial representations. We validated our approach on three diverse large-scale public CT datasets. Our experimental results show that MAESIL demonstrates significant improvements over existing methods such as AE, VAE and VQ-VAE in key reconstruction metrics such as PSNR and SSIM. This establishes MAESIL as a robust and practical pre-training solution for 3D medical imaging tasks.

Abstract

Explore similar work

MAESIL: Masked Autoencoder for Enhanced Self-supervised Medical Image Learning

MAE-Based Self-Supervised Pretraining for Data-Efficient Medical Image Segmentation Using nnFormer

DALE-CT: Depth-Aware Foundation Models for Computed Tomography