Training-Free Zero-Shot Anomaly Detection in 3D Brain MRI with 2D Foundation Models

Tai Le-Gia; Jaehyun Ahn

cs.CVFeb 17, 2026

Training-Free Zero-Shot Anomaly Detection in 3D Brain MRI with 2D Foundation Models

Authors: Tai Le-Gia, Jaehyun Ahn

Organizations: Department of Mathematics, Chungnam National University, South Korea

Abstract

Zero-shot anomaly detection (ZSAD) has gained increasing attention in medical imaging as a way to identify abnormalities without task-specific supervision, but most advances remain limited to 2D datasets. Extending ZSAD to 3D medical images has proven challenging, with existing methods relying on slice-wise features and vision-language models, which fail to capture volumetric structure. In this paper, we introduce a fully training-free framework for ZSAD in 3D brain MRI that constructs localized volumetric tokens by aggregating multi-axis slices processed by 2D foundation models. These 3D patch tokens restore cubic spatial context and integrate directly with distance-based, batch-level anomaly detection pipelines. The framework provides compact 3D representations that are practical to compute on standard GPUs and require no fine-tuning, prompts, or supervision. Our results show that training-free, batch-based ZSAD can be effectively extended from 2D encoders to full 3D MRI volumes, offering a simple and robust approach for volumetric anomaly detection.

Figures & tables

Method	Training Source	Patient-level		Voxel-level
Method	Training Source	AUROC	AP	AUROC	AP	Dice	IoU
Zero-shot Methods (No medical training)
CoDeGraph3D	—	96.9	92.7	92.2	49.1	41.3	31.6
WinCLIP	—	23.2	24.5	84.9	5.8	8.1	4.4
AnomalyCLIP	Industrial	36.4	28.6	91.2	11.8	14.1	8.4
APRIL-GAN	Industrial	3.5	21.2	90.0	8.7	12.7	7.3

Table 1: Quantitative comparison on T2-weighted MRI. Bold indicates the best performance within each category (Zero-Shot vs. Reference).

Method	Training Source	Patient-level		Voxel-level
Method	Training Source	AUROC	AP	AUROC	AP	Dice	IoU
Zero-shot Methods (No medical training)
CoDeGraph3D	—	97.5	94.1	90.2	33.8	29.5	20.8
WinCLIP	—	34.6	38.1	70.0	2.6	4.4	2.3
AnomalyCLIP	Industrial	75.1	66.7	84.4	6.6	7.7	4.2
APRIL-GAN	Industrial	19.1	26.5	79.2	3.8	6.4	3.4

Table 2: Quantitative comparison on T1-weighted MRI. Bold indicates the best performance within each category (Zero-Shot vs. Reference).

Figure 1: Anomaly segmentation on 3D T2-weighted BraTS scans. Yellow contours show ground-truth boundaries, and blue regions denote predicted anomaly masks. Rows illustrate high-, medium-, and low-Dice cases for CoDeGraph3D and APRIL-GAN baselines.

Table 4

Number of Chunks	Batch Size ( $B$ )	Patient AUROC	Dice
1 (Full Batch)	180	96.9	41.3
2	90	96.7	40.8
4	45	96.4	40.0
6	30	96.0	39.8
12	15	95.2	37.5

Table 5: Performance Stability with Chunking. Patient-level AUROC and voxel-level Dice when the full test set is divided into random non-overlapping chunks of varying sizes.

Method	BraTS-2021 GLI (Tumor)				ATLAS R2.0 (Stroke)
Method	P-AUROC	P-AP	V-AP	Dice	P-AUROC	P-AP	V-AP	Dice
CoDeGraph3D	99.4	98.9	51.6	61.0	87.2	72.6	34.4	31.6
AnomalyCLIP	68.1	57.4	17.1	22.5	53.7	47.2	4.1	4.8
APRIL-GAN	26.4	31.1	13.6	18.8	17.9	23.4	4.0	6.0
DAE	99.2	98.5	40.7	49.1	98.5	93.9	9.6	13.1

Table 6: Generalization to Glioma and Stroke. Evaluation on BraTS-2021 GLI (T2w) and ATLAS R2.0 (T1w) against 115 IXI normals. Baselines: AnomalyCLIP/APRIL-GAN are fine-tuned on MVTec (Industrial); DAE is trained on IXI (Medical-Unsupervised).

Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

Backbone	Patient AUROC (%)	Dice (%)
CLIP	95.5	36.6
DINOv2	96.9	41.3

Appendix

Table 7: Effect of Backbone Choice on BraTS T2-weighted. Patient-level AUROC and voxel-level Dice of CoDeGraph3D using different 2D ViT encoders.

Batch Size	Extraction	Scoring	Total Time	Avg Time	Peak VRAM
( $B$ )	(s)	(s)	(s)	(s / vol)	(GB)
30	84.9	14.2	99.1	3.30	7.16
60	169.5	37.6	207.1	3.45	7.16
90	254.4	73.8	328.1	3.65	7.69
120	339.2	111.6	450.9	3.76	7.86
150	423.2	150.4	573.5	3.82	8.72

Appendix

Table 8: Computational Cost Analysis. Breakdown of runtime and memory usage for increasing batch sizes. Extraction includes encoder forward pass and 3D tokenization; Scoring includes random projection, batch-graph construction, and voxel-level anomaly scoring.

Setting	AnomalyCLIP	APRIL-GAN
Backbone	ViT-L-14-336	ViT-L-14-336
Input size	$224\times 224$	$224\times 224$
Batch size	8	8
Epochs	15	15
Learning rate	0.001	0.001
Feature layers	[24]	[6, 12, 18, 24]

Appendix

Table 9: Training configuration for CLIP-based baselines.

Dataset	AUROC	AP
BraTS-2025 METS (T1w)	86.9	78.7
BraTS-2025 METS (T2w)	58.4	42.8
BraTS-2021 GLI (T2w)	75.4	68.8
ATLAS R2.0 (T1w)	62.5	51.5

Appendix

Table 10: WinCLIP Patient-Level Performance with Slice-wise Aggregation. Patient-level anomaly classification results obtained by aggregating slice-wise CLS-token anomaly scores (max over slices).

Setting	Value
Input crop size	$160^{3}$
Voxel spacing	$1.0\,\text{mm}$ isotropic
Z-normalization	Yes
Noise level	Gaussian, $\sigma=3.0$
Noise resolution	$20\times 20\times 20$
Epochs	100

Appendix

Table 11: Configuration of the 3D DAE baseline.

Medical Image Anomaly Detection

Medical image anomaly detection finds and localizes unusual anatomy or pathology in scans, often when labeled disease examples are scarce.

See all 87 papers

Toward Training-Free Zero-Shot Anomaly Detection in 3D Medical Images: A Batch-Based Approach Using 2D Foundation Models
Jun 17, 2026Tai Le-GiaMedical Image Anomaly Detection Zero-Shot Anomaly Detection
Department of Mathematics, Chungnam National University, Daejeon, Republic of Korea
HiMatch-AD: DINOv3-driven Hierarchical Matching for Training-free Medical Anomaly Detection
Jun 21, 2026Jiayu Huo, Jingyuan Hong, Meng Zhou +2Medical Image Anomaly Detection
Imperial College London, London, UK · King’s College London, London, UK · SonoScape Medical Corp. +1
VisualAD: Language-Free Zero-Shot Anomaly Detection via Vision Transformer
Mar 9, 2026Yanning Hou, Peiyuan Li, Zirui Liu +4Zero-Shot Anomaly Detection Vision Foundation Model Adaptation
College of Intelligence Science and Technology, National University of Defense Technology, Changsha, China · School of Artificial Intelligence, Anhui University, Hefei, China · State Key Laboratory of Opto-Electronic Information Acquisition and Protection Technology, Anhui University, Hefei, China

Jun 17, 2026J/K move · Enter open · S save

Toward Training-Free Zero-Shot Anomaly Detection in 3D Medical Images: A Batch-Based Approach Using 2D Foundation Models

Tai Le-Gia

Department of Mathematics, Chungnam National University, Daejeon, Republic of Korea

Medical Image Anomaly Detection Zero-Shot Anomaly Detection Medical Image Localization Anomaly Detection

Open paper AskPDF arXiv

Zero-shot anomaly detection (ZSAD) is attractive for medical imaging because clinical systems must handle heterogeneous acquisition protocols, changing patient populations, and pathologies for which annotated training data may be unavailable. Most existing zero-shot anomaly detection methods are designed for 2D images, and their direct extension to 3D medical volumes is limited by the scarcity of large-scale volumetric foundation models or by the difficulty of utilizing volumetric context. We propose CS3F, a training-free batch-based framework for ZSAD in 3D medical images using 2D foundation models. Each volume is decomposed along multiple anatomical axes and encoded slice-wise by a 2D vision transformer. These are then converted into localized volumetric tokens by pooling neighboring slice features. Anomaly scores are obtained from cross-subject mutual similarity: tokens that lack close analogues in other subjects are assigned higher anomaly scores. To reduce the attenuation of focal lesion signals caused by depth pooling, we introduce a coarse-to-fine tokenization strategy that enables fine-resolution volumetric scoring without exhaustive matching. CS3F is evaluated on brain MRI across metastases, glioma, and stroke, as well as validated on lung CT to test generalizability beyond atlas-aligned brain MRI. The results show that frozen 2D foundation models can support anomaly localization in 3D medical images, and that the benefit of fine tokenization depends strongly on lesion contrast and imaging modality.

Abstract

Medical Image Anomaly Detection

Explore similar work

Toward Training-Free Zero-Shot Anomaly Detection in 3D Medical Images: A Batch-Based Approach Using 2D Foundation Models

HiMatch-AD: DINOv3-driven Hierarchical Matching for Training-free Medical Anomaly Detection

VisualAD: Language-Free Zero-Shot Anomaly Detection via Vision Transformer