Medical image anomaly detection faces unique challenges due to subtle, heterogeneous anomalies embedded in complex anatomical structures. Through systematic Grad-CAM analysis, we reveal that discriminative activation maps fail on medical data, unlike their success on industrial datasets, motivating the need for manifold-level modeling. We propose PDD (Manifold-Prior Diverse Distillation), a framework that unifies dual-teacher priors into a shared high-dimensional manifold and distills this knowledge into dual students with complementary behaviors. Specifically, frozen VMamba-Tiny and wide-ResNet50 encoders provide global contextual and local structural priors, respectively. Their features are unified through a Manifold Matching and Unification (MMU) module, while an Inter-Level Feature Adaption (InA) module enriches intermediate representations. The unified manifold is distilled into two students: one performs layer-wise distillation via InA for local consistency, while the other receives skip-projected representations through a Manifold Prior Affine (MPA) module to capture cross-layer dependencies. A diversity loss prevents representation collapse while maintaining detection sensitivity. Extensive experiments on multiple medical datasets demonstrate that PDD significantly outperforms existing state-of-the-art methods, achieving improvements of up to 11.8%, 5.1%, and 8.5% in AUROC on HeadCT, BrainMRI, and ZhangLab datasets, respectively, and 3.4% in F1 max on the Uni-Medical dataset, establishing new state-of-the-art performance in medical image anomaly detection. The implementation will be released at https://github.com/OxygenLu/PDD
Figures & tables
Figure 1: Grad-CAM visualization of frozen Vmamba and ResNet across medical and industrial images. Within each group, feature maps progress from low-dimensional to high-dimensional feature representations (top to bottom). At the same feature dimension, the dispersed and aggregated activation patterns of the two backbones form complementary prior information.
Figure 2: Overview of the proposed PDD framework. The framework employs a dual-teacher and dual-student architecture. The teachers consist of frozen VMamba-Tiny and frozen wide-ResNet50 encoders, whose intermediate features are fused via the InA module (shown in (b)) to obtain fbi . The two teacher encoders compress input images into distinct high-dimensional manifold spaces, which are then aligned through the MMU module. The aligned features are fed into two student networks: Student 1 distills features from InA via FEui , while Student 2 incorporates multi-scale manifold space features from the unified manifold through MLP-based skip connections to FEpi , distilling both prior and InA features. This enables diverse reconstruction of normal samples and effective separation of anomalies.
Figure 3: Visualization of PDD pipeline and loss functions. Figure (a) illustrates the training process of PDD, which is trained exclusively on normal samples. The training simultaneously optimizes three loss functions: lprp , lkr , and ldiv . Figure (c) shows the loss function curves over 100 epochs on the ZhangLab Chest-Xray dataset. The combined direction of the three curves is represented in the simulated loss landscape in Figure (d), with arrows indicating the direction of loss descent. In the final 100 epochs, the training converges near the optimal solution, demonstrating that PDD effectively learns features of normal samples and can separate anomalies.
Method →
f-AnoGAN [ 35 ]
CutPaste [ 22 ]
RD4AD [ 6 ]
SQUID [ 40 ]
SIMSID [ 41 ]
Skip-TS [ 30 ]
Ours
Dataset ↓
MIA’19
CVPR’21
CVPR’22
CVPR’23
TPAMI’24
TIM’24
HeadCT
82.6
73.0
74.3
75.4
74.9
85.7
97.5
Zhanglab
75.5
73.3
87.5
87.6
91.1
79.2
94.0
BrainMRI
77.1
67.0
80.9
74.7
81.5
88.2
96.7
CheXpert
65.8
65.5
71.9
78.1
79.7
68.7
79.1
Table 1: Comparison with six baselines on four medical datasets. Evaluation metric: AUROC ( % ). Bold : best; underline : second best.
Method →
RD4AD [ 6 ]
UniAD [ 43 ]
SimpleNet [ 32 ]
DeSTSeg [ 45 ]
DiAD [ 13 ]
MambaAD [ 12 ]
Ours
Category ↓
CVPR’22
NeurIPS’22
CVPR’23
CVPR’23
AAAI’24
NeurIPS’24
brain
82.4/94.4/91.5
89.9/97.5/92.6
82.3/95.6/90.9
84.5/95.0/92.1
93.7 / 98.1 / 95.0
94.2 / 98.6 /94.5
90.9/96.7/ 96.7
liver
55.1/46.3/64.1
55.1/46.3/64.1
55.8/47.6/60.9
69.2 / 60.6 / 64.7
59.2/ 55.6 /60.9
63.2 /53.1/ 64.7
59.7/50.5/ 70.5
retinal
89.2 /86.7/78.5
84.6/79.4/73.9
88.8/87.6/78.6
88.3/83.8/79.2
88.3/86.6/77.7
93.6 / 88.7 / 86.6
93.6 / 92.7 / 88.9
Mean
75.6/75.8/78.0
78.5/75.2/76.6
75.6/76.9/76.8
80.7/79.8/78.7
80.4/ 80.1 /77.8
83.7 / 80.1 / 82.0
81.4 / 80.0 / 85.4
Table 2: Comparison with SoTA methods on Uni-Medical dataset for multi-class anomaly localization with AU-ROC / AP / F1_max metrics. Bold numbers denote the best (SoTA) results in each row, and underlined numbers denote the second-best results.
Model
InA w. MMU (2t1s)
InA w. MMU (2t2s)
RD (1t1s)
MPA
AUROC ↑
AUPR ↑
F1 ↑
M1
✗
✗
✓
✗
81.5
85.5
84.8
M2
✗
✗
✓
✓
82.0
86.1
85.7
M3
✓
✗
✗
✗
90.8
95.7
89.7
M4
✓
✗
✗
✓
92.9
96.6
90.3
Ours
✗
✓
✗
✓
94.0
99.0
96.6
Table 3: Ablation on distillation paradigm and modular components on ZhangLab . Metrics: AUROC / AUPR / F1 max (%).
Model
Ldiv
cos(t1,s1)+cos(t2,s2)
cos(s1,s2)
AUROC ↑
AUPR ↑
F1 ↑
M1
✓
✗
✓
32.50
81.23
93.02
M2
✗
✗
✓
70.46
97.26
95.65
M3
✗
✓
✗
93.41
98.81
95.93
Ours
✓
✓
✗
96.67
99.51
97.56
Table 4: Ablation on student consistency supervision and anomaly map computation strategies on BrainMRI . Metrics: AUROC / AUPR / F1 max (%).
ZhangLab
HeadCT
BrainMRI
τhigh
0.40
0.30
0.75
0.30
0.30
0.75
0.75
τlow
0.65
0.75
0.40
0.65
0.75
0.40
0.40
AUROC
94.52
96.18
95.92
96.92
97.31
96.15
93.30
Table 5: Ablation on τlow and τhigh across datasets. Metric: AUROC (%).
Figure 4: Anomaly localization comparison between PDD and Skip-TS on the HeadCT dataset.
Figure 5: Anomaly localization comparison between PDD and RD4AD on the ZhangLab dataset. PDD produces significantly fewer false positives on normal samples, demonstrating stronger specificity.
Anomaly detection is essential for medical image analysis, where pathological regions often appear as rare deviations from normal anatomical structures. While training-based methods have achieved promising performance, they require task-specific optimization and extensive normal data, which limits scalability across modalities and institutions. Training-free approaches offer greater flexibility by leveraging pretrained visual representations, yet existing methods typically rely on simple nearest-neighbor retrieval and naive aggregation strategies, which may fail to capture hierarchical semantics and ignore the reliability of multiple anomaly responses. In this work, we propose HiMatch-AD, a DINOv3-driven hierarchical matching framework for training-free medical anomaly detection. Our method first retrieves semantically relevant normal references via dual-branch matching that jointly considers global CLS-token similarity and patch-level representations. Hierarchical anomaly maps are then generated across multiple transformer stages by comparing clustered normal features with query representations. To robustly aggregate anomaly responses, we introduce a unified uncertainty-based fusion mechanism that adaptively weights maps according to their reliability. The entire framework operates without any task-specific training. Extensive experiments on the BMAD benchmark, including brain MRI, liver CT, and retinal OCT datasets, demonstrate that HiMatch-AD consistently outperforms both training-based and DINO-based state-of-the-art methods, which highlights the effectiveness of multi-level matching and uncertainty-aware fusion for scalable medical anomaly detection.
Jiayu Huo, Jingyuan Hong, Meng Zhou +2
Imperial College London, London, UK · King’s College London, London, UK · SonoScape Medical Corp. +1
Anomaly detection is a critical task in computer vision with profound implications for medical imaging, where identifying pathologies early can directly impact patient outcomes. While recent unsupervised anomaly detection approaches show promise, they require substantial normal training data and struggle to generalize across anatomical contexts. We introduce D24FAD, a novel dual distillation framework for few-shot anomaly detection that identifies anomalies in previously unseen tasks using only a small number of normal reference images. Our approach leverages a pre-trained encoder as a teacher network to extract multi-scale features from both support and query images, while a student decoder learns to distill knowledge from the teacher on query images and self-distill on support images. We further propose a learn-to-weight mechanism that dynamically assesses the reference value of each support image conditioned on the query, optimizing anomaly detection performance. To evaluate our method, we curate a comprehensive benchmark dataset comprising 13,084 images across four organs, four imaging modalities, and five disease categories. Extensive experiments demonstrate that D24FAD significantly outperforms existing approaches, establishing a new state-of-the-art in few-shot medical anomaly detection. Code is available at https://github.com/ttttqz/D24FAD.
Le Dong, Qinzhong Tan, Chunlei Li +5
Xidian University, China · MedAI Technology (Wuxi) Co. Ltd., China · Technical University of Munich, Germany
Unsupervised medical anomaly detection learns normal anatomical patterns from healthy training images and identifies deviations at test time. Reconstruction-based and diffusion-based methods commonly use the difference between an input image and its reconstruction as anomaly evidence. However, this residual can be ambiguous. Expressive models may preserve pathological structures, while benign anatomical variation, imaging noise, and acquisition differences may also produce large reconstruction errors. We propose discriminative mask-guided diffusion (DMD), a medical anomaly detection framework that complements residual-based localization with reconstruction-shift discrimination. DMD first learns a compact quantized latent representation of normal images. Localized masks then perturb selected latent regions, and a latent diffusion model reconstructs the perturbed representations. The resulting reconstructions are paired with their original normal images to define a self-supervised classification task. At inference, the classifier provides a learned image-level anomaly score, while the residual between the input and its diffusion-based reconstruction yields a pixel-level anomaly map. Experiments on five datasets spanning brain MRI, breast ultrasound, and chest radiography show that DMD achieves the best overall performance among the state-of-the-art baseline methods.
Yibo Wan, Jinyu Cai, Yunhe Zhang +2
School of Computing, National University of Singapore, Singapore · Institute of Data Science, National University of Singapore, Singapore · Department of Computer and Information Science, SKL-IOTSC, University of Macau, China +1