In recent years, VAD (Visual Anomaly Detection) has attracted significant attention due to its high interpretability, as it provides visual explanations by highlighting anomalous regions within an image. However, although these visual explanations can be helpful, they lack a direct and semantically meaningful interpretation for users. To address this limitation, we propose extending CBMs (Concept Bottleneck Models) to the VAD setting. By learning meaningful concepts, the network can provide human-interpretable descriptions of anomalies, offering a novel and more insightful way to explain them. Our main contributions are: (i) we introduce a concept-based framework for anomaly explanation by extending CBMs to the VAD setting for the first time; (ii) we evaluate multiple supervision regimes, ranging from supervised to unsupervised anomaly settings; (iii) we propose a dual-branch architecture that combines a CBM-based Concept branch for concept-level explanations with a Visual branch for pixel-level anomaly localization, bridging semantic and spatial interpretability. When evaluated across three well-established VAD benchmarks, our approach, Concept-Aware Visual Anomaly Detection (\ourmethod), achieves performance comparable to classic VAD methods (up to 0.97 I-AUROC on MVTec AD) while providing richer, concept-driven explanations. A user study (582 responses) shows that \ourmethod{} explanations are rated effective in 73.5% of cases (vs. 60.7% for the SOTA VLM-based explainer Anomaly-OV) and misleading in only 3.1% (vs. 11.7%). Beyond explanation quality, CBMs enable human-in-the-loop correction: concept-level interventions further boost detection in the challenging unsupervised setting, closing part of the gap to fully supervised performance.
Figures & tables
Figure 1: (Left) CONVAD Architecture, (Right) Output example of CONVAD: the input image, the anomaly heatmap, a list of most active and inactive concepts, and the prediction.
Figure 2: Pipeline for creating the Concept Dataset through concept annotation of a VLM .
Figure 3: Example of concept vocabulary and distribution across defect types for the MVTec hazelnut category.
Metric
Zero-Shot
Unsupervised (Zero-Anomaly-Shot)
Semi-Supervised
Supervised
Anomaly-OV
CONVAD
PatchCore
STFPM
Tiny-Dinomaly
AnomalyGPT
CONVAD
DevNet
DRA
CONVAD
UNet
I-AUC
0.94
0.81
0.86
0.88
0.91
0.97
0.87
0.80
0.76
0.96
0.65
P-AUC
✗
0.91
0.96
0.88
0.96
0.93
0.92
0.92
✗
0.95
0.77
C-AUC
✗
0.73
✗
✗
✗
✗
0.72
✗
✗
0.86
✗
Memory [MB] ↓
1,768
17.6
38.4
11.1
36.5
30,520
17.6
44.7
44.7
17.6
10.3
Table 1: Comparison of CONVAD and other VAD methods across unsupervised, semi-supervised, and supervised scenarios on the MVTec AD dataset. ✗ indicates that the method cannot produce the corresponding output (pixel localization or concept-level prediction).
CONVAD
PatchCore
AnomalyGPT
Dataset
C-AUC
I-AUC
I-AUC
I-AUC
MVTec AD
0.86
0.97
0.99
0.96
VisA
0.76
0.94
0.93
0.91
Real-IAD
0.88
0.94
0.90
—
Table 2: (Left) Average results on MVTec AD, VisA, and Real-IAD (Supervised) for the Concept branch vs. PatchCore (WideResNet-50, although unsupervised, is a strong baseline) and AnomalyGPT (4-normal-shot). (Right) Ablation of CONVAD on MVTec AD across supervision levels; I-AUC reported for the Concept branch, Visual branch, and their fusion. Concept Rate is the fraction of samples where the Concept branch drives the final prediction.
Method
Helpful
Partial
Unhelpful ↓
AnomalyGPT [ 14 ]
33.2%
22.6%
44.2%
Anomaly-OV [ 42 ]
59.2%
22.5%
18.3%
CONVAD
55.6%
28.6%
15.8%
Table 3: User study results (582 responses, 32 participants). Left : perceived helpfulness of explanations. Right : explanation quality signals; effective : the explanation enabled the user to verify whether the prediction was correct; misleading : it convinced the user to trust a wrong prediction (counting defined in Eq. 11 ).
Figure 4: Performance gain in MVTec categories for increasing number of intervened concepts. Unsupervised setting.
Figure 5: t-SNE of concept-logit embeddings for metal_nut in the (Left) unsupervised and (Right) semi-supervised settings. Color encodes the split/label ( ∙ train normal, ∙ train anomalous (generated), ∙ test normal, ∙ test anomalous (real), ∙ train anomalous (real)), while marker shape encodes the defect type ( ∙ good, ■ bent, ◀ color, + flip, ★ scratch). Generated anomalies (Left) cluster apart from the real test anomalies; adding one real anomaly per defect type (Right) pulls real and synthetic anomalies into alignment.
Figure 6: Examples of well-generated synthetic anomalous images.
All
Anomaly Concepts
Normal Concepts
Acc
0.93 ± 0.08
0.92 ± 0.08
0.98 ± 0.01
Prec
0.76 ± 0.24
0.71 ± 0.23
0.99 ± 0.00
Rec
0.77 ± 0.28
0.72 ± 0.28
0.99 ± 0.01
Table 4: Average value and standard deviation of accuracy, precision and recall of predicted concepts over the hazelnut category.
Model type
C-AUC
I-AUC
Joint
0.86
0.97
Sequential
0.85
0.95
Independent
0.87
0.96
Table 5: Comparison of the results of the three CBM learning paradigms over the MVTec AD dataset, in terms of concept prediction performance and anomaly detection.
Rank
Isotonic Reg.
Fusion
AUROC
Agree
AUROC
Agree
Max
0.81
87.31%
0.76
84.62%
Mean
0.88
82.95%
0.87
73.29%
Learned
0.90
79.49%
0.87
73.28%
Learned+C
0.91
76.52%
0.88
78.50%
Table 6: Ablation study on calibration (columns) and fusion (rows) strategies in the unsupervised setting on MVTec AD.
Figure 7: Hazelnut t-SNE embeddings for the SAG scenario.
Figure 8: t-SNE representations of concept logits embeddings using metal_nut . Black markers highlight the few real samples used for Supervised+SAG scenarios.
Figure 9: Concept distribution for the capsule category (MVTec AD).
Figure 10: Concept distribution for the metal_nut category (MVTec AD).
Figure 11: Concept distribution for the pcb2 category (VisA).
Figure 12: Concept distribution for the rolled_strip_base category (Real-IAD).