In recent years, VAD (Visual Anomaly Detection) has attracted significant attention due to its high interpretability, as it provides visual explanations by highlighting anomalous regions within an image. However, although these visual explanations can be helpful, they lack a direct and semantically meaningful interpretation for users. To address this limitation, we propose extending CBMs (Concept Bottleneck Models) to the VAD setting. By learning meaningful concepts, the network can provide human-interpretable descriptions of anomalies, offering a novel and more insightful way to explain them. Our main contributions are: (i) we introduce a concept-based framework for anomaly explanation by extending CBMs to the VAD setting for the first time; (ii) we evaluate multiple supervision regimes, ranging from supervised to unsupervised anomaly settings; (iii) we propose a dual-branch architecture that combines a CBM-based Concept branch for concept-level explanations with a Visual branch for pixel-level anomaly localization, bridging semantic and spatial interpretability. When evaluated across three well-established VAD benchmarks, our approach, Concept-Aware Visual Anomaly Detection (\ourmethod), achieves performance comparable to classic VAD methods (up to 0.97 I-AUROC on MVTec AD) while providing richer, concept-driven explanations. A user study (582 responses) shows that \ourmethod{} explanations are rated effective in 73.5% of cases (vs. 60.7% for the SOTA VLM-based explainer Anomaly-OV) and misleading in only 3.1% (vs. 11.7%). Beyond explanation quality, CBMs enable human-in-the-loop correction: concept-level interventions further boost detection in the challenging unsupervised setting, closing part of the gap to fully supervised performance.
Figures & tables
Figure 1: (Left) CONVAD Architecture, (Right) Output example of CONVAD: the input image, the anomaly heatmap, a list of most active and inactive concepts, and the prediction.
Figure 2: Pipeline for creating the Concept Dataset through concept annotation of a VLM .
Figure 3: Example of concept vocabulary and distribution across defect types for the MVTec hazelnut category.
Metric
Zero-Shot
Unsupervised (Zero-Anomaly-Shot)
Semi-Supervised
Supervised
Anomaly-OV
CONVAD
PatchCore
STFPM
Tiny-Dinomaly
AnomalyGPT
CONVAD
DevNet
DRA
CONVAD
UNet
I-AUC
0.94
0.81
0.86
0.88
0.91
0.97
0.87
0.80
0.76
0.96
0.65
P-AUC
✗
0.91
0.96
0.88
0.96
0.93
0.92
0.92
✗
0.95
0.77
C-AUC
✗
0.73
✗
✗
✗
✗
0.72
✗
✗
0.86
✗
Memory [MB] ↓
1,768
17.6
38.4
11.1
36.5
30,520
17.6
44.7
44.7
17.6
10.3
Table 1: Comparison of CONVAD and other VAD methods across unsupervised, semi-supervised, and supervised scenarios on the MVTec AD dataset. ✗ indicates that the method cannot produce the corresponding output (pixel localization or concept-level prediction).
CONVAD
PatchCore
AnomalyGPT
Dataset
C-AUC
I-AUC
I-AUC
I-AUC
MVTec AD
0.86
0.97
0.99
0.96
VisA
0.76
0.94
0.93
0.91
Real-IAD
0.88
0.94
0.90
—
Table 2: (Left) Average results on MVTec AD, VisA, and Real-IAD (Supervised) for the Concept branch vs. PatchCore (WideResNet-50, although unsupervised, is a strong baseline) and AnomalyGPT (4-normal-shot). (Right) Ablation of CONVAD on MVTec AD across supervision levels; I-AUC reported for the Concept branch, Visual branch, and their fusion. Concept Rate is the fraction of samples where the Concept branch drives the final prediction.
Method
Helpful
Partial
Unhelpful ↓
AnomalyGPT [ 14 ]
33.2%
22.6%
44.2%
Anomaly-OV [ 42 ]
59.2%
22.5%
18.3%
CONVAD
55.6%
28.6%
15.8%
Table 3: User study results (582 responses, 32 participants). Left : perceived helpfulness of explanations. Right : explanation quality signals; effective : the explanation enabled the user to verify whether the prediction was correct; misleading : it convinced the user to trust a wrong prediction (counting defined in Eq. 11 ).
Figure 4: Performance gain in MVTec categories for increasing number of intervened concepts. Unsupervised setting.
Figure 5: t-SNE of concept-logit embeddings for metal_nut in the (Left) unsupervised and (Right) semi-supervised settings. Color encodes the split/label ( ∙ train normal, ∙ train anomalous (generated), ∙ test normal, ∙ test anomalous (real), ∙ train anomalous (real)), while marker shape encodes the defect type ( ∙ good, ■ bent, ◀ color, + flip, ★ scratch). Generated anomalies (Left) cluster apart from the real test anomalies; adding one real anomaly per defect type (Right) pulls real and synthetic anomalies into alignment.
Figure 6: Examples of well-generated synthetic anomalous images.
All
Anomaly Concepts
Normal Concepts
Acc
0.93 ± 0.08
0.92 ± 0.08
0.98 ± 0.01
Prec
0.76 ± 0.24
0.71 ± 0.23
0.99 ± 0.00
Rec
0.77 ± 0.28
0.72 ± 0.28
0.99 ± 0.01
Table 4: Average value and standard deviation of accuracy, precision and recall of predicted concepts over the hazelnut category.
Model type
C-AUC
I-AUC
Joint
0.86
0.97
Sequential
0.85
0.95
Independent
0.87
0.96
Table 5: Comparison of the results of the three CBM learning paradigms over the MVTec AD dataset, in terms of concept prediction performance and anomaly detection.
Rank
Isotonic Reg.
Fusion
AUROC
Agree
AUROC
Agree
Max
0.81
87.31%
0.76
84.62%
Mean
0.88
82.95%
0.87
73.29%
Learned
0.90
79.49%
0.87
73.28%
Learned+C
0.91
76.52%
0.88
78.50%
Table 6: Ablation study on calibration (columns) and fusion (rows) strategies in the unsupervised setting on MVTec AD.
Figure 7: Hazelnut t-SNE embeddings for the SAG scenario.
Figure 8: t-SNE representations of concept logits embeddings using metal_nut . Black markers highlight the few real samples used for Supervised+SAG scenarios.
Figure 9: Concept distribution for the capsule category (MVTec AD).
Figure 10: Concept distribution for the metal_nut category (MVTec AD).
Figure 11: Concept distribution for the pcb2 category (VisA).
Figure 12: Concept distribution for the rolled_strip_base category (Real-IAD).
Visual anomaly detectors identify deviations from known-normal data, but their anomaly signals may mix evidence of actual anomalies with benign visual variation. We investigate whether automated interpretability can augment visual anomaly detectors by identifying and intervening on different components of this signal. We decompose PatchCore nearest-normal residuals into sparse features using Sparse Autoencoders (SAEs), and provide high-activation and contrastive non-active examples to a Multimodal LLM, which describes each feature and labels it as anomaly, distractor, or uncertain. These labels guide interventions in the SAE hidden representation, where distractor features are suppressed and anomaly features amplified. The edited representation is then used to reconstruct patch embeddings, which are rescored with PatchCore. Across 40 categories from four benchmarks, applying both interventions jointly improves macro-average image-level AUROC from 0.8724 to 0.8857 on source data and from 0.8066 to 0.8210 under synthetic corruptions. On three additional RobustAD categories with real acquisition shifts, the same interventions improve AUROC from 0.8745 to 0.9056 on source data and from 0.6069 to 0.6599 under real acquisition shifts. Finally, individual feature interventions across all 43 categories show that the MLLM labels are aligned in aggregate with how features differently affect normal and anomalous images.
Vision-language models (VLMs) have recently emerged as a promising paradigm for video anomaly detection (VAD) due to their strong visual reasoning ability and natural language-based explainability. In this paper, we aim to address a key limitation of such pipelines, which perform segment-level inference independently owing to token constraints and reason without structured temporal context, allowing VLMs to interpret anomalies as deviations from evolving video dynamics rather than producing fragmented predictions and explanations. To specify, we propose a context-aware framework named LATERN, which reformulates VAD as a temporal evidence aggregation process. LATERN consists of two complementary modules: Context-Aware Anomaly Scoring (CEA) and Recursive Evidence Aggregation (REA). CEA introduces a novel image-grounded memory mechanism, which selectively chooses historical content via frame diversity and visual-textual alignment as expanded context to help generate reliable anomaly scores. Building upon these scores, REA performs recursive temporal aggregation to identify coherent anomaly intervals and produce event-level decisions and explanations grounded in visual-textual evidence. Extensive experiments on challenging benchmarks, including UCF-Crime and XD-Violence, show that LATERN enhances detection accuracy and explanation consistency for frozen VLMs during test time, while generating temporally coherent and semantically grounded event-level explanations.
Video Anomaly Detection (VAD) has traditionally been framed as binary classification or outlier detection, providing neither interpretable reasoning nor precise spatial localization of anomalous events. While Vision-Language Models (VLMs) offer rich scene understanding, they struggle with reliable spatial grounding - often producing hallucinated or geometrically invalid bounding boxes when asked to localize objects. We propose VANGUARD (Video Anomaly Understanding through Reasoning and Grounding), a framework that unifies anomaly classification, spatial grounding, and chain-of-thought reasoning within a single VLM. VANGUARD introduces a three-stage curriculum that progressively layers training objectives: (1) classifier warmup on frozen backbone features, (2) LoRA-adapted spatial grounding, and (3) chain-of-thought generation. To overcome the sparse annotation typical of VAD benchmarks, we employ a teacher-student annotation pipeline in which a VLM (Qwen3-VL-4B) generates structured per-subclip reasoning trajectories based on manual annotations available from the UCA Dataset. Further, GroundingDINO provides bounding box supervision. On UCF-Crime, VANGUARD achieves 94% ROC-AUC with 84% F1 while simultaneously producing interpretable chain-of-thought explanations and spatial grounding of anomalous objects - capabilities absent from prior VAD methods. Ablations confirm that staged training outperforms monolithic optimization, and that structured reasoning acts as an implicit regularizer yielding more balanced predictions than classification-only fine-tuning. Zero-shot transfer to XD-Violence and ShanghaiTech demonstrates cross-domain generalization without target-domain adaptation.