Multi-Modal Anomaly Detection: A Survey

Xudong Mou; Zexin Wu; Chuan Luo; Shiru Chen; Xudong Liu; Chunming Hu; Renyu Yang

cs.LGAug 24, 2026

Multi-Modal Anomaly Detection: A Survey

Authors: Xudong Mou, Zexin Wu, Chuan Luo, Shiru Chen, Xudong Liu, Chunming Hu, Renyu Yang

Organizations: School of Computer Science and Engineering, Beihang University, Beijing 100191, China · School of Software, Beihang University, Beijing 100191, China · Shandong Inspur Intelligent Production Technology Co., Ltd, Jinan, 250101, China

Abstract

Multi-Modal Anomaly Detection (MMAD) detects rare abnormal events from heterogeneous data sources and is increasingly used in safety- and reliability-critical applications such as industrial inspection and cybersecurity. Yet the literature is fragmented across domains and modality combinations, and existing surveys usually group methods by architecture rather than by how abnormality is defined and separated in multi-modal settings. We survey MMAD from an assumption-driven perspective. We formalize the problem, identify five intrinsic characteristics underlying its core challenges, and organize prior work into two complementary paradigms. The first, normality-assumption methods, models regularity via representation learning, cross-modal alignment, and knowledge enhancement. The second, anomaly-assumption methods, sharpens decision boundaries through coarse-grained, structural, and semantic anomaly injection. We also investigate how foundation models are reshaping MMAD through scalable pretraining, flexible cross-modal transfer, and emerging reasoning capabilities. Finally, we compile representative benchmarks and evaluation protocols across domains and highlight open problems and future directions for robust, adaptive, and interpretable MMAD systems.

Figures & tables

Fig. 1 : Conceptual roadmap of multi-modal anomaly detection. The figure illustrates the progression from the intrinsic nature of anomaly detection to multi-modality-induced challenges, the assumption-driven modeling paradigms, and the emerging future opportunities in MMAD.

Panel of Figure 2 — Fig. 2 : Overview of normality-assumption methods across three groups: a) normal feature learning learns compact normal representations; b) cross-modal representation learning captures inter-modality relations to enrich the normal space; and c) domain knowledge enhancement leverages priors to refine the normal boundary. The right panel shows schematic or qualitative results adapted from Deep SVDD [ 126 ] , WinCLIP [ 64 ] , and VeXKD [ 68 ] .

Panel of Figure 3 — Fig. 3 : Overview of abnormality-assumption methods that inject pseudo anomalies at three granularities: a) coarse-grained injection corrupts low-level statistics via within-modality perturbations or outlier exposure; b) structural injection breaks relational integrity by disturbing links or temporal order within/across modalities; and c) semantic injection synthesizes high-level logical inconsistencies using external knowledge or generative models. The right panel shows illustrative pseudo-anomaly visualizations adapted from CutPaste [ 78 ] , STEAL Net [ 5 ] , and AnomalyDiffusion [ 60 ] .

Method	Modality	Paradigm	FM	Key Mechanism & Intuition	Representative Benchmark	Performance (Metrics)
Image Anomaly Detection (Industrial, Medical & General Vision - Metrics: Image/Pixel-AUROC)
CutPaste [ 78 ]	I	AA (Coarse)	-	Heuristic patch self-augmentation	MVTec AD	95.2% (I-AUROC)
TaylorOE [ 41 ]	I	AA (Coarse)	-	Polynomial Taylor-expansion OE	CIFAR / ImageNet	99.3%+ (AUROC)
M3DM [ 154 ]	I+3D	NA (Alignment)	✓	Point-to-feature semantic alignment	MVTec 3D-AD	94.5% (I-AUROC)
WinCLIP [ 64 ]	I+L	AA (Semantic)	✓	Compositional prompt ensembling	MVTec AD / VisA (Zero-shot)	91.8% / 78.1% (AUROC)
PromptAD [ 85 ]	I+L	AA (Semantic)	✓	Knowledge-guided text descriptions	MVTec AD (Few-shot)	94.6% (I-AUROC) 95.9% (P-AUROC)

TABLE I : Quantitative performance and theoretical taxonomy of representative MMAD methods across diverse domains.

Dataset	Year	Modality	Target / Highlight	Scale / Size	Anomaly Ratio	Annotation Quality	Dynamics
Industrial & Manufacturing Inspection
MVTec AD [ 10 ]	2019	I	Structural defects (scratches, dents)	5,354 images	$\sim$ 23%	Pixel-level mask	Static
MVTec LOCO [ 9 ]	2022	I	Structural + Logical (rule violations)	3,644 images	$\sim$ 16%	Pixel-level mask	Static
MVTec 3D-AD [ 11 ]	2021	I+3D	Geometric shapes (holes, protrusions)	4,147 scans	$\sim$ 23%	Pixel & Voxel-level	Static
Real3D-AD [ 90 ]	2023	High-res 3D	Largest high-precision 360 ∘ dataset	1,250 3D models	$\sim$ 48%	Voxel-level mask	Static
VisA [ 192 ]	2022	I	Complex surfaces, multiple instances	10,821 images	$\sim$ 11%	Pixel-level mask	Static

TABLE II : Summary of multi-modal and unimodal anomaly detection datasets across diverse application domains.

Explore similar work

Out of Context: Reliability in Multimodal Anomaly Detection Requires Contextual Inference
Apr 14, 2026Kevin Wilkinghoff, Neelu Madan, Juan Miguel Valverde +6Anomaly Detection Multimodal Anomaly Detection
Aalborg University, Denmark · Pioneer Centre for Artificial Intelligence, Denmark · Technical University of Denmark, Denmark +3
MMR-AD: A Large-Scale Multimodal Dataset for Benchmarking General Anomaly Detection with Multimodal Large Language Models
Apr 13, 2026Xincheng Yao, Zefeng Qian, Chao Shi +2Anomaly Detection Industrial Anomaly Detection
School of Information Science and Electronic Engineering, Shanghai Jiao Tong University · MoE Key Lab of Artificial Intelligence, AI Institute, Shanghai Jiao Tong University
Understanding and Overcoming Cross-modal Fusion Bias in Multimodal Anomaly Detection From A Fisher Information Perspective
Aug 2, 2026Kaifang Long, Lianbo Ma, Liming Liu +1Representational Similarity Analysis Multimodal Anomaly Detection
Northeastern University Shenyang, China · CATL Ningde, China

Apr 14, 2026J/K move · Enter open · S save

Out of Context: Reliability in Multimodal Anomaly Detection Requires Contextual Inference

Kevin Wilkinghoff, Neelu Madan, Juan Miguel Valverde, Kamal Nasrollahi+5

Aalborg University, Denmark · Pioneer Centre for Artificial Intelligence, Denmark · Technical University of Denmark, Denmark+3

Anomaly Detection Multimodal Anomaly Detection Unsupervised Anomaly Detection

Open paper AskPDF arXiv

Anomaly detection aims to identify observations that deviate from expected behavior. Because anomalous events are inherently sparse, most frameworks are trained exclusively on normal data to learn a single reference model of normality. This implicitly assumes that normal behavior can be captured by a single, unconditional reference distribution. In practice, however, anomalies are often context-dependent: A specific observation may be normal under one operating condition, yet anomalous under another. As machine learning systems are deployed in dynamic and heterogeneous environments, these fixed-context assumptions introduce structural ambiguity, i.e., the inability to distinguish contextual variation from genuine abnormality under marginal modeling, leading to unstable performance and unreliable anomaly assessments. While modern sensing systems frequently collect multimodal data capturing complementary aspects of both system behavior and operating conditions, existing methods treat all data streams equally, without distinguishing contextual information from anomaly-relevant signals. As a result, abnormality is often evaluated without explicitly conditioning on operating conditions. We argue that multimodal anomaly detection should be reframed as a cross-modal contextual inference problem, in which modalities play asymmetric roles, separating context from observation, to define abnormality conditionally rather than relative to a single global reference. This perspective has implications for model design, evaluation protocols, and benchmark construction, and outline open research challenges toward robust, context-aware multimodal anomaly detection.

Abstract

Explore similar work

Out of Context: Reliability in Multimodal Anomaly Detection Requires Contextual Inference

MMR-AD: A Large-Scale Multimodal Dataset for Benchmarking General Anomaly Detection with Multimodal Large Language Models

Understanding and Overcoming Cross-modal Fusion Bias in Multimodal Anomaly Detection From A Fisher Information Perspective