Exploring Multimodal Prompts For Unsupervised Continuous Anomaly Detection

Mingle Zhou; Jiahui Liu; Jin Wan; Gang Li; Min Li

cs.CVMar 23, 2026

Exploring Multimodal Prompts For Unsupervised Continuous Anomaly Detection

Authors: Mingle Zhou, Jiahui Liu, Jin Wan, Gang Li, Min Li

Organizations: Key Laboratory of Computing Power Network and Information Security, Ministry of Education, Shandong Computer Science Center (National Supercomputer Center in Jinan), Qilu University of Technology (Shandong Academy of Sciences), Jinan, China · Shandong Provincial Key Laboratory of Computing Power Internet and Service Computing, Shandong Fundamental Research Center for Computer Science, Jinan, China · Faculty of Data Science, City University of Macau, Macau, China

Abstract

Unsupervised Continuous Anomaly Detection (UCAD) is gaining attention for effectively addressing the catastrophic forgetting and heavy computational burden issues in traditional Unsupervised Anomaly Detection (UAD). However, existing UCAD approaches that rely solely on visual information are insufficient to capture the manifold of normality in complex scenes, thereby impeding further gains in anomaly detection accuracy. To overcome this limitation, we propose an unsupervised continual anomaly detection framework grounded in multimodal prompting. Specifically, we introduce a Continual Multimodal Prompt Memory Bank (CMPMB) that progressively distills and retains prototypical normal patterns from both visual and textual domains across consecutive tasks, yielding a richer representation of normality. Furthermore, we devise a Defect-Semantic-Guided Adaptive Fusion Mechanism (DSG-AFM) that integrates an Adaptive Normalization Module (ANM) with a Dynamic Fusion Strategy (DFS) to jointly enhance detection accuracy and adversarial robustness. Benchmark experiments on MVTec AD and VisA datasets show that our approach achieves state-of-the-art (SOTA) performance on image-level AUROC and pixel-level AUPR metrics.

Figures & tables

Figure 1. Different frameworks in anomaly detection. (A) illustrates the prevalent One-for-One Anomaly Detection paradigm ( Liu et al., 2023 ; Liu et al., 2024b ; Reiss et al., 2021 ) . (B) depicts the unified multiclass model, streamlining the anomaly detection process across multiple classes ( You et al., 2022a ) . (C) showcases the integration of continual learning into anomaly detection ( Liu et al., 2024a ) . (D) represents our proposed multimodal continual anomaly detection model. Panels A–D contrast four anomaly-detection frameworks: one-vs-one, unified multiclass, continual-learning integration, and our proposed multimodal continual approach.

Figure 2. The architecture of our model consists of two core components. The Continuous Multimodal Prompts Memory Block (CMPMB) and the Defect Semantics-Guided Adaptive Fusion Mechanism (DSG-AFM). The CMPMB endows the model with continuous learning ability and facilitates unsupervised anomaly detection in continuous scenarios. The DSG-AFM further enhances anomaly detection accuracy by refining the anomaly map based on defect semantics and a dynamic normalization mechanism. The architecture of our model consists of two core components.

Methods	Bottle	cable	capsule	carpet	grid	hazelnut	leather	metal_nut	pill	screw	tile	toothbrush	transistor	wood	zipper	average	avg FM
CFA	0.309	0.489	0.275	0.834	0.571	0.903	0.935	0.464	0.528	0.528	0.763	0.519	0.320	0.923	0.984	0.623	0.361
CSFlow	0.129	0.420	0.363	0.978	0.602	0.269	0.906	0.220	0.263	0.434	0.697	0.569	0.432	0.802	0.997	0.539	0.426
CutPaste	0.111	0.422	0.373	0.198	0.214	0.578	0.007	0.517	0.371	0.356	0.112	0.158	0.340	0.150	0.775	0.312	0.510
DRAEM	0.793	0.411	0.517	0.537	0.799	0.524	0.480	0.422	0.452	1.000	0.548	0.625	0.307	0.517	0.996	0.595	0.371
FastFlow	0.454	0.512	0.517	0.489	0.482	0.522	0.487	0.476	0.575	0.402	0.489	0.267	0.526	0.616	0.867	0.512	0.279
FAVAE	0.666	0.396	0.357	0.610	0.644	0.884	0.406	0.416	0.531	0.624	0.563	0.503	0.331	0.728	0.544	0.547	0.102

Table 1. Image-level AUROC

\uparrow

and corresponding FM

\downarrow

on MVTec AD dataset ( Bergmann et al., 2019 ) after training on the last subdataset. Note that * signifies the usage of a cache pool for rehearsal during training, which may not be possible in real applications. The best results are highlighted in bold , and suboptimal results are highlighted in underline .

Methods	Bottle	cable	capsule	carpet	grid	hazelnut	leather	metal_nut	pill	screw	tile	toothbrush	transistor	wood	zipper	average	avg FM
CFA	0.068	0.056	0.050	0.271	0.004	0.341	0.393	0.255	0.080	0.015	0.155	0.053	0.056	0.281	0.573	0.177	0.083
DRAEM	0.117	0.019	0.044	0.018	0.005	0.036	0.036	0.142	0.104	0.002	0.130	0.039	0.040	0.033	0.734	0.098	0.116
FastFlow	0.044	0.021	0.013	0.013	0.005	0.028	0.007	0.090	0.029	0.003	0.060	0.015	0.036	0.037	0.264	0.044	0.214
FAVAE	0.086	0.048	0.039	0.015	0.004	0.389	0.389	0.174	0.070	0.017	0.064	0.043	0.046	0.093	0.039	0.083	0.083
PaDiM	0.072	0.037	0.030	0.023	0.006	0.183	0.039	0.155	0.044	0.014	0.065	0.044	0.049	0.080	0.452	0.086	0.366
PatchCore	0.048	0.029	0.035	0.552	0.003	0.338	0.279	0.248	0.051	0.061	0.249	0.034	0.079	0.304	0.595	0.190	0.371

Table 2. Pixel-level AUPR

\uparrow

and corresponding FM

\downarrow

on MVTec AD dataset ( Bergmann et al., 2019 ) after training on the last subdataset.

Figure 3. Visualization and analysis of the model performance on the MVTec-AD and VisA Datasets. Visualization and analysis of the model performance

Methods	candle	capsules	cashew	chewinggum	fryum	macaroni1	macaroni2	pcb1	pcb2	pcb3	pcb4	pipe_fryum	average	avg FM
CFA	0.512	0.672	0.873	0.753	0.304	0.557	0.422	0.698	0.472	0.449	0.407	0.998	0.593	0.327
RD4AD	0.380	0.385	0.737	0.539	0.533	0.607	0.487	0.437	0.672	0.343	0.187	0.999	0.525	0.423
PatchCore	0.401	0.605	0.624	0.907	0.334	0.538	0.437	0.527	0.597	0.507	0.588	0.998	0.589	0.361
SimpleNet	0.504	0.474	0.794	0.721	0.684	0.567	0.447	0.598	0.629	0.538	0.493	0.945	0.616	0.283
UniAD	0.573	0.599	0.661	0.758	0.504	0.559	0.644	0.749	0.523	0.547	0.562	0.989	0.639	0.297
DNE	0.486	0.413	0.735	0.585	0.691	0.584	0.546	0.633	0.693	0.642	0.562	0.747	0.610	0.179

Table 3. Image-level AUROC

\uparrow

and corresponding FM

\downarrow

on VisA dataset ( Zou et al., 2022 ) after training on the last subdataset.

Methods	candle	capsules	cashew	chewinggum	fryum	macaroni1	macaroni2	pcb1	pcb2	pcb3	pcb4	pipe_fryum	average	avg FM
CFA	0.017	0.005	0.059	0.243	0.085	0.001	0.001	0.013	0.006	0.008	0.015	0.592	0.087	0.184
RD4AD	0.002	0.005	0.061	0.045	0.098	0.001	0.001	0.013	0.008	0.008	0.013	0.576	0.069	0.201
PatchCore	0.012	0.007	0.055	0.315	0.082	0.000	0.000	0.008	0.004	0.007	0.010	0.585	0.090	0.311
SimpleNet	0.001	0.004	0.017	0.007	0.047	0.000	0.000	0.013	0.003	0.004	0.009	0.058	0.014	0.016
UniAD	0.006	0.013	0.040	0.185	0.087	0.002	0.002	0.015	0.005	0.015	0.013	0.576	0.080	0.218
PatchCore*	0.018	0.010	0.047	0.202	0.081	0.003	0.001	0.008	0.004	0.008	0.010	0.443	0.070	0.327

Table 4. Pixel-level AUPR

\uparrow

and corresponding FM

\downarrow

on VisA dataset ( Zou et al., 2022 ) after training on the last subdataset.

CMPMB		DSG-AFM		MVtec AD	VisA
Text Prompts	Vision Prompts	DFS	ANM	MVtec AD	VisA
X	X	X	X	0.760/0.249	0.808/0.127
$\checkmark$	$\checkmark$	$\checkmark$	X	0.971/0.523	0.899/0.317
X	$\checkmark$	X	$\checkmark$	0.973/0.597	0.891/0.347
$\checkmark$	$\checkmark$	$\checkmark$	$\checkmark$	0.974/0.604	0.901/0.365

Table 5. Image-level AUROC and Pixel-level AUPR Performance on MVTec AD and VisA dataset with CMPMB and DSG-AFM.

$\alpha$	MVTec AD	VisA
0.1	0.592	0.364
0.3	0.601	0.361
0.5	0.603	0.364
0.7	0.603	0.364
0.9	0.604	0.365

Table 6. Pixel-level AUPR Performance on MVTec AD and VisA dataset with different configurations of

\alpha

Layer	MVTec AD	VisA
1	0.884/0.498	0.815/0.230
3	0.956/0.603	0.881/0.361
5	0.974/0.604	0.901 /0.365
7	0.969/0.578	0.901/0.379
9	0.946/0.481	0.886/0.204

Table 7. Image-level AUROC and Pixel-level AUPR Performance on MVTec AD and VisA dataset with different configurations of ViT encoder layer.

Figure 4. Visualization of anomaly segmentation under different modules on the MVTec AD dataset. Visualization of anomaly segmentation under different modules on the MVTec AD dataset

Explore similar work

RAID: Retrieval-Augmented Anomaly Detection
Feb 23, 2026Mingxiu Cai, Zhe Zhang, Gaochang Wu +2Unsupervised Anomaly Detection Anomaly Detection
State Key Laboratory of Synthetical Automation for Process Industries, Northeastern University · University of Surrey
XMatchAD: A Cross-Modal Matching Perspective on Reconstruction-based Anomaly Detection
Jul 26, 2026Mingxiu Cai, Zhe Zhang, Gaochang Wu +1Industrial Anomaly Detection Anomaly Localization
State Key Laboratory of Synthetical Automation for Process Industries, Northeastern University, Shenyang, China
Towards an Incremental Unified Multimodal Anomaly Detection: Augmenting Multimodal Denoising From an Information Bottleneck Perspective
Mar 3, 2026Kaifang Long, Lianbo Ma, Jiaqi Liu +2Multimodal Anomaly Detection Unified Multimodal Models
Software College, Northeastern University, China · Independent Researcher · CATL, China

Feb 23, 2026J/K move · Enter open · S save

RAID: Retrieval-Augmented Anomaly Detection

Mingxiu Cai, Zhe Zhang, Gaochang Wu, Tianyou Chai+1

State Key Laboratory of Synthetical Automation for Process Industries, Northeastern University · University of Surrey

Unsupervised Anomaly Detection Anomaly Detection Anomaly Localization Few-Shot Anomaly Detection

Open paper AskPDF arXiv

Unsupervised Anomaly Detection (UAD) aims to identify abnormal regions by establishing correspondences between test images and normal templates. Existing methods primarily rely on image reconstruction or template retrieval but face a fundamental challenge: matching between test images and normal templates inevitably introduces noise due to intra-class variations, imperfect correspondences, and limited templates. Observing that Retrieval-Augmented Generation (RAG) leverages retrieved samples directly in the generation process, we reinterpret UAD through this lens and introduce RAID, a retrieval-augmented UAD framework designed for noise-resilient anomaly detection and localization. Unlike standard RAG that enriches context or knowledge, we focus on using retrieved normal samples to guide noise suppression in anomaly map generation. RAID retrieves class-, semantic-, and instance-level representations from a hierarchical vector database, forming a coarse-to-fine pipeline. A matching cost volume correlates the input with retrieved exemplars, followed by a guided Mixture-of-Experts (MoE) network that leverages the retrieved samples to adaptively suppress matching noise and produce fine-grained anomaly maps. RAID achieves state-of-the-art performance across full-shot, few-shot, and multi-dataset settings on MVTec, VisA, MPDD, and BTAD benchmarks. https://github.com/Mingxiu-Cai/RAID.

Abstract

Explore similar work

RAID: Retrieval-Augmented Anomaly Detection

XMatchAD: A Cross-Modal Matching Perspective on Reconstruction-based Anomaly Detection

Towards an Incremental Unified Multimodal Anomaly Detection: Augmenting Multimodal Denoising From an Information Bottleneck Perspective