InCTRLv2: Generalist Residual Models for Few-Shot Anomaly Detection and Segmentation

Jiawen Zhu; Mengjia Niu; Guansong Pang

cs.CVApr 6, 2026

InCTRLv2: Generalist Residual Models for Few-Shot Anomaly Detection and Segmentation

Authors: Jiawen Zhu, Mengjia Niu, Guansong Pang

Organizations: School of Computing and Information Systems, Singapore Management University, Singapore. · Dyson School of Design Engineering, Imperial College London, United Kingdom.

Abstract

While recent anomaly detection (AD) methods have made substantial progress in recognizing abnormal patterns within specific domains, most of them are specialist models that are trained on large training samples from a specific target dataset, struggling to generalize to unseen datasets. To address this limitation, the paradigm of Generalist Anomaly Detection (GAD) has emerged in recent years, aiming to learn a single generalist model to detect anomalies across diverse domains without retraining. To this end, this work introduces InCTRLv2, a novel few-shot Generalist Anomaly Detection and Segmentation (GADS) framework that significantly extends our previously proposed GAD model, InCTRL. Building on the idea of learning in-context residuals with few-shot normal examples to detect anomalies as in InCTRL, InCTRLv2 introduces two new, complementary perspectives of anomaly perception under a dual-branch framework. This is accomplished by two novel modules upon InCTRL: i) Discriminative Anomaly Score Learning (DASL) with both normal and abnormal data in the main branch, which learns a semantic-guided abnormality and normality space that supports the classification of query samples from both the abnormality and normality perspectives; and ii) One-class Anomaly Score Learning (OASL) using only the normal data, which learns generalized normality patterns in a semantic space via an auxiliary branch, focusing on detecting anomalies through the lens of normality solely. Both branches are guided by rich visual-text semantic priors encoded by large-scale vision-language models. Together, they offer a dual semantic perspective for AD: one emphasizes normal-abnormal discriminations, while the other emphasizes normality-deviated semantics. Extensive experiments on ten AD datasets demonstrate that InCTRLv2 achieves SotA performance in both anomaly detection and segmentation tasks across various settings.

Figures & tables

Figure 1: Illustration of the Generalist Anomaly Detection and Segmentation (GADS) paradigm for the few-shot AD task. A model is trained on auxiliary datasets and leverages few-shot normal images as in-context sample prompts paired with query samples ( Top ). It can then be directly applied to diverse target datasets from different domains without requiring domain-specific retraining ( Bottom ).

Figure 2: Overview of the training process of InCTRLv2. It extends the single-branch residual learning framework of InCTRL into a dual-branch architecture, consisting of a main branch and an auxiliary branch. The main branch employs a Discriminative Anomaly Score Learning (DASL) module to learn a semantic-guided decision space that jointly models abnormality and normality, enabling anomaly discrimination from both perspectives. In parallel, the auxiliary branch adopts a One-class Anomaly Score Learning (OASL) module, which is trained exclusively on normal samples to capture generalized normality patterns in the semantic space. Together, these two branches complement each other by combining discriminative abnormality modeling with normality-driven guidance.

Figure 3: Detailed illustration of the image-level in-context learning and the multi-layer patch-level in-context learning mechanisms in the DASL module.

Normal Text Prompt Examples
‘a photo of a flawless [c] for visual inspection.’ ‘a cropped photo of a perfect [c].’ ‘a blurry photo of the [c] without defect.’ ‘a dark photo of the unblemished [c].’ ‘a jpeg corrupted photo of a [c] without flaw.’
Abnormal Text Prompt Examples
‘a photo of a [c] with flaw for visual inspection.’ ‘a cropped photo of a [c] with damage.’ ‘a blurry photo of the [c] with defect.’ ‘a dark photo of the [c] with flaw.’ ‘a jpeg corrupted photo of a [c] with defect.’

Table 1: Examples of normal and abnormal text prompts used in InCTRLv2. [c] represents a category-level label, e.g. , cable.

Setup	Domain	Dataset	Non-VLM-based Methods			VLM-based Methods
Setup	Domain	Dataset	SPADE	PaDiM	Patchcore	WinCLIP	ResAD	InCTRL	InCTRL+	InCTRLv2
1-shot	Industrial	VisA	(67.2 ±3.8 , 72.1 ±2.9 )	(61.8 ±0.3 , 67.2 ±1.1 )	(75.8 ±2.8 , 79.9 ±1.9 )	(82.5 ±0.1 , 83.5 ±0.1 )	(79.4 ±3.4 , 81.9 ±1.1 )	(84.8 ±0.1 , 85.2 ±1.6 )	( 85.1 ±0.9 , 86.6 ±0.9 )	( 89.0 ±0.5 , 90.9 ±0.5 )
		MVTecAD	(74.5 ±1.3 , 87.9 ±0.9 )	(76.0 ±0.5 , 87.2 ±0.6 )	(82.8 ±0.8 , 91.7 ±1.0 )	( 92.7 ±0.7 , 95.9 ±0.3 )	(89.3 ±1.4 , 92.7 ±1.0 )	(92.2 ±1.1 , 95.2 ±0.9 )	(90.6 ±0.5 , 95.2 ±0.6 )	( 94.5 ±0.3 , 97.2 ±0.2 )
		ELPV	(50.0 ±1.0 , 61.0± 1.0 )	(54.3 ±4.6 , 65.7 ±1.0 )	(69.5 ±2.6 , 82.2 ±1.9 )	(72.0 ±2.6 , 84.4 ±2.2 )	(80.9 ±2.9 , 77.4 ±5.7 )	( 81.1 ±0.4 , 89.4 ±1.3 )	(80.9 ±1.8 , 88.4 ±3.9 )	( 82.2 ±2.6 , 90.7 ±1.6 )
		SDD	(72.5 ±1.0 , 36.3 ±8.3 )	(71.9 ±1.9 , 34.5 ±1.2 )	(90.2 ±0.6 , 67.1 ±1.0 )	(94.1 ±0.3 , 86.3 ±0.9 )	(91.8 ±1.6 , 79.1 ±1.1 )	(95.9 ±1.1 , 88.5 ±3.8 )	( 96.2 ±0.5 , 88.5 ±2.3 )	( 98.1 ±0.8 , 94.9 ±2.0 )
		AITEX	(70.7 ±1.4 , 44.7 ±4.4 )	(72.4 ±3.8 , 48.0 ±4.4 )	(72.9 ±2.2 , 37.8 ±0.7 )	(72.2 ±3.6 , 48.6 ±5.3 )	( 87.7 ±1.1 , 73.5 ±3.8 )	(76.9 ±3.7 , 48.7 ±1.7 )	(77.8 ±0.5 , 50.4 ±1.1 )	( 78.6 ±0.5 , 52.3 ±0.5 )

Table 2: Image-level (AUROC, AP) results on ten real-world AD datasets under various few-shot AD settings. Best results and the second-best results are respectively highlighted in red and blue .

Setup	Domain	Dataset	Non-VLM-based Methods			VLM-based Methods
Setup	Domain	Dataset	SPADE	PaDiM	Patchcore	WinCLIP	ResAD	InCTRLv2
1-shot	Industrial	VisA	(62.8 ±1.2 , 28.7 ±1.2 )	(87.5 ±0.6 , 60.7 ±1.1 )	( 95.0 ±0.3 , 79.9 ±0.3 )	(94.7 ±0.1 , 80.3 ±0.3 )	(90.1 ±0.3 , 68.9 ±0.4 )	( 95.3 ±0.1 , 87.8 ±0.2 )
		MVTecAD	(61.7 ±1.3 , 24.4 ±0.9 )	(88.8 ±0.2 , 70.3 ±0.3 )	(92.7 ±0.3 , 77.2 ±0.3 )	( 93.7 ±0.1 , 84.2 ±0.2 )	(90.6 ±1.2 , 81.8 ±1.4 )	( 95.2 ±0.1 , 90.4 ±0.2 )
		SDD	(55.1 ±0.3 , 19.9 ±0.2 )	(89.9 ±1.3 , 67.5 ±3.6 )	(95.3 ±0.7 , 82.5 ±1.2 )	( 97.2 ±0.7 , 85.3 ±0.7 )	( 97.2 ±0.0 , 83.9 ±1.3 )	( 97.4 ±0.1 , 94.5 ±0.6 )
		AITEX	(55.7 ±3.8 , 31.8 ±1.1 )	(71.4 ±9.7 , 63.2 ±6.2 )	(68.9 ±1.2 , 31.4 ±19.1 )	(74.1 ±3.6 , 53.2 ±4.5 )	( 88.9 ±1.5 , 78.7 ±2.3 )	( 83.5 ±1.5 , 71.2 ±1.7 )
	Medical	BraTS	(72.4 ±1.4 , 29.7 ±1.2 )	(91.7 ±0.2 , 69.5 ±0.4 )	( 96.4 ±0.9 , 80.8 ±3.5 )	(93.5 ±0.3 , 71.0 ±0.5 )	(92.2 ±1.3 , 68.5 ±4.6 )	( 97.1 ±0.1 , 81.6 ±0.5 )

Table 3: Pixel-level (AUROC, PRO) results on five real-world AD datasets under various few-shot AD settings. Best results and the second-best results are respectively highlighted in red and blue .

$\mathcal{L}_{OASL}$	$\mathcal{L}_{P}$	$\mathcal{L}_{I}$	Industrial				Medical			Semantical
			VisA	SDD	ELPV	AITEX	BrainMRI	HeadCT	BraTS	MINIST		CIFAR-10
			VisA	SDD	ELPV	AITEX	BrainMRI	HeadCT	BraTS	1 vs All	Multi-class	1 vs All	Multi-class
NA	$\times$	$\times$	(89.1, 90.9)	(96.6, 94.0)	(73.2, 84.9)	(70.0, 49.5)	(97.8, 99.6 )	(88.0, 97.1)	(77.6, 96.0)	(83.6, 87.9)	(60.3, 58.4)	(87.8, 93.4)	(88.5, 89.0)
	✓	$\times$	(90.8, 91.2)	(98.4, 96.1 )	(83.9, 91.4)	( 80.8 , 63.0 )	(97.5, 99.6 )	(89.3, 97.2)	(86.1, 97.3)	(87.9, 96.2)	( 68.3 , 65.8)	(94.6, 98.3)	( 95.1 , 93.2)
	$\times$	✓	( 91.2 , 92.5 )	( 98.5 , 95.9)	( 84.0 , 91.9 )	(79.2, 62.8)	(98.2, 99.7 )	( 93.4 , 98.0 )	( 87.9 , 97.7 )	(88.4, 97.3 )	(67.9, 66.5 )	( 95.1 , 98.4 )	(95.0, 93.3 )
	✓	✓	( 92.0 , 93.1 )	( 98.9 , 96.7 )	( 84.6 , 91.6)	( 82.6 , 63.4 )	( 98.4 , 99.7 )	( 93.7 , 98.4 )	( 89.5 , 98.9 )	( 90.4 , 98.3 )	( 70.0 , 67.4 )	( 97.8 , 99.7 )	( 97.1 , 93.9 )

Table 4: Image-level AUROC and AP results for ablation study under four-shot setting. Best results and the second-best results are respectively in red and blue . The results for VisA, and the one-vs-all settings of MNIST and CIFAR-10 represent an average result across their respective data subsets.

$\mathcal{L}_{OASL}$	$\mathcal{L}_{P}$	$\mathcal{L}_{I}$	Industrial			Medical
$\mathcal{L}_{OASL}$	$\mathcal{L}_{P}$	$\mathcal{L}_{I}$	VisA	SDD	AITEX	BraTS
$\times$	$\times$	$\times$	(94.1, 86.8)	(95.8, 84.9)	(81.2, 71.3)	(96.3, 80.7)
	✓	$\times$	(95.3, 89.4)	(97.4, 94.4)	(85.7, 74.4)	(96.8, 81.7)
	$\times$	✓	(94.4, 86.2)	(95.9, 85.8)	(83.4, 71.5)	(96.2, 79.9)
	✓	✓	(95.7, 89.3)	(96.8, 89.0)	(87.6, 76.7)	(96.9, 82.1)
✓	$\times$	$\times$	(94.8, 87.1)	(97.5, 92.5)	(88.0, 76.6)	(97.1, 82.0)

Table 5: Pixel-level AUROC and AP results for ablation study under four-shot setting. Best results and the second-best results are respectively in red and blue . The results for VisA represents an average result across their respective data subsets

Figure 4: Left : Image-level AUROC results based on different value of $\alpha$ . Right : Pixel-level AUROC results based on different value of $\beta$ .

Figure 5: Visualization of anomaly maps generated by DASL module ( $\mathbf{M}_{p}$ ) alone and InCTRLv2 ( $\mathbf{M}$ ). The anomaly score maps are generated under the ‘VisA to MVTecAD’ setting.

Figure 6: Visualization of anomaly maps generated by different GADS methods. The anomaly score maps are generated under the ‘VisA to MVTecAD’ setting.

Methods	Parameters(M)	Infer time(fps)
SPADE	74.5	4.8
PaDiM	686.9	14.1
PatchCore	69.5	21.5
WinCLIP	165.9	0.51
ResAD	442.6	18.8
InCTRL	117.5	0.53

Table 6: Complexity comparison between our InCTRLv2 and other competing methods

Anomaly Detection

Anomaly detection finds rare or unexpected patterns in data, from defective products and system faults to fraud and security threats.

See all 652 papers

May 11, 2026

Hypergraph-Enhanced Training-Free and Language-Free Few-Shot Anomaly Detection

Few-shot anomaly detection (FSAD) has made significant strides, yet existing methods still face critical challenges: (i) dependence on task- or dataset-specific training/fine-tuning, (ii) reliance on language supervision or carefully hand-crafted prompts, and (iii) limited robustness across domains. In this paper, we introduce HyperFSAD, a novel FSAD framework that is training-free, language-free, and robust across domains, offering a powerful solution to these challenges. Built upon DINOv3 and a hypergraph-based inference mechanism, our approach performs inference without any task-specific optimization or text prompts, while remaining competitive. Specifically, we replace sensitive nearest-neighbor / top-

n

matching with Sparse Hyper Matching: sparsemax first selects the most relevant support patches, which are then aggregated into a hyperedge as compact normal evidence to suppress background noise and distractors. We further introduce Dual-Branch Image Scoring, which fuses spatial anomaly evidence from the patch-grid anomaly map with global semantic deviation captured by support-aware CLS matching, yielding a robust image-level anomaly score in a strictly visual manner. Notably, all components of HyperFSAD are purely visual, eliminating the need for labor-intensive hand-crafted text prompts. Under the stringent training-free and language-free setting, HyperFSAD achieves state-of-the-art performance across six datasets spanning four industrial datasets (MVTecAD, VisA, MPDD, BTAD) and two medical datasets (RESC, BraTS).

Guohuan Xie, Xin He, Dingying Fan +2

Nankai University · Tianjin University of Technology · Tsinghua University

Few-Shot Anomaly Detection Medical Image Anomaly Detection Industrial Anomaly Detection Anomaly Detection

Ask PDF

Jul 27, 2026

DuoAD: Leveraging [CLS] Dual Characteristics for Training-Free Few-Shot Anomaly Detection

Vision foundation models have enabled strong training-free anomaly detection (AD). However, most existing approaches rely primarily on independent local patch features, leaving the global contextual information encoded by Vision Transformers (ViTs) underexploited. In this work, we identify the dual characteristics of the ViT [CLS] token: its embedding provides anomaly-invariant global semantic representation, while its attention maps implicitly highlight spatially abnormal regions. Building on this observation, we propose a fully automated AD framework leveraging global context to remove manual tunings. Our framework introduces (1) an automatic augmentation selection strategy driven by [CLS]-level semantic consistency, and (2) an attention-guided feature reweighting mechanism that dynamically adjusts patch contributions according to [CLS] attention saliency. By integrating these components over multi-level features, our method achieves stable anomaly scoring and precise localization without training or parameter tuning. Under the one-shot setting, it achieves Image-AUC scores of 97.7%, 93.2%, and 84.5% on MVTec-AD, VisA, and Real-IAD. Using a single fixed configuration across categories, backbones, and datasets, the method establishes a new state-of-the-art for plug-and-play, training-free anomaly detection while maintaining strong robustness and practical scalability.

Jyun-Ze Tang, Po-Han Huang, Ming-Ching Chang +2

Inventec Corporation, Taipei, Taiwan · University at Albany, State University of New York, NY, USA

Few-Shot Anomaly Detection Anomaly Localization Anomaly Detection

Ask PDF

May 22, 2026

Beyond Normal References: Discriminative Few-Shot Anomaly Detection

This paper considers a practical few-shot anomaly detection (FSAD) setting, termed discriminative FSAD, where a limited number of both normal and anomalous examples are available as references during inference. Existing FSAD methods rely on normal-only references through normality matching, ignoring the discriminative clues in anomalous references, while directly fitting both references can overfit to the seen anomalies. We introduce IDEAL, an intrinsic deviation learning framework that leverages both reference types to learn intrinsic deviation patterns characterizing generalizable abnormality as deviations from normality. IDEAL decomposes the learning process into two novel components: 1) a Normal Variation Eraser to suppress nuisance normal variations that may lead to noisy deviations from normality, thereby highlighting anomaly-relevant deviation representations; 2) an Intrinsic Deviation Encoder to decompose these denoised deviation representations into intrinsic deviation vectors capturing the most discriminative orthogonal deviation directions. At inference, IDEAL scores query-to-normal deviations preserved after projection onto the learned intrinsic deviation vectors, enabling generalization for both seen and unseen anomalies. Extensive experiments on eight real-world datasets show that IDEAL generalizes effectively to unseen anomalies and consistently outperforms existing state-of-the-art FSAD methods. Code and data are available at https://github.com/mala-lab/IDEAL.

Huan Wang, Jun Shen, Jun Yan +1

Singapore Management University, Singapore · University of Wollongong, Australia

Few-Shot Anomaly Detection Few-Shot Learning Anomaly Detection

Ask PDF

Abstract

Anomaly Detection

Explore similar work

Hypergraph-Enhanced Training-Free and Language-Free Few-Shot Anomaly Detection

DuoAD: Leveraging [CLS] Dual Characteristics for Training-Free Few-Shot Anomaly Detection

Beyond Normal References: Discriminative Few-Shot Anomaly Detection