InCTRLv2: Generalist Residual Models for Few-Shot Anomaly Detection and Segmentation

Jiawen Zhu; Mengjia Niu; Guansong Pang

cs.CVApr 6, 2026

InCTRLv2: Generalist Residual Models for Few-Shot Anomaly Detection and Segmentation

Authors: Jiawen Zhu, Mengjia Niu, Guansong Pang

Organizations: School of Computing and Information Systems, Singapore Management University, Singapore. · Dyson School of Design Engineering, Imperial College London, United Kingdom.

Abstract

While recent anomaly detection (AD) methods have made substantial progress in recognizing abnormal patterns within specific domains, most of them are specialist models that are trained on large training samples from a specific target dataset, struggling to generalize to unseen datasets. To address this limitation, the paradigm of Generalist Anomaly Detection (GAD) has emerged in recent years, aiming to learn a single generalist model to detect anomalies across diverse domains without retraining. To this end, this work introduces InCTRLv2, a novel few-shot Generalist Anomaly Detection and Segmentation (GADS) framework that significantly extends our previously proposed GAD model, InCTRL. Building on the idea of learning in-context residuals with few-shot normal examples to detect anomalies as in InCTRL, InCTRLv2 introduces two new, complementary perspectives of anomaly perception under a dual-branch framework. This is accomplished by two novel modules upon InCTRL: i) Discriminative Anomaly Score Learning (DASL) with both normal and abnormal data in the main branch, which learns a semantic-guided abnormality and normality space that supports the classification of query samples from both the abnormality and normality perspectives; and ii) One-class Anomaly Score Learning (OASL) using only the normal data, which learns generalized normality patterns in a semantic space via an auxiliary branch, focusing on detecting anomalies through the lens of normality solely. Both branches are guided by rich visual-text semantic priors encoded by large-scale vision-language models. Together, they offer a dual semantic perspective for AD: one emphasizes normal-abnormal discriminations, while the other emphasizes normality-deviated semantics. Extensive experiments on ten AD datasets demonstrate that InCTRLv2 achieves SotA performance in both anomaly detection and segmentation tasks across various settings.

Figures & tables

Figure 1: Illustration of the Generalist Anomaly Detection and Segmentation (GADS) paradigm for the few-shot AD task. A model is trained on auxiliary datasets and leverages few-shot normal images as in-context sample prompts paired with query samples ( Top ). It can then be directly applied to diverse target datasets from different domains without requiring domain-specific retraining ( Bottom ).

Figure 2: Overview of the training process of InCTRLv2. It extends the single-branch residual learning framework of InCTRL into a dual-branch architecture, consisting of a main branch and an auxiliary branch. The main branch employs a Discriminative Anomaly Score Learning (DASL) module to learn a semantic-guided decision space that jointly models abnormality and normality, enabling anomaly discrimination from both perspectives. In parallel, the auxiliary branch adopts a One-class Anomaly Score Learning (OASL) module, which is trained exclusively on normal samples to capture generalized normality patterns in the semantic space. Together, these two branches complement each other by combining discriminative abnormality modeling with normality-driven guidance.

Figure 3: Detailed illustration of the image-level in-context learning and the multi-layer patch-level in-context learning mechanisms in the DASL module.

Normal Text Prompt Examples
‘a photo of a flawless [c] for visual inspection.’ ‘a cropped photo of a perfect [c].’ ‘a blurry photo of the [c] without defect.’ ‘a dark photo of the unblemished [c].’ ‘a jpeg corrupted photo of a [c] without flaw.’
Abnormal Text Prompt Examples
‘a photo of a [c] with flaw for visual inspection.’ ‘a cropped photo of a [c] with damage.’ ‘a blurry photo of the [c] with defect.’ ‘a dark photo of the [c] with flaw.’ ‘a jpeg corrupted photo of a [c] with defect.’

Table 1: Examples of normal and abnormal text prompts used in InCTRLv2. [c] represents a category-level label, e.g. , cable.

Setup	Domain	Dataset	Non-VLM-based Methods			VLM-based Methods
Setup	Domain	Dataset	SPADE	PaDiM	Patchcore	WinCLIP	ResAD	InCTRL	InCTRL+	InCTRLv2
1-shot	Industrial	VisA	(67.2 ±3.8 , 72.1 ±2.9 )	(61.8 ±0.3 , 67.2 ±1.1 )	(75.8 ±2.8 , 79.9 ±1.9 )	(82.5 ±0.1 , 83.5 ±0.1 )	(79.4 ±3.4 , 81.9 ±1.1 )	(84.8 ±0.1 , 85.2 ±1.6 )	( 85.1 ±0.9 , 86.6 ±0.9 )	( 89.0 ±0.5 , 90.9 ±0.5 )
		MVTecAD	(74.5 ±1.3 , 87.9 ±0.9 )	(76.0 ±0.5 , 87.2 ±0.6 )	(82.8 ±0.8 , 91.7 ±1.0 )	( 92.7 ±0.7 , 95.9 ±0.3 )	(89.3 ±1.4 , 92.7 ±1.0 )	(92.2 ±1.1 , 95.2 ±0.9 )	(90.6 ±0.5 , 95.2 ±0.6 )	( 94.5 ±0.3 , 97.2 ±0.2 )
		ELPV	(50.0 ±1.0 , 61.0± 1.0 )	(54.3 ±4.6 , 65.7 ±1.0 )	(69.5 ±2.6 , 82.2 ±1.9 )	(72.0 ±2.6 , 84.4 ±2.2 )	(80.9 ±2.9 , 77.4 ±5.7 )	( 81.1 ±0.4 , 89.4 ±1.3 )	(80.9 ±1.8 , 88.4 ±3.9 )	( 82.2 ±2.6 , 90.7 ±1.6 )
		SDD	(72.5 ±1.0 , 36.3 ±8.3 )	(71.9 ±1.9 , 34.5 ±1.2 )	(90.2 ±0.6 , 67.1 ±1.0 )	(94.1 ±0.3 , 86.3 ±0.9 )	(91.8 ±1.6 , 79.1 ±1.1 )	(95.9 ±1.1 , 88.5 ±3.8 )	( 96.2 ±0.5 , 88.5 ±2.3 )	( 98.1 ±0.8 , 94.9 ±2.0 )
		AITEX	(70.7 ±1.4 , 44.7 ±4.4 )	(72.4 ±3.8 , 48.0 ±4.4 )	(72.9 ±2.2 , 37.8 ±0.7 )	(72.2 ±3.6 , 48.6 ±5.3 )	( 87.7 ±1.1 , 73.5 ±3.8 )	(76.9 ±3.7 , 48.7 ±1.7 )	(77.8 ±0.5 , 50.4 ±1.1 )	( 78.6 ±0.5 , 52.3 ±0.5 )

Table 2: Image-level (AUROC, AP) results on ten real-world AD datasets under various few-shot AD settings. Best results and the second-best results are respectively highlighted in red and blue .

Setup	Domain	Dataset	Non-VLM-based Methods			VLM-based Methods
Setup	Domain	Dataset	SPADE	PaDiM	Patchcore	WinCLIP	ResAD	InCTRLv2
1-shot	Industrial	VisA	(62.8 ±1.2 , 28.7 ±1.2 )	(87.5 ±0.6 , 60.7 ±1.1 )	( 95.0 ±0.3 , 79.9 ±0.3 )	(94.7 ±0.1 , 80.3 ±0.3 )	(90.1 ±0.3 , 68.9 ±0.4 )	( 95.3 ±0.1 , 87.8 ±0.2 )
		MVTecAD	(61.7 ±1.3 , 24.4 ±0.9 )	(88.8 ±0.2 , 70.3 ±0.3 )	(92.7 ±0.3 , 77.2 ±0.3 )	( 93.7 ±0.1 , 84.2 ±0.2 )	(90.6 ±1.2 , 81.8 ±1.4 )	( 95.2 ±0.1 , 90.4 ±0.2 )
		SDD	(55.1 ±0.3 , 19.9 ±0.2 )	(89.9 ±1.3 , 67.5 ±3.6 )	(95.3 ±0.7 , 82.5 ±1.2 )	( 97.2 ±0.7 , 85.3 ±0.7 )	( 97.2 ±0.0 , 83.9 ±1.3 )	( 97.4 ±0.1 , 94.5 ±0.6 )
		AITEX	(55.7 ±3.8 , 31.8 ±1.1 )	(71.4 ±9.7 , 63.2 ±6.2 )	(68.9 ±1.2 , 31.4 ±19.1 )	(74.1 ±3.6 , 53.2 ±4.5 )	( 88.9 ±1.5 , 78.7 ±2.3 )	( 83.5 ±1.5 , 71.2 ±1.7 )
	Medical	BraTS	(72.4 ±1.4 , 29.7 ±1.2 )	(91.7 ±0.2 , 69.5 ±0.4 )	( 96.4 ±0.9 , 80.8 ±3.5 )	(93.5 ±0.3 , 71.0 ±0.5 )	(92.2 ±1.3 , 68.5 ±4.6 )	( 97.1 ±0.1 , 81.6 ±0.5 )

Table 3: Pixel-level (AUROC, PRO) results on five real-world AD datasets under various few-shot AD settings. Best results and the second-best results are respectively highlighted in red and blue .

$\mathcal{L}_{OASL}$	$\mathcal{L}_{P}$	$\mathcal{L}_{I}$	Industrial				Medical			Semantical
			VisA	SDD	ELPV	AITEX	BrainMRI	HeadCT	BraTS	MINIST		CIFAR-10
			VisA	SDD	ELPV	AITEX	BrainMRI	HeadCT	BraTS	1 vs All	Multi-class	1 vs All	Multi-class
NA	$\times$	$\times$	(89.1, 90.9)	(96.6, 94.0)	(73.2, 84.9)	(70.0, 49.5)	(97.8, 99.6 )	(88.0, 97.1)	(77.6, 96.0)	(83.6, 87.9)	(60.3, 58.4)	(87.8, 93.4)	(88.5, 89.0)
	✓	$\times$	(90.8, 91.2)	(98.4, 96.1 )	(83.9, 91.4)	( 80.8 , 63.0 )	(97.5, 99.6 )	(89.3, 97.2)	(86.1, 97.3)	(87.9, 96.2)	( 68.3 , 65.8)	(94.6, 98.3)	( 95.1 , 93.2)
	$\times$	✓	( 91.2 , 92.5 )	( 98.5 , 95.9)	( 84.0 , 91.9 )	(79.2, 62.8)	(98.2, 99.7 )	( 93.4 , 98.0 )	( 87.9 , 97.7 )	(88.4, 97.3 )	(67.9, 66.5 )	( 95.1 , 98.4 )	(95.0, 93.3 )
	✓	✓	( 92.0 , 93.1 )	( 98.9 , 96.7 )	( 84.6 , 91.6)	( 82.6 , 63.4 )	( 98.4 , 99.7 )	( 93.7 , 98.4 )	( 89.5 , 98.9 )	( 90.4 , 98.3 )	( 70.0 , 67.4 )	( 97.8 , 99.7 )	( 97.1 , 93.9 )

Table 4: Image-level AUROC and AP results for ablation study under four-shot setting. Best results and the second-best results are respectively in red and blue . The results for VisA, and the one-vs-all settings of MNIST and CIFAR-10 represent an average result across their respective data subsets.

$\mathcal{L}_{OASL}$	$\mathcal{L}_{P}$	$\mathcal{L}_{I}$	Industrial			Medical
$\mathcal{L}_{OASL}$	$\mathcal{L}_{P}$	$\mathcal{L}_{I}$	VisA	SDD	AITEX	BraTS
$\times$	$\times$	$\times$	(94.1, 86.8)	(95.8, 84.9)	(81.2, 71.3)	(96.3, 80.7)
	✓	$\times$	(95.3, 89.4)	(97.4, 94.4)	(85.7, 74.4)	(96.8, 81.7)
	$\times$	✓	(94.4, 86.2)	(95.9, 85.8)	(83.4, 71.5)	(96.2, 79.9)
	✓	✓	(95.7, 89.3)	(96.8, 89.0)	(87.6, 76.7)	(96.9, 82.1)
✓	$\times$	$\times$	(94.8, 87.1)	(97.5, 92.5)	(88.0, 76.6)	(97.1, 82.0)

Table 5: Pixel-level AUROC and AP results for ablation study under four-shot setting. Best results and the second-best results are respectively in red and blue . The results for VisA represents an average result across their respective data subsets

Figure 4: Left : Image-level AUROC results based on different value of $\alpha$ . Right : Pixel-level AUROC results based on different value of $\beta$ .

Figure 5: Visualization of anomaly maps generated by DASL module ( $\mathbf{M}_{p}$ ) alone and InCTRLv2 ( $\mathbf{M}$ ). The anomaly score maps are generated under the ‘VisA to MVTecAD’ setting.

Figure 6: Visualization of anomaly maps generated by different GADS methods. The anomaly score maps are generated under the ‘VisA to MVTecAD’ setting.

Methods	Parameters(M)	Infer time(fps)
SPADE	74.5	4.8
PaDiM	686.9	14.1
PatchCore	69.5	21.5
WinCLIP	165.9	0.51
ResAD	442.6	18.8
InCTRL	117.5	0.53

Table 6: Complexity comparison between our InCTRLv2 and other competing methods

Anomaly Detection

Anomaly detection finds rare or unexpected patterns in data, from defective products and system faults to fraud and security threats.

See all 652 papers

Hypergraph-Enhanced Training-Free and Language-Free Few-Shot Anomaly Detection
May 11, 2026Guohuan Xie, Xin He, Dingying Fan +2Few-Shot Anomaly Detection Medical Image Anomaly Detection
Nankai University · Tianjin University of Technology · Tsinghua University
DuoAD: Leveraging [CLS] Dual Characteristics for Training-Free Few-Shot Anomaly Detection
Jul 27, 2026Jyun-Ze Tang, Po-Han Huang, Ming-Ching Chang +2Few-Shot Anomaly Detection Anomaly Localization
Inventec Corporation, Taipei, Taiwan · University at Albany, State University of New York, NY, USA
Beyond Normal References: Discriminative Few-Shot Anomaly Detection
May 22, 2026Huan Wang, Jun Shen, Jun Yan +1Few-Shot Anomaly Detection Few-Shot Learning
Singapore Management University, Singapore · University of Wollongong, Australia

May 11, 2026J/K move · Enter open · S save

Hypergraph-Enhanced Training-Free and Language-Free Few-Shot Anomaly Detection

Guohuan Xie, Xin He, Dingying Fan, Siqi Li+1

Nankai University · Tianjin University of Technology · Tsinghua University

Few-Shot Anomaly Detection Medical Image Anomaly Detection Industrial Anomaly Detection Anomaly Detection

Open paper AskPDF arXiv

Few-shot anomaly detection (FSAD) has made significant strides, yet existing methods still face critical challenges: (i) dependence on task- or dataset-specific training/fine-tuning, (ii) reliance on language supervision or carefully hand-crafted prompts, and (iii) limited robustness across domains. In this paper, we introduce HyperFSAD, a novel FSAD framework that is training-free, language-free, and robust across domains, offering a powerful solution to these challenges. Built upon DINOv3 and a hypergraph-based inference mechanism, our approach performs inference without any task-specific optimization or text prompts, while remaining competitive. Specifically, we replace sensitive nearest-neighbor / top-

n

matching with Sparse Hyper Matching: sparsemax first selects the most relevant support patches, which are then aggregated into a hyperedge as compact normal evidence to suppress background noise and distractors. We further introduce Dual-Branch Image Scoring, which fuses spatial anomaly evidence from the patch-grid anomaly map with global semantic deviation captured by support-aware CLS matching, yielding a robust image-level anomaly score in a strictly visual manner. Notably, all components of HyperFSAD are purely visual, eliminating the need for labor-intensive hand-crafted text prompts. Under the stringent training-free and language-free setting, HyperFSAD achieves state-of-the-art performance across six datasets spanning four industrial datasets (MVTecAD, VisA, MPDD, BTAD) and two medical datasets (RESC, BraTS).

Abstract

Anomaly Detection

Explore similar work

Hypergraph-Enhanced Training-Free and Language-Free Few-Shot Anomaly Detection

DuoAD: Leveraging [CLS] Dual Characteristics for Training-Free Few-Shot Anomaly Detection

Beyond Normal References: Discriminative Few-Shot Anomaly Detection