UniSpector: Towards Universal Open-set Defect Recognition via Spectral-Contrastive Visual Prompting

Geonuk Kim; Minhoi Kim; Kangil Lee; Minsu Kim; Hyeonseong Jeon; Jeonghoon Han; Hyoungjoon Lim; Junho Yim

cs.CVApr 3, 2026

UniSpector: Towards Universal Open-set Defect Recognition via Spectral-Contrastive Visual Prompting

Authors: Geonuk Kim, Minhoi Kim, Kangil Lee, Minsu Kim, Hyeonseong Jeon, Jeonghoon Han, Hyoungjoon Lim, Junho Yim

Organizations: LG Energy Solution

Abstract

Although industrial inspection systems should be capable of recognizing unprecedented defects, most existing approaches operate under a closed-set assumption, which prevents them from detecting novel anomalies. While visual prompting offers a scalable alternative for industrial inspection, existing methods often suffer from prompt embedding collapse due to high intra-class variance and subtle inter-class differences. To resolve this, we propose UniSpector, which shifts the focus from naive prompt-to-region matching to the principled design of a semantically structured and transferable prompt topology. UniSpector employs the Spatial-Spectral Prompt Encoder to extract orientation-invariant, fine-grained representations; these serve as a solid basis for the Contrastive Prompt Encoder to explicitly regularize the prompt space into a semantically organized angular manifold. Additionally, Prompt-guided Query Selection generates adaptive object queries aligned with the prompt. We introduce Inspect Anything, the first benchmark for visual-prompt-based open-set defect localization, where UniSpector significantly outperforms baselines by at least 19.7% and 15.8% in AP50b and AP50m, respectively. These results show that our method enable a scalable, retraining-free inspection paradigm for continuously evolving industrial environments, while offering critical insights into the design of generic visual prompting.

Figures & tables

Figure 1: Comparison of visual inspection paradigms: (a) closed-set detectors fail on novel defect types and require costly retraining, (b) anomaly detectors cannot distinguish between defect classes, and (c) visual prompting enables open-set recognition by aligning unseen defects with exemplar prompts, providing a scalable visual inspection framework.

Figure 2: Examples from the InsA benchmark. Top: Samples from the same defect class showing high intra-class appearance variance. Bottom: Samples from different classes exhibiting similar visual patterns, resulting in low inter-class separability. Such ambiguities highlight the inherent difficulty of defect recognition.

Figure 3: Overview of UniSpector , an open-set defect detection and segmentation framework. The Spatial–Spectral Prompt Encoder extracts orientation-invariant spectral cues fused with spatial features to distinguish visually similar defects. Building on these, Contrastive Prompt Encoding regularizes the prompt embedding space into a structured manifold for robust open-set generalization. A Prompt-guided Query Selection mechanism dynamically selects prompt-aware queries. Refer to Appendix C.1 for the inference phase architecture.

Dataset	Count			Class Split
	Images	Defect Instances	Defect Categories	In-domain		Cross-domain
	Images	Defect Instances	Defect Categories	Seen (Train)	Unseen (Test)	Cross-domain
GC10 [ 24 ]	2,292	3,563	10	7	3	-
MagneticTile [ 11 ]	388	514	5	4	1	-
Real-IAD [ 28 ]	49,237	55,326	111	83	28	-
MVTec [ 3 ]	1,258	1,836	73	53	20	-

Table 1: Overview of the InsA benchmark composition. The in-domain split contains seen sets for training and unseen sets held out for open-set evaluation, while the cross-domain split is held out for cross-domain evaluation. Refer to the Appendix A for the details.

Methods	In-domain								Cross-domain						Overall
	GC10		MagneticTile		Real-IAD		MVTec AD		3CAD		VISION		VisA		Avg
	$\mathrm{AP}^{b}_{50}$	$\mathrm{AP}^{m}_{50}$	$\mathrm{AP}^{b}_{50}$	$\mathrm{AP}^{m}_{50}$	$\mathrm{AP}^{b}_{50}$	$\mathrm{AP}^{m}_{50}$	$\mathrm{AP}^{b}_{50}$	$\mathrm{AP}^{m}_{50}$	$\mathrm{AP}^{b}_{50}$	$\mathrm{AP}^{m}_{50}$	$\mathrm{AP}^{b}_{50}$	$\mathrm{AP}^{m}_{50}$	$\mathrm{AP}^{b}_{50}$	$\mathrm{AP}^{m}_{50}$	$\mathrm{AP}^{b}_{50}$	$\mathrm{AP}^{m}_{50}$
visual grounding
GroundingDINO [ 19 ]	9.6	-	26.7	-	0.3	-	1.4	-	0.0	-	0.0	-	0.0	-	5.4	-
GroundingDINO † [ 19 ]	8.0	-	11.0	-	1.2	-	18.8	-	0.0	-	0.0	-	0.1	-	5.6	-
YOLO-World [ 5 ]	7.0	2.1	30.7	28.6	0.3	0.2	2.7	2.4	0.0	0.0	0.3	0.3	1.2	1.2	6.0	5.0

Table 2: Open-set detection and segmentation performance on the InsA. † Denotes models fine-tuned on in-domain seen sets of InsA.

SSPE	CPE	PQS	Detection						Segmentation
SSPE	CPE	PQS	$\mathrm{AP}^{b}$	$\mathrm{AP}^{b}_{50}$	$\mathrm{AP}^{b}_{75}$	$\mathrm{AP}^{b}_{s}$	$\mathrm{AP}^{b}_{m}$	$\mathrm{AP}^{b}_{l}$	$\mathrm{AP}^{m}$	$\mathrm{AP}^{m}_{50}$	$\mathrm{AP}^{m}_{75}$	$\mathrm{AP}^{m}_{s}$	$\mathrm{AP}^{m}_{m}$	$\mathrm{AP}^{m}_{l}$
			13.6	24.0	14.5	17.9	16.3	24.0	7.7	20.0	5.1	7.2	12.3	29.6
✓			27.9	43.0	31.0	38.3	31.4	39.8	17.7	34.8	16.9	17.2	24.8	33.7
✓	✓		43.8	65.8	48.9	47.4	47.6	44.7	26.0	53.1	22.3	21.0	32.7	36.2
✓	✓	✓	46.3	69.1	51.9	50.3	50.1	47.8	28.9	56.7	24.5	23.3	34.1	38.0

Table 3: Ablation study on individual modules in UniSpector . All experiments were conducted on the Real-IAD test set.

Strategy	GC10		VisA
Strategy	$\mathrm{AP}^{b}_{50}$	$\mathrm{AP}^{m}_{50}$	$\mathrm{AP}^{b}_{50}$	$\mathrm{AP}^{m}_{50}$
Learnable Parameter	34.4	34.2	29.2	24.7
Heuristic Top- $K$	35.6	33.9	30.1	25.4
PQS	38.2	36.9	32.8	27.8

Table 4: Comparison of query selection strategies.

Figure 4: 3D PCA projection of L2-normalized prompt embeddings learned by UniSpector (left) and DINOv (right). Circle markers ( $\circ$ ) denote seen sets and cross markers ( $\times$ ) denote unseen sets.

Figure 5: Intra-class cosine similarity comparison across seen and unseen defect classes on RealIAD.

Figure 6: Effect of the number of prompt samples per defect class.

Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

Dataset	Product	Material	Color Distribution Statistics
Dataset	Product	Material	H	S	V
GC10 [ 24 ]	Steel	Steel	$0.0\pm 0.0$	$0.0\pm 0.0$	$85.7\pm 39.6$
MagneticTile [ 11 ]	Magnetic Tile	Steel	$0.0\pm 0.0$	$0.0\pm 0.0$	$109.4\pm 47.4$
Real-IAD [ 28 ]	PCB, Toy Brick, Transistor, etc.	Plastic, Rubber, Wood, etc.	$33.4\pm 49.5$	$40.6\pm 65.7$	$110.2\pm 105.5$
MVTec AD [ 3 ]	Cable, Hazelnut, Tile, etc.	Glass, Metal, Fabric, etc.	$46.7\pm 55.1$	$46.0\pm 47.8$	$120.5\pm 66.7$
3CAD [ 35 ]	Camera Cover, Tablet PC, etc.	Aluminum, Copper, etc.	$7.2\pm 19.2$	$16.7\pm 49.0$	$78.6\pm 70.5$

Appendix

Table 6: Dataset statistics used in the InsA benchmark, including their product types, materials, and the color distribution statistics (mean

\pm

std) for each HSV channel.

Figure 7: Distribution of Prompt-to-Target Ratios across Defect Classes. It illustrates the prompt-to-target ratio for each defect class (with sample sizes). We compare our adaptive prompt strategy (Blue) against the fixed prompt strategy (Orange). By reducing the number of prompts to one for tail classes (total images $\leq 10$ ), the adaptive strategy prevents an irrational ratio in the tail classes, ensuring a more balanced and fair evaluation across the entire InsA dataset.

Figure 8: UniSpector is capable of recognizing unseen defects via visual prompts. (a) Orange box: user-specified prompt region. (b) Blue box: corresponding ground-truth in the target image. (c) Green boxes: correct predictions by UniSpector , accurately localizing subtle defects. (d) Red boxes: DINOv predictions, showing failure to localize the prompted defect.

Figure 9: Detailed view of the inference phase. Unlike the training phase (see Fig. 3 in the main manuscript), inference is decoupled into “Prompt Embedding Extraction” and “Prompt-guided Prediction” to minimize computational overhead by leveraging cached embeddings.

Figure 10: Robustness against prompt annotation (averaged over 10 trials per IoU). Each perturbed box is constrained to a specific IoU level relative to the ground-truth label.

Dataset	Metric	Clean	$0.5\sigma$	$1.0\sigma$	$1.5\sigma$	$2.0\sigma$	$2.5\sigma$	$3.0\sigma$
VisA	$AP^{b}_{50}$	32.5	33.3	32.0	30.0	27.4	25.8	23.4
	$AP^{m}_{50}$	28.2	28.3	27.6	26.3	23.8	22.3	20.4
MTile	$AP^{b}_{50}$	64.2	64.9	64.1	63.5	62.0	60.1	57.0
	$AP^{m}_{50}$	38.9	39.1	38.8	37.8	36.4	34.3	31.0

Appendix

Table 7: Experimental analysis under Gaussian Blur on the prompt images. We evaluate the impact of varying blur levels (

\sigma

) on the performance.

Method	GC10	MTile	RIAD	MVTe	3CAD	VISN	VisA	Avg
PerSAM	0.0	33.0	0.8	4.8	0.1	0.3	3.0	6.0
PerSAM †	0.1	34.2	1.5	8.7	0.2	1.0	6.0	7.4
VRP-SAM †	0.4	12.5	22.3	18.8	3.2	4.7	12.2	10.6

Appendix

Table 8: Results of in-context learning methods (

AP^{m}_{50}

) on InsA benchmark. † indicates results obtained by fine-tuning on the in-domain seen split of the proposed protocol.

Industrial Anomaly Detection

Industrial anomaly detection finds defective products and abnormal equipment behavior from visual, sensor, audio, and other production data.

See all 261 papers

Unification of Closed-Open Industrial Detection Scenarios: New Large-Scale Benchmarks,Challenges and Baselines
Jun 6, 2026Zekai Zhang, Jinglin Zhang, Qinghui Chen +8Open-Vocabulary Object Detection VLM Adaptation
School of Control Science and Engineering, Shandong University, Jinan 250061, China · Shandong Computer Science Center, Qilu University of Technology, Jinan, China · CEREMADE, University Paris Dauphine, PSL Research University, CNRS, UMR 7534, 75775 Paris, France +4
InspectorGPT: A Comparative Reasoning Enhanced VLM for Comprehensive Industrial Anomaly Detection
Aug 30, 2026Weifei Chen, Honghao Zhang, Zhiyuan You +1Industrial Anomaly Detection Zero-Shot Anomaly Detection
Shanghai Jiao Tong University · The Chinese University of Hong Kong
GenAU: Language-Grounded Industrial Anomaly Understanding with Vision-Language Models
Jul 1, 2026Hongkuan Zhou, Tristan Rehm, Nadeem Nazer +3Industrial Anomaly Detection Vision-Language Grounding
Corporate Research, Robert Bosch GmbH, Renningen, Germany · Institute for Artificial Intelligence University of Stuttgart, Stuttgart, Germany · Otto-von-Guericke-University Magdeburg, Magdeburg, Germany +1

Jun 6, 2026J/K move · Enter open · S save

Unification of Closed-Open Industrial Detection Scenarios: New Large-Scale Benchmarks,Challenges and Baselines

Zekai Zhang, Jinglin Zhang, Qinghui Chen, Gang Li+7

School of Control Science and Engineering, Shandong University, Jinan 250061, China · Shandong Computer Science Center, Qilu University of Technology, Jinan, China · CEREMADE, University Paris Dauphine, PSL Research University, CNRS, UMR 7534, 75775 Paris, France+4

Open-Vocabulary Object Detection VLM Adaptation Industrial Anomaly Detection Visual Prompt Learning

Open paper AskPDF arXiv

Large-scale Visual-Language Models (LVLMs) have achieved remarkable success in natural visual tasks, yet their application to industrial defect detection remains challenging due to two fundamental limitations: (i) the scarcity of large-scale industrial datasets that cover diverse defect categories across multiple domains, and (ii) the reliance on manual prompts (points, boxes, masks) that introduce subjective noise and lack text-visual interaction for fine-grained understanding. To address these challenges, we introduce a Large-Scale Multi-Modal Industrial Open-Closed benchmark (MMIOC-1M) containing over one million samples across

14

super-categories,

29

industrial scenes, and

351

defect subcategories. To our knowledge, MMIOC-1M is the first unified largest benchmark supporting both open-vocabulary and closed-set industrial detection, providing valuable pre-training data for LVLMs in industrial scenarios. Furthermore, we propose a Refined Text-Visual Prompt Network (RTVPNet) that incorporates three key innovations: (1) an expert-assisted domain projection mechanism that enables rapid adaptation of general vision models to industrial domains, (2) an energy-based sparse sampling strategy that automatically generates refined visual prompts without manual intervention, and (3) a bidirectional text-visual interaction module that enhances cross-modal semantic alignment and understanding. Extensive experiments demonstrate that RTVPNet achieves state-of-the-art performance on MMIOC-1M, LVIS, and COCO benchmarks while maintaining computational efficiency. The dataset and code are available at https://github.com/hellozzk/MMIO.

Abstract

Industrial Anomaly Detection

Explore similar work

Unification of Closed-Open Industrial Detection Scenarios: New Large-Scale Benchmarks,Challenges and Baselines

InspectorGPT: A Comparative Reasoning Enhanced VLM for Comprehensive Industrial Anomaly Detection

GenAU: Language-Grounded Industrial Anomaly Understanding with Vision-Language Models