Industrial Anomaly Detection via Defect-Grounded Reasoning in Visual Latent Space

Jaron Yeh; Yen-Wei Chang; Jiang Liu; Shao-Yuan Lo

cs.CVSep 24, 2026

Industrial Anomaly Detection via Defect-Grounded Reasoning in Visual Latent Space

Authors: Jaron Yeh, Yen-Wei Chang, Jiang Liu, Shao-Yuan Lo

Organizations: National Taiwan University · AMD GenAI

Abstract

Industrial anomaly detection (IAD) is evolving beyond conventional detection and localization toward multimodal inspection systems that can describe, explain, and reason about fine-grained defects. Although recent multimodal large language model (MLLM)-based methods improve anomaly understanding through textual reasoning and visual guidance, they face two limitations in fine-grained inspection. First, their visual refinement often requires iteratively revisiting local image regions or augmenting with additional tools. Second, the resulting local defect evidence may not be reliably preserved throughout subsequent reasoning. To address these, we propose Anomaly-LR, a defect-grounded latent reasoning framework that first forms a global understanding of the input and then progressively refines anomaly-relevant representations directly in the visual latent space. We further construct IAD-LR-22K, the first IAD instruction dataset designed for latent reasoning, containing 22,228 image-question instances from 4,523 industrial images, with global textual reasoning traces and region-level visual annotations. Extensive experiments show that Anomaly-LR achieves state-of-the-art performance among comparable-scale methods across multiple IAD benchmarks, without requiring external references or tools. The code and data will be released at https://github.com/Yen666/Anomaly-LR.

Figures & tables

Figure 1: Overview of the proposed Anomaly-LR built on an MLLM backbone. Its defect-grounded reasoning process consists of three phases: global textual reasoning, $N$ steps of local latent visual reasoning delimited by the <lvr_start> and <lvr_end> tokens, and final answer generation. At training time, the latent positions are supervised to reconstruct the visual embeddings of the annotated defect region. At inference time, the model feeds its own hidden states back for $K$ steps.

				Anomaly	Defect				Object
Type	Method	Scale	Aux.	Disc.	Cls.	Loc.	Desc.	Anal.	Cls.	Anal.	Average
Human	Human (expert)	–	–	95.04	75.00	92.31	83.33	94.20	86.11	80.37	86.65
reference	Human (ordinary)	–	–	86.90	66.25	85.58	71.25	81.52	89.58	69.72	78.69
Generic	GPT-4o	–	–	68.63	65.80	55.62	73.21	83.41	94.98	82.80	74.92
MLLMs	Qwen2.5-VL	72B	–	72.66	62.31	67.16	73.56	81.95	94.30	86.78	76.96
	AnomalyGPT (AAAI’24) [ 8 ]	7B	$\times$	65.57	27.49	27.97	36.86	32.11	29.84	35.82	36.52

Table 1: Per-subtask accuracy on the MMAD dataset [ 15 ] .

\ddagger

denotes methods that adopt the same protocol of the 20%/80% MMAD train/test split as ours. All baseline numbers are taken from their original papers. Several methods use reference retrieval and external tool use, whereas Anomaly-LR requires neither. “Aux.”: use auxiliary reference images, domain information, or external tools; “Disc.”: Discrimination; “Cls.”: classification; “Loc.”: localization; “Desc.”: description; “Anal.”: analysis. Bold denotes the best value, and underline denotes the second-best value in each column, excluding the human reference.

Method	Scale	MVTec	MPDD	VisA	DAGM	DTD	SDD	Average
GPT-4o [ 12 ]	–	69.6	60.3	63.5	63.0	69.9	65.7	65.3
Claude Sonnet 4 [ 1 ]	–	67.6	65.9	63.5	69.2	88.4	81.7	72.7
Qwen2.5-VL [ 3 ]	3B	62.6	52.9	58.4	54.2	64.4	50.3	57.1
Qwen2.5-VL [ 3 ]	7B	66.0	56.0	58.4	57.7	59.2	67.4	60.8
AnomalyGPT (AAAI’24) [ 8 ]	7B	46.6	54.2	57.3	49.6	64.1	49.5	53.6
Anomaly-OV (CVPR’25) [ 26 ]	7B	74.3	70.3	74.3	77.5	90.7	88.7	78.9

Table 2: OOD evaluation on six IAD benchmarks: MVTec [ 4 ] , MPDD [ 13 ] , VisA [ 29 ] , DAGM [ 25 ] , DTD [ 2 ] , and SDD [ 23 ] . Results are reported in terms of balanced accuracy, following IAD-R1 [ 18 ] . Bold denotes the best value and underline denotes the second-best value.

(a) Architecture design
Configuration	3B	7B
Qwen2.5-VL backbone [ 3 ]	65.25	71.07
Phase 1 + Phase 3	85.49	87.03
Phase 2 + Phase 3	83.37	84.01
Phase 1 + Phase 2 + Phase 3 (ours)	86.00	87.12
(b) Latent reconstruction loss ( ${\cal L}_{\text{recon}}$ )

Table 3: Ablation results on the MMAD dataset [ 15 ] . All models are trained using the same protocol.

Figure 2: Attention maps of the models on a VisA [ 29 ] test image (candle). The green circle denotes the ground-truth defect region. Compared to the Qwen2.5-VL-3B [ 3 ] backbone, Anomaly-LR-3B focuses much more strongly on the defect region.

Explore similar work

JUDO: A Juxtaposed Domain-Oriented Multimodal Reasoner for Industrial Anomaly QA
May 19, 2026Hyunju Kang, Woohyun Lee, Jaewon Kim +1Multimodal Reasoning Visual Reasoning
Sungkyunkwan University · Seoul National University
InspectorGPT: A Comparative Reasoning Enhanced VLM for Comprehensive Industrial Anomaly Detection
Aug 30, 2026Weifei Chen, Honghao Zhang, Zhiyuan You +1Multimodal Industrial Anomaly Detection Comparison
Shanghai Jiao Tong University · The Chinese University of Hong Kong
Global Logic and Local Search: Dual-Stream Multimodal In-Context Learning for Verifiable Industrial Anomaly Detection
Jul 4, 2026Runzhi Deng, Yundi Hu, Yiming Zhong +5Multimodal Industrial Anomaly Detection Reference-Guided Multimodal In-Context Verification
Nanjing University, Nanjing, Jiangsu, China · China Mobile Zijin Innovation Institute, Nanjing, Jiangsu, China · Southeast University, Nanjing, Jiangsu, China

May 19, 2026, cs.CVJ/K move · Enter open · S save

JUDO: A Juxtaposed Domain-Oriented Multimodal Reasoner for Industrial Anomaly QA

Hyunju Kang, Woohyun Lee, Jaewon Kim, Hogun Park

Sungkyunkwan University · Seoul National University

Multimodal Reasoning Visual Reasoning

Open paper AskPDF arXiv

Industrial anomaly detection has been significantly advanced by Large Multimodal Models (LMMs), enabling diverse human instructions beyond detection, particularly through visually grounded reasoning for better image understanding. However, LMMs lack domain-specific knowledge, which limits their ability to generate accurate responses in complex industrial scenarios. In this work, we present JUDO, Juxtaposed Domain-Oriented Multimodal Reasoner, a framework that efficiently incorporates domain knowledge and context in visual and textual reasoning. Through visual reasoning, our model segments the defect region by juxtaposing query images with normal images as visual domain context, enabling a fine-grained visual comparative inspection. Furthermore, we inject domain knowledge through supervised fine-tuning (SFT) to enhance context understanding and subsequently guide domain reasoning through reinforcement learning (GRPO) with tailored rewards, opting for a domain-oriented reasoning process. Experimental results demonstrate that JUDO achieves superior performance on the MMAD benchmark, surpassing models such as Qwen2.5-VL-7B and GPT-4o. These results highlight the importance of enhancing domain knowledge and context for effective reasoning in anomaly understanding.

Abstract

Explore similar work

JUDO: A Juxtaposed Domain-Oriented Multimodal Reasoner for Industrial Anomaly QA

InspectorGPT: A Comparative Reasoning Enhanced VLM for Comprehensive Industrial Anomaly Detection

Global Logic and Local Search: Dual-Stream Multimodal In-Context Learning for Verifiable Industrial Anomaly Detection