Is In-Domain Training Enough for Fine-Grained Industrial Anomaly Understanding?

Xingwu Zhang; Duanyang Du; Huiling Zhu; Jiayue Dai; Yixiao Liu; Guozhi Liu; Zhihan Zhang; Zijun Long

cs.CVOct 8, 2026

Is In-Domain Training Enough for Fine-Grained Industrial Anomaly Understanding?

Authors: Xingwu Zhang, Duanyang Du, Huiling Zhu, Jiayue Dai, Yixiao Liu, Guozhi Liu, Zhihan Zhang, Zijun Long

Organizations: Hunan University · University of Aberdeen · South China Normal University · South China University of Technology

Abstract

A single multimodal large language model (MLLM) struggles to excel simultaneously at detection, localization, description, and reasoning in multimodal industrial anomaly understanding (MM-IAU). We show that in-domain training does not close this gap. On MMAD, a widely adopted MM-IAU benchmark, trained specialists reach at most 75.5% accuracy in defect localization, against 92.3% for human experts, and even detect anomalies less accurately than their untrained base model. Meanwhile, different MLLMs offer complementary strengths but share this weakness in fine-grained perception, so combining them alone cannot remove it. We therefore propose SiGMA, a spatially grounded multi-agent framework that divides labor between heterogeneous MLLM agents and a dedicated visual defect expert. A multimodal searcher supplies industrial knowledge and normal references, the defect expert turns query-reference comparison into calibrated anomaly evidence, and a label-free reliability controller weighs each source by task-wise competence and query-level evidence quality. SiGMA reaches 85.2% average accuracy on MMAD, 4.0% above the strongest trained specialist and Gemini-2.5-Pro and within 1.5% of human experts. Even with three agents of at most 9B parameters, it reaches 84.4%, and new MLLMs join without retraining.

Figures & tables

Figure 1: Fine-grained defect visualization on MMAD shows that SiGMA concentrates its response on the defect while the baselines show diffuse background activations.

Figure 2: Overview of the search–reason–ground–aggregate workflow in SiGMA.

	Anomaly	Defect				Object
Method	Detection	Classification	Localization	Description	Analysis	Classification	Analysis	Average
Human (Expert)	95.2	75.0	92.3	83.3	94.2	86.1	80.4	86.7
Large-scale MLLMs
InternVL2-76B ( Chen et al., 2024 )	68.3	54.2	56.7	66.3	80.5	86.4	82.9	70.8
GPT-4o ( Hurst et al., 2024 )	68.6	65.8	55.6	73.2	83.4	95.0	82.8	74.9
GPT-5-mini ( Singh et al., 2025 )	64.1	67.4	69.1	79.0	86.7	94.0	83.4	77.7

Table 1: MMAD performance comparison (accuracy, %). In the one-shot setting, each MLLM receives one normal reference image alongside the query. Bold and underlined values mark the best and second-best results.

Configuration	AD	DL	Others	Avg.
(a) Best single (Qwen3-VL-4B)	74.1	63.6	77.1	74.8
(b) $+$ Multimodal searcher	74.3	64.8	82.6	78.9
(c) $+$ Heterogeneous agents (MV)	74.7	65.4	85.8	81.3
(d) $+$ Defect expert (MV)	80.5	73.6	85.8	83.3
(e) $+$ Reliability controller	82.6	79.6	86.8	85.2
(f) SiGMA without defect expert	75.1	65.8	86.7	82.0

Table 2: Component ablation result of SiGMA (accuracy, %). AD, anomaly detection; DL, defect localization; Others, mean of the other five tasks on MMAD; MV, majority vote.

Figure 3: Scaling behavior of SiGMA. (a) Best-subset accuracy by agent count $k$ (Appendix B ). (b) Defect localization by normal-bank size $K$ .

Appendix figures & tables8 assets

Supplementary material from the paper’s appendix.

Appendix

Component	Symbol	Value	Role
Base agents	–	1	Retrieved normal references per agent input
Defect expert	–	$\leq 256$	Size of the category-specific normal bank
Noise-robust calibration	$p_{\mathrm{low}}$	0.99	Quantile of normal residuals defining $\theta_{i,\mathrm{low}}$
	$\kappa$	3	Expansion coefficient defining $\theta_{i,\mathrm{high}}$
	$p_{\mathrm{area}}$	0.99	Quantile of normal area fractions defining $\theta_{i,\mathrm{area}}$
	$\delta$	0.01	Margin of the spatial-coherence threshold

Appendix

Table 3: Hyperparameters of SiGMA, fixed for all reported results.

Readers	Similar reference accuracy (%)	Random reference accuracy (%)
1	84.7	84.7
3	85.2	85.2
4	85.1	85.0
5	85.2	85.1
10	85.2	85.0

Appendix

Table 4: Expert-reader team-size ablation under a fixed nested expansion. Accuracy is measured on the full MMAD Benchmark.

		Anomaly	Defect				Object
	Configuration	Det.	Cls.	Loc.	Desc.	Anal.	Cls.	Anal.	Avg.
(a)	Best single MLLM (Qwen3-VL-4B)	74.1	59.7	63.6	70.1	80.0	92.4	83.4	74.8
(b)	+ Multimodal searcher	74.3	72.0	64.8	78.8	82.1	94.8	85.5	78.9
(c)	+ Heterogeneous agents (majority vote)	74.7	77.6	65.4	82.4	85.6	96.3	87.1	81.3
(d)	+ Defect expert (majority vote)	80.5	77.6	73.6	82.4	85.6	96.3	87.1	83.3
(e)	+ Reliability controller (SiGMA)	82.6	79.2	79.6	83.2	86.4	97.3	88.1	85.2

Appendix

Table 5: Per-task ablation on MMAD (accuracy, %). Rows (a) to (e) add components cumulatively, while rows (f) and (g) ablate expert evidence and base-agent diversity, respectively. Expert readers act only on anomaly detection and defect localization, so row (d) equals row (c) on the other five tasks. Row (g) uses Qwen3-VL-4B and retains the three expert readers.

Use of expert evidence	AD	DL	FPR $\downarrow$
No defect expert	75.1	65.8	17.8
Raw overlay on agent inputs	69.8	75.0	30.6
Raw map through readers	73.5	76.8	24.2
Calibrated map through readers	80.9	79.6	12.8
+ Coherence gate (SiGMA)	82.6	79.6	9.4

Appendix

Table 6: Using expert evidence (accuracy, %). (a) Ways of supplying anomaly maps, with the base agents and controller fixed. FPR is the fraction of normal images judged defective in anomaly-detection questions. (b) Aggregation rules, with all prediction sources fixed. AD and DL denote anomaly detection and defect localization.

Method	Accuracy (%)	In-domain training/ optimizing	Inference and token cost	Latency (s/sample)
Echo	77.3	$\times$	Retrieval and expert-guided MLLM reasoning	1.51
Reason-IAD	79.4	$\checkmark$	10 latent-reasoning iterations	—
JUDO	81.2	$\checkmark$	Single-pass inference with a specialized MLLM	1.23
AgentIAD	82.9	$\checkmark$	Multi-round reasoning with adaptive tool use	2.90
SiGMA	85.2	$\times$	Parallel agent inference and CPU-based reliability aggregation; 25.04–228.44 tokens/question	1.88

Appendix

Table 7: Accuracy and computational cost.

\checkmark

\times

indicate whether in-domain model-parameter training is required. Baseline accuracies follow the original studies ( Chen et al., 2025 ; Chen et al., 2026 ; Kang et al., 2026 ; Miao et al., 2025 ) : Echo reports five tasks on MVTec-AD and VisA, and AgentIAD uses a 6,400-sample MMAD evaluation split. Latencies for Echo, JUDO, and SiGMA, and token usage for SiGMA, are measured in our evaluation. AgentIAD’s latency is reported on 1,600 Brain Tumor MRI samples at batch size one. Evaluation settings differ across rows. A dash denotes an unavailable measurement.

Configuration	MLLM calls	Latency (s)	Peak mem. (GiB)	Avg. acc. (%)
Single 7B MLLM	1	1.2	15.2	81.2
SiGMA, three base agents	3 + up to 3	1.5	85	84.4
SiGMA, eight base agents	8 + up to 3	1.88	105	85.2

Appendix

Table 8: Inference cost per question on 4

\times

NVIDIA L20 GPUs. Peak memory is reported across the four GPUs, not per device. Reader calls occur only for admitted anomaly-detection and defect-localization questions. Latency includes retrieval and the defect expert.

Panel of Figure 4 — Figure 4: Effect of Noise-Robust Calibration on defect-bearing queries. Raw residual maps include query–reference differences beyond the defect; the calibrated maps and final overlays concentrate the evidence supplied to expert readers.

Panel of Figure 5 — Figure 5: Normal queries under raw per-image normalization and normal-bank calibration. The calibrated maps attenuate responses caused by benign query–reference variation before the expert-reader query gate.

Explore similar work

Global Logic and Local Search: Dual-Stream Multimodal In-Context Learning for Verifiable Industrial Anomaly Detection
Jul 4, 2026Runzhi Deng, Yundi Hu, Yiming Zhong +5Multimodal ICL Industrial Anomaly Detection
Nanjing University, Nanjing, Jiangsu, China · China Mobile Zijin Innovation Institute, Nanjing, Jiangsu, China · Southeast University, Nanjing, Jiangsu, China
Industrial Anomaly Detection via Defect-Grounded Reasoning in Visual Latent Space
Sep 24, 2026Jaron Yeh, Yen-Wei Chang, Jiang Liu +1Industrial Anomaly Detection Anomaly Localization
National Taiwan University · AMD GenAI
DifferAD-R1: A Difference-Guided IndustrialAnomaly Localization with Multimodal LargeLanguage Models
Jun 15, 2026Dingrong Wang, Xian Tao, Zhen Qu +5Reinforcement Learning Industrial Anomaly Detection
Institute of Automation, Chinese Academy of Sciences (CAS), Beijing 100190, China · School of Artificial Intelligence, University of Chinese Sciences, Beijing 100049, China · Institute of Automation, CAS, Beijing 100190, China +4

Jul 4, 2026, cs.CVJ/K move · Enter open · S save

Global Logic and Local Search: Dual-Stream Multimodal In-Context Learning for Verifiable Industrial Anomaly Detection

Runzhi Deng, Yundi Hu, Yiming Zhong, Zhao Wang+4

Nanjing University, Nanjing, Jiangsu, China · China Mobile Zijin Innovation Institute, Nanjing, Jiangsu, China · Southeast University, Nanjing, Jiangsu, China

Multimodal ICL Industrial Anomaly Detection Anomaly Localization Multimodal Reasoning Multimodal Anomaly Detection Anomaly Detection

Open paper AskPDF arXiv

Large Multimodal Models (LMMs) show strong few-shot generalization, but industrial anomaly detection remains difficult because defects are small, input resolution is limited, and textual standards are not always grounded in visual evidence. Recent optimization-based methods improve alignment through fine-tuning, but they often require many defective samples, which are unavailable in early deployment. We present Global Logic and Local Search (GLLS), a training-free framework for reference-guided multimodal in-context verification. GLLS uses a Part-Aware Visual-Logical Atlas to organize normal references and structured specifications in the inference context. It combines a Global & Logic Stream, where SAM 3 extracts partially checkable visual facts, with a Fine-Grained & Actions Stream, where MCTS selects local evidence crops under a fixed budget. Experiments on MMAD-QA and additional anomaly detection datasets show consistent gains over matched and general-purpose baselines, while keeping the final diagnostic decision traceable to explicit visual evidence throughout the inspection trace.

Abstract

Explore similar work

Global Logic and Local Search: Dual-Stream Multimodal In-Context Learning for Verifiable Industrial Anomaly Detection

Industrial Anomaly Detection via Defect-Grounded Reasoning in Visual Latent Space

DifferAD-R1: A Difference-Guided IndustrialAnomaly Localization with Multimodal LargeLanguage Models