Multimodal large language models (MLLMs) can enrich industrial anomaly detection with semantic descriptions and anomaly reasoning, but they still lag specialist anomaly detectors in binary detection accuracy. Existing approaches address this gap by fine-tuning MLLMs or training bridging modules to align expert outputs with MLLM inputs, limiting flexibility across backbones. We propose EAGLE, a tuning-free framework that integrates expert anomaly detectors with frozen MLLMs. EAGLE consists of Threshold-Guided Prompt Selection (TGPS), which estimates a decision threshold from expert model statistics and selects textual and visual prompts, and Confidence-Aware Attention Sharpening (CAAS), which shifts MLLM attention toward visual evidence when expert confidence is low. Beyond improving accuracy, we analyze MLLM attention and find that correct anomaly predictions are associated with stronger focus on ground-truth defect regions; EAGLE consistently strengthens this alignment. On MVTec-AD and VisA, EAGLE improves five MLLM backbones without parameter updates, reaching up to 94.4% and 88.1% in anomaly discrimination accuracy, respectively, and achieving performance competitive with fine-tuning-based methods while largely preserving MLLM semantic reasoning ability.
Figures & tables
Figure 1: Attention map visualizations from Qwen2.5-VL-7B on the MVTec-AD and VisA.
Figure 2: Pipeline of EAGLE. Given a query image, an expert model produces an image-level anomaly score simg and an anomaly map. The anomaly score is compared against a threshold τ to select the appropriate prompt. The MLLM incorporates the proposed CAAS module to amplify attention on visual tokens, ultimately generating the final answer.
Figure 3: Normal training images are converted into patch-level features F , which are stored in a memory bank constructed via greedy coreset sampling G . For each training image xi , its unsampled patch set Pi(un) is used to compute the image-level anomaly score through nearest-neighbor search. Finally, the anomaly score distribution of the training set is used to estimate the threshold τ .
Figure 4
Method
Scale
MVTec-AD
VisA
RAD
Accuracy ∗
Precision
Recall
F1
Accuracy ∗
Precision
Recall
F1
Accuracy ∗
Precision
Recall
F1
PatchCore
-
92.1
96.7
96.3
96.4
88.0
96.1
79.7
85.4
94.6
98.6
94.8
96.5
LLaVA-1.5
7B
61.6
79.2
72.3
71.9
58.4
75.0
26.5
32.8
62.1
88.4
31.0
40.3
LLaVA-1.5 w/ours
7B
93.0
95.6
96.6
96.0
88.1
96.1
79.8
85.4
94.6
98.6
94.8
96.5
LLaVA-NeXT
7B
69.8
99.5
40.2
50.9
59.0
80.8
18.8
27.9
83.7
100.0
67.4
78.3
LLaVA-NeXT w/ours
7B
92.1
96.6
95.7
96.1
88.1
95.2
80.5
88.6
89.2
95.6
97.2
96.3
Table 1: Quantitative comparison between MLLMs and MLLMs with EAGLE on the MVTec-AD, VisA, and RAD datasets. Baseline MLLMs follow a 1-shot + template image setting(see Appendix C ). Accuracy ∗ denotes Balanced Accuracy, computed as the average of true positive rate and true negative rate, consistent with the Anomaly Discrimination metric in MMAD Jiang et al. (2024b) .
Method
Anomaly
Defect
Object
Average
Discrimination
Classification
Localization
Description
Analysis
Classification
Analysis
InternVL3
79.1
59.9
63.8
75.7
83.0
82.6
88.4
76.1
EAGLE
90.5
59.5
66.6
73.5
83.5
80.3
87.0
77.3
Table 2: Comparison of performance on the VisA and MVTec datasets within the MMAD benchmark.
Mode
Method
MVTec-AD
VisA
Fine-tuning
AnomalyGPT
86.1 ± 1.1
77.4 ± 1.0
Myriad
87.4 ± 0.9
80.0 ± 0.4
GRPO & Fine-tuning
LR-IAD
84.4 ± 0.0
87.6 ± 0.0
OmniAD
96.0 ± 0.0
86.6 ± 0.0
Tuning-free
EAGLE (LLaVA-1.5)
94.4 ± 0.0
86.9 ± 0.0
EAGLE (Qwen2.5-VL)
94.7 ± 0.0
85.1 ± 0.0
Table 3: Comparison of our method with existing approaches across training modes. Performance is evaluated using standard classification accuracy.
Table 8
Class
Qwen2.5VL-7B
PatchCore
EAGLE w/o CAAS
EAGLE w/ CAAS
Precision
Recall
Accuracy ∗
Precision
Recall
Accuracy ∗
Precision
Recall
Accuracy ∗
Precision
Recall
Accuracy ∗
bottle
96.4
85.7
87.9
98.4
100.0
97.5
95.5
100.0
99.2
95.5
100.0
99.2
cable
97.5
84.8
90.7
98.9
93.5
95.9
96.1
90.2
92.9
96.2
93.9
94.7
capsule
96.0
89.0
85.8
97.1
92.7
89.8
96.4
92.0
92.6
96.4
92.0
92.6
hazelnut
88.5
98.6
88.0
100.0
100.0
100.0
100.0
98.6
99.3
100.0
100.0
100.0
metal_nut
98.4
87.1
91.3
97.2
98.6
94.7
97.2
98.6
94.7
97.2
98.6
94.7
Table 6: Per-class anomaly detection performance on MVTec-AD. Highlighted categories represent cases where the expert model exhibits limitations and EAGLE provides consistent improvements.
Figure 6: Case of expert model misprediction.
Figure 7: Attention ratios to ground-truth anomalous regions across Transformer layers. Higher attention ratios indicate stronger focus on anomalies. Shaded regions denote standard deviation.
Figure 8: Attention map visualizations from Qwen2.5-VL-7B.
Appendix figures & tables12 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 9: Attention allocation between visual and textual tokens across different layers of the Qwen2.5VL-7B.
Dataset
Anomaly Detection
Object Classification
Object Analysis
Defect Classification
Defect Localization
Defect Description
Defect Analysis
Average
MVTec-AD
93.45
83.72
92.30
67.31
75.14
81.11
91.18
83.46
VisA
88.52
80.07
82.31
53.40
54.13
67.44
76.74
71.80
MVTec-LOCO
52.57
82.04
80.44
41.52
43.54
65.03
74.56
62.81
GoodsAD
55.36
89.68
77.13
50.93
48.19
58.55
75.25
65.01
Average
72.48
83.88
83.05
53.29
55.25
68.03
79.43
70.77
Appendix
Table 7: Comparison of average performance across tasks on MMAD.
Dataset
Class
PatchCore
Eagle
Qwen2.5VL-7B
Precision
Recall
F1
Accuracy
Precision
Recall
F1
Accuracy
Precision
Recall
F1
Accuracy
MVTec-AD
bottle
98.44
100.00
99.21
97.50
98.44
100.00
99.21
97.50
96.43
85.71
90.76
87.86
cable
100.00
92.39
96.05
96.20
100.00
91.30
95.45
95.65
97.50
84.78
90.70
90.67
capsule
97.22
96.33
96.77
91.64
98.13
96.33
97.22
93.82
96.04
88.99
92.38
85.80
carpet
90.63
97.75
94.05
82.80
94.44
95.51
94.97
88.82
98.77
89.89
94.12
93.16
grid
98.25
98.25
98.25
96.49
100.00
98.25
99.12
99.12
100.00
91.23
95.41
95.61
Appendix
Table 8: Per-class performance across six datasets. The decision threshold is estimated via the k -sigma method within TGPS, demonstrating consistent results with Tab. 6 .
Setting
MVTec-AD
VisA
Accuracy
F1
Accuracy
F1
K-sigma
93.1
95.6
88.3
85.7
EVT
92.9
95.5
87.8
85.6
Appendix
Table 9: Performance comparison under different thresholding strategies on InternVL3-8B.
Table 17
Figure 10: Ablation study of scaling factor a on the MVTec.
Figure 19
Figure 13: Attention ratios to ground-truth anomalous regions across Transformer layers.
A single multimodal large language model (MLLM) struggles to excel simultaneously at detection, localization, description, and reasoning in multimodal industrial anomaly understanding (MM-IAU). We show that in-domain training does not close this gap. On MMAD, a widely adopted MM-IAU benchmark, trained specialists reach at most 75.5% accuracy in defect localization, against 92.3% for human experts, and even detect anomalies less accurately than their untrained base model. Meanwhile, different MLLMs offer complementary strengths but share this weakness in fine-grained perception, so combining them alone cannot remove it. We therefore propose SiGMA, a spatially grounded multi-agent framework that divides labor between heterogeneous MLLM agents and a dedicated visual defect expert. A multimodal searcher supplies industrial knowledge and normal references, the defect expert turns query-reference comparison into calibrated anomaly evidence, and a label-free reliability controller weighs each source by task-wise competence and query-level evidence quality. SiGMA reaches 85.2% average accuracy on MMAD, 4.0% above the strongest trained specialist and Gemini-2.5-Pro and within 1.5% of human experts. Even with three agents of at most 9B parameters, it reaches 84.4%, and new MLLMs join without retraining.
Xingwu Zhang, Duanyang Du, Huiling Zhu +5
Hunan University · University of Aberdeen · South China Normal University +1
In the progress of industrial anomaly detection, general anomaly detection (GAD) is an emerging trend and also the ultimate goal. Unlike the conventional single- and multi-class AD, general AD aims to train a general AD model that can directly detect anomalies in diverse novel classes without any retraining or fine-tuning on the target data. Recently, Multimodal Large Language Models (MLLMs) have shown great promise in achieving general anomaly detection due to their revolutionary visual understanding and language reasoning capabilities. However, MLLM's general AD ability remains underexplored due to: (1) MLLMs are pretrained on amounts of data sourced from the Web, these data still have significant gaps with the data in AD scenarios. Moreover, the image-text pairs during pretraining are also not specifically for AD tasks. (2) The current mainstream AD datasets are image-based and not yet suitable for post-training MLLMs. To facilitate MLLM-based general AD research, we present MMR-AD, which is a comprehensive benchmark for both training and evaluating MLLM-based AD models. With MMR-AD, we reveal that the AD performance of current SOTA generalist MLLMs still falls far behind the industrial requirements. Based on MMR-AD, we also propose a baseline model, Anomaly-R1, which is a reasoning-based AD model that learns from the CoT data in MMR-AD and is further enhanced by reinforcement learning. Extensive experiments show that our Anomaly-R1 achieves remarkable improvements over generalist MLLMs in both anomaly detection and localization.
Xincheng Yao, Zefeng Qian, Chao Shi +2
School of Information Science and Electronic Engineering, Shanghai Jiao Tong University · MoE Key Lab of Artificial Intelligence, AI Institute, Shanghai Jiao Tong University
Multimodal Large Language Models (MLLMs) have achieved impressive success in natural visual understanding, yet they consistently underperform in industrial anomaly detection (IAD). This is because MLLMs trained mostly on general web data differ significantly from industrial images. Moreover, they encode each image independently and can only compare images in the language space, making them insensitive to subtle visual differences that are key to IAD. To tackle these issues, we present AD-Copilot, an interactive MLLM specialized for IAD via visual in-context comparison. We first design a novel data curation pipeline to mine inspection knowledge from sparsely labeled industrial images and generate precise samples for captioning, VQA, and defect localization, yielding a large-scale multimodal dataset Chat-AD rich in semantic signals for IAD. On this foundation, AD-Copilot incorporates a novel Comparison Encoder that employs cross-attention between paired image features to enhance multi-image fine-grained perception, and is trained with a multi-stage strategy that incorporates domain knowledge and gradually enhances IAD skills. In addition, we introduce MMAD-BBox, an extended benchmark for anomaly localization with bounding-box-based evaluation. The experiments show that AD-Copilot achieves 82.3% accuracy on the MMAD benchmark, outperforming all other models without any data leakage. In the MMAD-BBox test, it achieves a maximum improvement of 3.35× over the baseline. AD-Copilot also exhibits excellent generalization of its performance gains across other specialized and general-purpose benchmarks. Remarkably, AD-Copilot surpasses human expert-level performance on several IAD tasks, demonstrating its potential as a reliable assistant for real-world industrial inspection. All datasets and models will be released for the broader benefit of the community.
Xi Jiang, Yue Guo, Jian Li +7
Department of Computer Science and Engineering, Southern University of Science and Technology, Shenzhen, China · Macau University of Science and Technology, Macau, China · Tencent YouTu Lab, Shenzhen and Shanghai, China +3