EAGLE: Expert-Augmented Attention Guidance for Tuning-Free Industrial Anomaly Detection in Multimodal Large Language Models

Xiaomeng Peng; Xilang Huang; Seon Han Choi

cs.CVFeb 19, 2026

EAGLE: Expert-Augmented Attention Guidance for Tuning-Free Industrial Anomaly Detection in Multimodal Large Language Models

Authors: Xiaomeng Peng, Xilang Huang, Seon Han Choi

Organizations: Ewha Womans University

Abstract

Multimodal large language models (MLLMs) can enrich industrial anomaly detection with semantic descriptions and anomaly reasoning, but they still lag specialist anomaly detectors in binary detection accuracy. Existing approaches address this gap by fine-tuning MLLMs or training bridging modules to align expert outputs with MLLM inputs, limiting flexibility across backbones. We propose EAGLE, a tuning-free framework that integrates expert anomaly detectors with frozen MLLMs. EAGLE consists of Threshold-Guided Prompt Selection (TGPS), which estimates a decision threshold from expert model statistics and selects textual and visual prompts, and Confidence-Aware Attention Sharpening (CAAS), which shifts MLLM attention toward visual evidence when expert confidence is low. Beyond improving accuracy, we analyze MLLM attention and find that correct anomaly predictions are associated with stronger focus on ground-truth defect regions; EAGLE consistently strengthens this alignment. On MVTec-AD and VisA, EAGLE improves five MLLM backbones without parameter updates, reaching up to 94.4% and 88.1% in anomaly discrimination accuracy, respectively, and achieving performance competitive with fine-tuning-based methods while largely preserving MLLM semantic reasoning ability.

Figures & tables

Figure 1: Attention map visualizations from Qwen2.5-VL-7B on the MVTec-AD and VisA.

Figure 2: Pipeline of EAGLE. Given a query image, an expert model produces an image-level anomaly score $s_{img}$ and an anomaly map. The anomaly score is compared against a threshold $\tau$ to select the appropriate prompt. The MLLM incorporates the proposed CAAS module to amplify attention on visual tokens, ultimately generating the final answer.

Figure 3: Normal training images are converted into patch-level features $\mathcal{F}$ , which are stored in a memory bank constructed via greedy coreset sampling $\mathcal{G}$ . For each training image $x_{i}$ , its unsampled patch set $P_{i}^{(un)}$ is used to compute the image-level anomaly score through nearest-neighbor search. Finally, the anomaly score distribution of the training set is used to estimate the threshold $\tau$ .

Method	Scale	MVTec-AD				VisA				RAD
Method	Scale	Accuracy ∗	Precision	Recall	F1	Accuracy ∗	Precision	Recall	F1	Accuracy ∗	Precision	Recall	F1
PatchCore	-	92.1	96.7	96.3	96.4	88.0	96.1	79.7	85.4	94.6	98.6	94.8	96.5
LLaVA-1.5	7B	61.6	79.2	72.3	71.9	58.4	75.0	26.5	32.8	62.1	88.4	31.0	40.3
LLaVA-1.5 w/ours	7B	93.0	95.6	96.6	96.0	88.1	96.1	79.8	85.4	94.6	98.6	94.8	96.5
LLaVA-NeXT	7B	69.8	99.5	40.2	50.9	59.0	80.8	18.8	27.9	83.7	100.0	67.4	78.3
LLaVA-NeXT w/ours	7B	92.1	96.6	95.7	96.1	88.1	95.2	80.5	88.6	89.2	95.6	97.2	96.3

Table 1: Quantitative comparison between MLLMs and MLLMs with EAGLE on the MVTec-AD, VisA, and RAD datasets. Baseline MLLMs follow a 1-shot + template image setting(see Appendix C ). Accuracy ∗ denotes Balanced Accuracy, computed as the average of true positive rate and true negative rate, consistent with the Anomaly Discrimination metric in MMAD Jiang et al. (2024b) .

Method	Anomaly	Defect				Object		Average
Method	Discrimination	Classification	Localization	Description	Analysis	Classification	Analysis	Average
InternVL3	79.1	59.9	63.8	75.7	83.0	82.6	88.4	76.1
EAGLE	90.5	59.5	66.6	73.5	83.5	80.3	87.0	77.3

Table 2: Comparison of performance on the VisA and MVTec datasets within the MMAD benchmark.

Mode	Method	MVTec-AD	VisA
Fine-tuning	AnomalyGPT	86.1 $\pm$ 1.1	77.4 $\pm$ 1.0
Fine-tuning	Myriad	87.4 $\pm$ 0.9	80.0 $\pm$ 0.4
GRPO & Fine-tuning	LR-IAD	84.4 $\pm$ 0.0	87.6 $\pm$ 0.0
GRPO & Fine-tuning	OmniAD	96.0 $\pm$ 0.0	86.6 $\pm$ 0.0
Tuning-free	EAGLE (LLaVA-1.5)	94.4 $\pm$ 0.0	86.9 $\pm$ 0.0
Tuning-free	EAGLE (Qwen2.5-VL)	94.7 $\pm$ 0.0	85.1 $\pm$ 0.0

Table 3: Comparison of our method with existing approaches across training modes. Performance is evaluated using standard classification accuracy.

Table 8

Class	Qwen2.5VL-7B			PatchCore			EAGLE w/o CAAS			EAGLE w/ CAAS
Class	Precision	Recall	Accuracy ∗	Precision	Recall	Accuracy ∗	Precision	Recall	Accuracy ∗	Precision	Recall	Accuracy ∗
bottle	96.4	85.7	87.9	98.4	100.0	97.5	95.5	100.0	99.2	95.5	100.0	99.2
cable	97.5	84.8	90.7	98.9	93.5	95.9	96.1	90.2	92.9	96.2	93.9	94.7
capsule	96.0	89.0	85.8	97.1	92.7	89.8	96.4	92.0	92.6	96.4	92.0	92.6
hazelnut	88.5	98.6	88.0	100.0	100.0	100.0	100.0	98.6	99.3	100.0	100.0	100.0
metal_nut	98.4	87.1	91.3	97.2	98.6	94.7	97.2	98.6	94.7	97.2	98.6	94.7

Table 6: Per-class anomaly detection performance on MVTec-AD. Highlighted categories represent cases where the expert model exhibits limitations and EAGLE provides consistent improvements.

Figure 6: Case of expert model misprediction.

Panel of Figure 7 — Figure 7: Attention ratios to ground-truth anomalous regions across Transformer layers. Higher attention ratios indicate stronger focus on anomalies. Shaded regions denote standard deviation.

Figure 8: Attention map visualizations from Qwen2.5-VL-7B.

Appendix figures & tables12 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 9: Attention allocation between visual and textual tokens across different layers of the Qwen2.5VL-7B.

Dataset	Anomaly Detection	Object Classification	Object Analysis	Defect Classification	Defect Localization	Defect Description	Defect Analysis	Average
MVTec-AD	93.45	83.72	92.30	67.31	75.14	81.11	91.18	83.46
VisA	88.52	80.07	82.31	53.40	54.13	67.44	76.74	71.80
MVTec-LOCO	52.57	82.04	80.44	41.52	43.54	65.03	74.56	62.81
GoodsAD	55.36	89.68	77.13	50.93	48.19	58.55	75.25	65.01
Average	72.48	83.88	83.05	53.29	55.25	68.03	79.43	70.77

Appendix

Table 7: Comparison of average performance across tasks on MMAD.

Dataset	Class	PatchCore				Eagle				Qwen2.5VL-7B
Dataset	Class	Precision	Recall	F1	Accuracy	Precision	Recall	F1	Accuracy	Precision	Recall	F1	Accuracy
MVTec-AD	bottle	98.44	100.00	99.21	97.50	98.44	100.00	99.21	97.50	96.43	85.71	90.76	87.86
	cable	100.00	92.39	96.05	96.20	100.00	91.30	95.45	95.65	97.50	84.78	90.70	90.67
	capsule	97.22	96.33	96.77	91.64	98.13	96.33	97.22	93.82	96.04	88.99	92.38	85.80
	carpet	90.63	97.75	94.05	82.80	94.44	95.51	94.97	88.82	98.77	89.89	94.12	93.16
	grid	98.25	98.25	98.25	96.49	100.00	98.25	99.12	99.12	100.00	91.23	95.41	95.61

Appendix

Table 8: Per-class performance across six datasets. The decision threshold is estimated via the

k

-sigma method within TGPS, demonstrating consistent results with Tab. 6 .

Setting	MVTec-AD		VisA
Setting	Accuracy	F1	Accuracy	F1
K-sigma	93.1	95.6	88.3	85.7
EVT	92.9	95.5	87.8	85.6

Appendix

Table 9: Performance comparison under different thresholding strategies on InternVL3-8B.

Table 17

Figure 10: Ablation study of scaling factor

a

on the MVTec.

Panel of Figure 13 — Figure 13: Attention ratios to ground-truth anomalous regions across Transformer layers.

Dataset	Mean IoU	[email protected]
MVTec-AD	0.38	3.9
VisA	0.16	5.5

Appendix

Table 12: Localization quality of visual prompts.

Panel of Figure 15 — Figure 15: Anomaly score distributions across datasets.

Explore similar work

Is In-Domain Training Enough for Fine-Grained Industrial Anomaly Understanding?
Oct 8, 2026Xingwu Zhang, Duanyang Du, Huiling Zhu +5Industrial Anomaly Detection Multimodal Anomaly Detection
Hunan University · University of Aberdeen · South China Normal University +1
MMR-AD: A Large-Scale Multimodal Dataset for Benchmarking General Anomaly Detection with Multimodal Large Language Models
Apr 13, 2026Xincheng Yao, Zefeng Qian, Chao Shi +2Anomaly Detection Industrial Anomaly Detection
School of Information Science and Electronic Engineering, Shanghai Jiao Tong University · MoE Key Lab of Artificial Intelligence, AI Institute, Shanghai Jiao Tong University
AD-Copilot: A Vision-Language Assistant for Industrial Anomaly Detection via Visual In-context Comparison
Mar 14, 2026Xi Jiang, Yue Guo, Jian Li +7Industrial Anomaly Detection Industrial Inspection
Department of Computer Science and Engineering, Southern University of Science and Technology, Shenzhen, China · Macau University of Science and Technology, Macau, China · Tencent YouTu Lab, Shenzhen and Shanghai, China +3

Oct 8, 2026J/K move · Enter open · S save

Is In-Domain Training Enough for Fine-Grained Industrial Anomaly Understanding?

Xingwu Zhang, Duanyang Du, Huiling Zhu, Jiayue Dai+4

Hunan University · University of Aberdeen · South China Normal University+1

Industrial Anomaly Detection Multimodal Anomaly Detection Anomaly Localization Multi-Agent LLMs

Open paper AskPDF arXiv

A single multimodal large language model (MLLM) struggles to excel simultaneously at detection, localization, description, and reasoning in multimodal industrial anomaly understanding (MM-IAU). We show that in-domain training does not close this gap. On MMAD, a widely adopted MM-IAU benchmark, trained specialists reach at most 75.5% accuracy in defect localization, against 92.3% for human experts, and even detect anomalies less accurately than their untrained base model. Meanwhile, different MLLMs offer complementary strengths but share this weakness in fine-grained perception, so combining them alone cannot remove it. We therefore propose SiGMA, a spatially grounded multi-agent framework that divides labor between heterogeneous MLLM agents and a dedicated visual defect expert. A multimodal searcher supplies industrial knowledge and normal references, the defect expert turns query-reference comparison into calibrated anomaly evidence, and a label-free reliability controller weighs each source by task-wise competence and query-level evidence quality. SiGMA reaches 85.2% average accuracy on MMAD, 4.0% above the strongest trained specialist and Gemini-2.5-Pro and within 1.5% of human experts. Even with three agents of at most 9B parameters, it reaches 84.4%, and new MLLMs join without retraining.

Abstract

Explore similar work

Is In-Domain Training Enough for Fine-Grained Industrial Anomaly Understanding?

MMR-AD: A Large-Scale Multimodal Dataset for Benchmarking General Anomaly Detection with Multimodal Large Language Models

AD-Copilot: A Vision-Language Assistant for Industrial Anomaly Detection via Visual In-context Comparison