IAD-Unify: Task-Specific Interfaces for Industrial Anomaly Understanding, Segmentation, and Generation

Haoyu Zheng; Jiang Liu; Jiaqi Zhu; Feifei Shao; Zheqi Lv; Wenqiao Zhang

cs.CVApr 14, 2026

IAD-Unify: Task-Specific Interfaces for Industrial Anomaly Understanding, Segmentation, and Generation

Authors: Haoyu Zheng, Jiang Liu, Jiaqi Zhu, Feifei Shao, Zheqi Lv, Wenqiao Zhang

Organizations: Zhejiang University China

Abstract

Industrial anomaly inspection requires complementary capabilities: explaining an observed defect, localizing its pixels, and synthesizing a controlled edit. We present IAD-Unify, a unified architecture connecting a multimodal language model (MLLM), dense visual expert, and diffusion editor through task-specific token interfaces. A multi-reference DINOv2 pathway forms a dense anomaly field and compresses its 1,369 cells into 81 structured evidence tokens. Qwen3.5 consumes these tokens for grounded answers and, with 32 task tokens, converts them into a semantic residual over the dense mask. A separate 256-query interface resamples Qwen states into Stable Diffusion's complete cross-attention context, while the editor retains its source latent and hard-mask inputs. The task pathways share one Qwen adaptation without forcing every task through the same visual bottleneck; in particular, evidence tokens are excluded from the generation pathway. Staged optimization initializes dense evidence, aligns it with language, calibrates segmentation, and then pretrains and specializes the diffusion editor while preserving earlier capabilities. We also construct Anomaly Evidence over 54,501 deduplicated industrial images. Its quality-controlled Anomaly Evidence Compiler and Industrial Edit-Pair Compiler produce family-disjoint, validated supervision through independent geometry, semantic-grounding, and edit-consistency checks. This design provides one fixed shared parameter set for understanding, segmentation, and localized generation without conflating their inputs, supervision, or outputs. The resulting model reaches 73.02% MMAD Macro $_7$ , the highest listed public segmentation AP average (57.10%) with one reference, and the lowest Controlled masked DINO distance (0.3765), with complementary strengths across reasoning, pixel ranking, and semantic edit fidelity.

Figures & tables

Figure 1: Capability overview of IAD-Unify. Reference-aware understanding reports anomaly status and grounded evidence; language-calibrated segmentation predicts anomalous pixels; and mask-guided generation follows region-specific instructions over a real normal source. The generation examples share the same source while varying the edit instruction and mask.

Figure 2: Anomaly Evidence and its evidence-grounded compilation. Product-level bars summarize defect diversity across 24 products. The two compilers create verified records for understanding and segmentation, and grounded source–instruction–target records for generation.

Figure 3: Overview of IAD-Unify. A shared multimodal language model connects three task pathways while preserving their task-specific inputs and outputs. Understanding and segmentation consume dense anomaly evidence from an observed image and optional normal references. Generation instead uses a source image, a user edit mask, and an instruction; the predicted anomaly mask never serves as an edit command.

Unified evaluation protocol
Method	AD	DefCls	DefLoc	DefDesc	DefAna	ObjCls	ObjAna	Macro 7
Qwen3.5-9B	70.88	56.94	62.51	63.17	74.88	92.31	78.86	71.36
InternVL3.5-8B	67.32	55.23	60.48	61.41	73.16	91.20	77.42	69.46
Ovis-U1-3B	65.51	53.64	58.29	59.52	71.84	90.45	75.83	67.87
Gemma3-4B	57.91	37.42	36.16	46.83	76.88	84.67	74.12	59.14
SPHINX-7B	53.62	34.17	51.78	51.44	71.86	84.69	73.58	60.16

Table 1: MMAD one-shot results with equal-weight averaging across datasets. All methods use one evaluation protocol; bold marks the column best. DefCls, DefLoc, DefDesc, and DefAna denote defect classification, localization, description, and analysis; ObjCls and ObjAna denote object classification and analysis. AD is balanced accuracy; all other columns are accuracy.

Figure 4: Replacing nuisance-matched shuffled representations with sample-aligned counterparts yields near-ordered, saturating gains under fixed final weights and test inputs. At full alignment, pixel AP improves by 1.76 points, while masked DINO distance decreases by 0.0305 and 0.0287 on Controlled and Target Pairs, respectively.

Variant	Dice	IoU	AP	AUROC	N-FPR
Dense baseline	72.52	61.07	72.90	91.46	0.0831
Without aligned tokens	72.52	61.07	72.92	91.50	0.0845
IAD-Unify	72.66	61.23	83.77	95.63	0.0140

Table 2: Internal segmentation on Anomaly Evidence under a fixed 29,588-sample evaluation set and protocol. Token suppression removes the aligned Evidence Tokens. Higher is better except for N-FPR.

Method	MVTec	VisA	MPDD	AP	AUROC	AUPRO
DINOv2-NN-1	47.54	28.73	30.62	35.63	93.18	84.86
AnomalyDINO-16	57.58	42.35	36.38	45.44	96.74	91.42
AnomalyDINO-1	51.55	37.48	32.10	40.38	95.18	87.76
AnomalyCLIP-0	34.42	21.41	27.87	27.90	94.16	86.99
PatchCore	47.83	33.69	37.17	39.56	97.52	87.56
IAD-Unify	81.18	44.94	45.18	57.10	92.55	77.77

Table 3: Public segmentation. Suffixes denote reference count (0: zero-shot); PatchCore uses its normal bank and IAD-Unify uses one reference. The final columns average category-macro results across the three datasets.

Figure 5: Segmentation of contamination, missing-part, and crack anomalies on held-out industrial images.

Figure 6: Matched mask-guided editing of real normal sources.

Controlled Pairs
Method	LPIPS	DINO	PSNR
SD2 Inpaint	0.0205	0.5675	13.99
PowerPaint v2	0.0195	0.5580	14.60
BrushNet	0.0257	0.4431	11.74
IAD-Unify	0.0217	0.3765	11.60
Target Pairs

Table 4: Generation under matched source, mask, instruction, sampling, and compositing. Lower LPIPS/DINO and higher PSNR are better.

Appendix figures & tables9 assets

Supplementary material from the paper’s appendix.

Appendix

Method / supervision	MVTec AP	VisA AP	MPDD AP
DINOv2-NN, frozen 1-shot	47.54	28.73	30.62
AnomalyDINO, 16-shot	57.58	42.35	36.38
AnomalyDINO, 1-shot	51.55	37.48	32.10
AnomalyCLIP, zero-shot	34.42	21.41	27.87
PatchCore, normal bank	47.83	33.69	37.17
Ours , 1-shot	81.18	44.94	45.18

Appendix

Table 5: Dataset-wise pixel AP under the public comparison regimes.

Method	AD	DefCls	DefLoc	DefDesc	DefAna	ObjCls	ObjAna	Macro 7
Hash-pinned local MMAD variant
Ours	73.47	58.72	64.36	64.89	76.60	93.01	80.13	73.02

Appendix

Table 6: MMAD capability breakdown on the hash-pinned 39,670-question local one-shot variant with equal-dataset aggregation.

	Controlled Pairs (100)			Target Pairs (63)
Method	LPIPS	DINO	PSNR	LPIPS	DINO	PSNR
SD2 Inpaint	0.0205	0.5675	13.99	0.0125	0.4231	15.56
PowerPaint v2	0.0195	0.5580	14.60	0.0115	0.3890	16.86
BrushNet	0.0257	0.4431	11.74	0.0201	0.5416	13.01
Ours	0.0217	0.3765	11.60	0.0133	0.3970	14.17

Appendix

Table 7: Generation on the exact frozen holdouts under matched source, mask, instruction, inference steps, and compositing. Lower LPIPS/DINO and higher PSNR are better.

Segmentation variant	Dice	IoU	AP	AUROC	N-FPR
Base	72.52	61.07	72.90	91.46	0.0831
Evidence suppressed	72.52	61.07	72.92	91.50	0.0845
Evidence shuffled	72.17	60.62	80.61	93.88	0.0143
Ours	72.66	61.23	83.77	95.63	0.0140

Appendix

Table 8: Evidence-interface ablation on the fixed internal segmentation test. Higher is better except for N-FPR.

Figure 7: Typed-interface counterfactuals under fixed visual inputs. Top: within each row, the query, normal reference, checkpoint, and threshold are fixed; only the segmentation-token state changes. Red denotes the predicted mask and teal the ground truth. Aligned tokens produce more complete masks in these cases where batch-shuffled tokens miss or fragment the defect, while Base evidence is the pre-token mask. Bottom: within each row, the source, edit region, checkpoint, seed, sampler, and 30-step CFG-2.0 protocol are fixed; only the MetaQuery condition changes. Null MQ under-edits. From top to bottom, the matched instructions request a concave defect with a dark interior, five yellow specks, and a reddish-orange protrusion; the shuffled column receives another controlled-holdout row’s instruction. Synthetic targets are shown only as visual references and are not diffusion inference inputs.

Figure 8: Additional understanding evidence from fixed held-out episodes. Each triplet shows the normal reference, query, and dense evidence region supplied to the language route. The cases span bracket, metal-nut, metal-plate, capsule, and bottle families; scored answers remain part of the complete quantitative test rather than a case-level metric.

Figure 9: Extended aligned segmentation comparison. Columns are input, ground truth, EfficientAD, PatchCore, and IAD-Unify. The five held-out products expose large structural anomalies, compact missing parts, and texture defects; cases are selected for visual coverage, while the main segmentation table uses the complete fixed tests.

Figure 10: Extended aligned generation comparison, part I. Columns show a real public train-good source, user edit mask, paired edit target, G0, and IAD-Unify for leather, cashew, metal-nut, and bracket defects. Every row uses identical conditioning across the two model stages; fallback and target-derived pseudo-normal sources are excluded.

Figure 11: Extended aligned generation comparison, part II, covering bracket, transistor, metal-plate, and tile defects under the same real-source, source-preserving paired-target protocol as Figure 10 .

Explore similar work

Jul 1, 2026cs.CV

GenAU: Language-Grounded Industrial Anomaly Understanding with Vision-Language Models

Industrial inspection requires more than binary anomaly detection: a practical system should determine whether an anomaly exists, localize the defective region, identify the defect type, and provide interpretable visual evidence. Existing CLIP-based methods detect and localize anomalies well but offer limited language-level defect understanding, while instruction-tuned vision-language models can describe defects but do not natively produce pixel-level masks. We introduce GenAU, a Generalist vision-language framework for industrial Anomaly Understanding that unifies image-level detection, pixel-level segmentation, multi-type anomaly detection, and defect analysis in a single instruction-following model. GenAU augments a vision-language model with two segmentation tokens, [SEG_defect] and [SEG_normal], whose hidden states act as language-grounded queries over multi-scale visual features for pixel-level localization; the image-level score fuses this map with the decoder's textual normal/defect decision, while the language decoder produces structured defect-aware responses. Trained with a joint language-modeling and segmentation objective, GenAU covers all four tasks within one architecture and recipe, adding zero-shot multi-type detection and language-grounded defect analysis at a quantified cost to detection and segmentation. Across cross-dataset benchmarks, GenAU attains the strongest image-level detection among CLIP-based zero-shot methods on VisA and Real-IAD, with segmentation approaching but not surpassing specialized CLIP baselines.

Hongkuan Zhou, Tristan Rehm, Nadeem Nazer +3

Corporate Research, Robert Bosch GmbH, Renningen, Germany · Institute for Artificial Intelligence University of Stuttgart, Stuttgart, Germany · Otto-von-Guericke-University Magdeburg, Magdeburg, Germany +1

Industrial Anomaly Detection Vision-Language Grounding Multimodal Anomaly Detection Zero-Shot Anomaly Detection

Ask PDF

Jul 21, 2026cs.CV

MOPDA: Mixed-Trajectory On-Policy Distillation for Language-Guided Industrial Anomaly Detection

Large vision-language models (LVLMs) have shown strong potential for industrial anomaly detection (IAD) by providing image-level anomaly judgments and interpretable reasoning. However, reliably translating generated judgments into precise pixel-level localization remains challenging. We propose \textbf{M}ixed-Trajectory \textbf{O}n-\textbf{P}olicy \textbf{D}istillation for Language-Guided Industrial \textbf{A}nomaly Detection (MOPDA), the first framework to introduce on-policy self-distillation into LVLM-based IAD. For judgment learning, \method introduces \textbf{Mixed-Trajectory Supervision}, combining student-generated on-policy trajectories with evidence-conditioned teacher trajectories under a shared token-level distillation objective. Student trajectories preserve supervision on deployment-relevant response paths, while teacher trajectories provide complementary evidence-conditioned supervision. For dense localization, \textbf{Language-guided Visual Anchoring} uses the final judgment as a compact semantic condition to construct image-specific normal and abnormal anchors, which are contrasted with dense visual features to produce anomaly maps. This keeps language as semantic guidance while grounding pixel-level responses in visual evidence. Under a strict cross-dataset zero-shot protocol on five IAD benchmarks, \method outperforms the evaluated LVLM-based baselines on most detection, localization, and judgment metrics while remaining competitive with CLIP-based methods. Ablations further validate both mixed-trajectory supervision and final-judgment conditioning.

Shuimu Chen, Jing Jin, Nan Su +5

Tsinghua University · Guangdong Laboratory of Artificial Intelligence and Digital Economy (SZ) · Shenzhen University

Industrial Anomaly Detection VLM Distillation Multimodal Anomaly Detection On-Policy Distillation Zero-Shot Anomaly Detection Anomaly Detection

Ask PDF

Aug 8, 2026cs.CV

UniScale: Arbitrary-Scale Industrial Anomaly Generation

Industrial anomaly inspection faces a major challenge due to the lack of real-world anomaly samples. While generative models are used to create anomaly data, existing methods still struggle when handling small-scale anomalies. This failure occurs because extreme downsampling in diffusion models causes the information of small anomalies to be lost in the latent space. To address this, we introduce UniScale, a unified training and inference framework for high-fidelity industrial anomaly generation across arbitrary scales. During training, we introduce an Error-Suppressed Multi-Scale Training (EMT) strategy, which enables the model to learn the rich location-aware textures of anomalies, while suppressing upsampling-induced interpolation errors in texture acquisition, ensuring the model is capable of learning small-scale anomalies, while remaining effective for regular scale anomalies. For inference, we propose Generation-then-Fusion Denoising. It decouples anomaly generation from background integration, preventing small anomalies from being overwhelmed. Extensive experiments demonstrate that our method outperforms state-of-the-art competitors in both anomaly generation quality and downstream detection performance. It achieves a relative IS(a) improvement of 45.86% (from 1.81 to 2.64) on VisA and 37.70% (from 1.22 to 1.68) on MVTec AD 2, while also improving the downstream pixel-level IoU by 4.22% on VisA and AUROC by 6.55% on MVTec AD 2. Code is available at https://github.com/HUST-SLOW/UniScale.

Shilei Zeng, Linxin Guan, Xurui Li +2

School of Electronic Information and Communications, Huazhong University of Science and Technology

Industrial Anomaly Detection Synthetic Anomaly Generation Diffusion Models

Ask PDF