IAD-Unify: Task-Specific Interfaces for Industrial Anomaly Understanding, Segmentation, and Generation

Haoyu Zheng; Jiang Liu; Jiaqi Zhu; Feifei Shao; Zheqi Lv; Wenqiao Zhang

cs.CVApr 14, 2026

IAD-Unify: Task-Specific Interfaces for Industrial Anomaly Understanding, Segmentation, and Generation

Authors: Haoyu Zheng, Jiang Liu, Jiaqi Zhu, Feifei Shao, Zheqi Lv, Wenqiao Zhang

Organizations: Zhejiang University China

Abstract

Industrial anomaly inspection requires complementary capabilities: explaining an observed defect, localizing its pixels, and synthesizing a controlled edit. We present IAD-Unify, a unified architecture connecting a multimodal language model (MLLM), dense visual expert, and diffusion editor through task-specific token interfaces. A multi-reference DINOv2 pathway forms a dense anomaly field and compresses its 1,369 cells into 81 structured evidence tokens. Qwen3.5 consumes these tokens for grounded answers and, with 32 task tokens, converts them into a semantic residual over the dense mask. A separate 256-query interface resamples Qwen states into Stable Diffusion's complete cross-attention context, while the editor retains its source latent and hard-mask inputs. The task pathways share one Qwen adaptation without forcing every task through the same visual bottleneck; in particular, evidence tokens are excluded from the generation pathway. Staged optimization initializes dense evidence, aligns it with language, calibrates segmentation, and then pretrains and specializes the diffusion editor while preserving earlier capabilities. We also construct Anomaly Evidence over 54,501 deduplicated industrial images. Its quality-controlled Anomaly Evidence Compiler and Industrial Edit-Pair Compiler produce family-disjoint, validated supervision through independent geometry, semantic-grounding, and edit-consistency checks. This design provides one fixed shared parameter set for understanding, segmentation, and localized generation without conflating their inputs, supervision, or outputs. The resulting model reaches 73.02% MMAD Macro $_7$ , the highest listed public segmentation AP average (57.10%) with one reference, and the lowest Controlled masked DINO distance (0.3765), with complementary strengths across reasoning, pixel ranking, and semantic edit fidelity.

Figures & tables

Figure 1: Capability overview of IAD-Unify. Reference-aware understanding reports anomaly status and grounded evidence; language-calibrated segmentation predicts anomalous pixels; and mask-guided generation follows region-specific instructions over a real normal source. The generation examples share the same source while varying the edit instruction and mask.

Figure 2: Anomaly Evidence and its evidence-grounded compilation. Product-level bars summarize defect diversity across 24 products. The two compilers create verified records for understanding and segmentation, and grounded source–instruction–target records for generation.

Figure 3: Overview of IAD-Unify. A shared multimodal language model connects three task pathways while preserving their task-specific inputs and outputs. Understanding and segmentation consume dense anomaly evidence from an observed image and optional normal references. Generation instead uses a source image, a user edit mask, and an instruction; the predicted anomaly mask never serves as an edit command.

Unified evaluation protocol
Method	AD	DefCls	DefLoc	DefDesc	DefAna	ObjCls	ObjAna	Macro 7
Qwen3.5-9B	70.88	56.94	62.51	63.17	74.88	92.31	78.86	71.36
InternVL3.5-8B	67.32	55.23	60.48	61.41	73.16	91.20	77.42	69.46
Ovis-U1-3B	65.51	53.64	58.29	59.52	71.84	90.45	75.83	67.87
Gemma3-4B	57.91	37.42	36.16	46.83	76.88	84.67	74.12	59.14
SPHINX-7B	53.62	34.17	51.78	51.44	71.86	84.69	73.58	60.16

Table 1: MMAD one-shot results with equal-weight averaging across datasets. All methods use one evaluation protocol; bold marks the column best. DefCls, DefLoc, DefDesc, and DefAna denote defect classification, localization, description, and analysis; ObjCls and ObjAna denote object classification and analysis. AD is balanced accuracy; all other columns are accuracy.

Figure 4: Replacing nuisance-matched shuffled representations with sample-aligned counterparts yields near-ordered, saturating gains under fixed final weights and test inputs. At full alignment, pixel AP improves by 1.76 points, while masked DINO distance decreases by 0.0305 and 0.0287 on Controlled and Target Pairs, respectively.

Variant	Dice	IoU	AP	AUROC	N-FPR
Dense baseline	72.52	61.07	72.90	91.46	0.0831
Without aligned tokens	72.52	61.07	72.92	91.50	0.0845
IAD-Unify	72.66	61.23	83.77	95.63	0.0140

Table 2: Internal segmentation on Anomaly Evidence under a fixed 29,588-sample evaluation set and protocol. Token suppression removes the aligned Evidence Tokens. Higher is better except for N-FPR.

Method	MVTec	VisA	MPDD	AP	AUROC	AUPRO
DINOv2-NN-1	47.54	28.73	30.62	35.63	93.18	84.86
AnomalyDINO-16	57.58	42.35	36.38	45.44	96.74	91.42
AnomalyDINO-1	51.55	37.48	32.10	40.38	95.18	87.76
AnomalyCLIP-0	34.42	21.41	27.87	27.90	94.16	86.99
PatchCore	47.83	33.69	37.17	39.56	97.52	87.56
IAD-Unify	81.18	44.94	45.18	57.10	92.55	77.77

Table 3: Public segmentation. Suffixes denote reference count (0: zero-shot); PatchCore uses its normal bank and IAD-Unify uses one reference. The final columns average category-macro results across the three datasets.

Figure 5: Segmentation of contamination, missing-part, and crack anomalies on held-out industrial images.

Figure 6: Matched mask-guided editing of real normal sources.

Controlled Pairs
Method	LPIPS	DINO	PSNR
SD2 Inpaint	0.0205	0.5675	13.99
PowerPaint v2	0.0195	0.5580	14.60
BrushNet	0.0257	0.4431	11.74
IAD-Unify	0.0217	0.3765	11.60
Target Pairs

Table 4: Generation under matched source, mask, instruction, sampling, and compositing. Lower LPIPS/DINO and higher PSNR are better.

Appendix figures & tables9 assets

Supplementary material from the paper’s appendix.

Appendix

Method / supervision	MVTec AP	VisA AP	MPDD AP
DINOv2-NN, frozen 1-shot	47.54	28.73	30.62
AnomalyDINO, 16-shot	57.58	42.35	36.38
AnomalyDINO, 1-shot	51.55	37.48	32.10
AnomalyCLIP, zero-shot	34.42	21.41	27.87
PatchCore, normal bank	47.83	33.69	37.17
Ours , 1-shot	81.18	44.94	45.18

Appendix

Table 5: Dataset-wise pixel AP under the public comparison regimes.

Method	AD	DefCls	DefLoc	DefDesc	DefAna	ObjCls	ObjAna	Macro 7
Hash-pinned local MMAD variant
Ours	73.47	58.72	64.36	64.89	76.60	93.01	80.13	73.02

Appendix

Table 6: MMAD capability breakdown on the hash-pinned 39,670-question local one-shot variant with equal-dataset aggregation.

	Controlled Pairs (100)			Target Pairs (63)
Method	LPIPS	DINO	PSNR	LPIPS	DINO	PSNR
SD2 Inpaint	0.0205	0.5675	13.99	0.0125	0.4231	15.56
PowerPaint v2	0.0195	0.5580	14.60	0.0115	0.3890	16.86
BrushNet	0.0257	0.4431	11.74	0.0201	0.5416	13.01
Ours	0.0217	0.3765	11.60	0.0133	0.3970	14.17

Appendix

Table 7: Generation on the exact frozen holdouts under matched source, mask, instruction, inference steps, and compositing. Lower LPIPS/DINO and higher PSNR are better.

Segmentation variant	Dice	IoU	AP	AUROC	N-FPR
Base	72.52	61.07	72.90	91.46	0.0831
Evidence suppressed	72.52	61.07	72.92	91.50	0.0845
Evidence shuffled	72.17	60.62	80.61	93.88	0.0143
Ours	72.66	61.23	83.77	95.63	0.0140

Appendix

Table 8: Evidence-interface ablation on the fixed internal segmentation test. Higher is better except for N-FPR.

Figure 7: Typed-interface counterfactuals under fixed visual inputs. Top: within each row, the query, normal reference, checkpoint, and threshold are fixed; only the segmentation-token state changes. Red denotes the predicted mask and teal the ground truth. Aligned tokens produce more complete masks in these cases where batch-shuffled tokens miss or fragment the defect, while Base evidence is the pre-token mask. Bottom: within each row, the source, edit region, checkpoint, seed, sampler, and 30-step CFG-2.0 protocol are fixed; only the MetaQuery condition changes. Null MQ under-edits. From top to bottom, the matched instructions request a concave defect with a dark interior, five yellow specks, and a reddish-orange protrusion; the shuffled column receives another controlled-holdout row’s instruction. Synthetic targets are shown only as visual references and are not diffusion inference inputs.

Figure 8: Additional understanding evidence from fixed held-out episodes. Each triplet shows the normal reference, query, and dense evidence region supplied to the language route. The cases span bracket, metal-nut, metal-plate, capsule, and bottle families; scored answers remain part of the complete quantitative test rather than a case-level metric.

Figure 9: Extended aligned segmentation comparison. Columns are input, ground truth, EfficientAD, PatchCore, and IAD-Unify. The five held-out products expose large structural anomalies, compact missing parts, and texture defects; cases are selected for visual coverage, while the main segmentation table uses the complete fixed tests.

Figure 10: Extended aligned generation comparison, part I. Columns show a real public train-good source, user edit mask, paired edit target, G0, and IAD-Unify for leather, cashew, metal-nut, and bracket defects. Every row uses identical conditioning across the two model stages; fallback and target-derived pseudo-normal sources are excluded.

Figure 11: Extended aligned generation comparison, part II, covering bracket, transistor, metal-plate, and tile defects under the same real-source, source-preserving paired-target protocol as Figure 10 .

Explore similar work

GenAU: Language-Grounded Industrial Anomaly Understanding with Vision-Language Models
Jul 1, 2026Hongkuan Zhou, Tristan Rehm, Nadeem Nazer +3Industrial Anomaly Detection Vision-Language Grounding
Corporate Research, Robert Bosch GmbH, Renningen, Germany · Institute for Artificial Intelligence University of Stuttgart, Stuttgart, Germany · Otto-von-Guericke-University Magdeburg, Magdeburg, Germany +1
MOPDA: Mixed-Trajectory On-Policy Distillation for Language-Guided Industrial Anomaly Detection
Jul 21, 2026Shuimu Chen, Jing Jin, Nan Su +5Industrial Anomaly Detection VLM Distillation
Tsinghua University · Guangdong Laboratory of Artificial Intelligence and Digital Economy (SZ) · Shenzhen University
UniScale: Arbitrary-Scale Industrial Anomaly Generation
Aug 8, 2026Shilei Zeng, Linxin Guan, Xurui Li +2Industrial Anomaly Detection Synthetic Anomaly Generation
School of Electronic Information and Communications, Huazhong University of Science and Technology

Jul 1, 2026, cs.CVJ/K move · Enter open · S save

GenAU: Language-Grounded Industrial Anomaly Understanding with Vision-Language Models

Hongkuan Zhou, Tristan Rehm, Nadeem Nazer, Lavdim Halilaj+2

Corporate Research, Robert Bosch GmbH, Renningen, Germany · Institute for Artificial Intelligence University of Stuttgart, Stuttgart, Germany · Otto-von-Guericke-University Magdeburg, Magdeburg, Germany+1

Industrial Anomaly Detection Vision-Language Grounding Multimodal Anomaly Detection Zero-Shot Anomaly Detection

Open paper AskPDF arXiv

Industrial inspection requires more than binary anomaly detection: a practical system should determine whether an anomaly exists, localize the defective region, identify the defect type, and provide interpretable visual evidence. Existing CLIP-based methods detect and localize anomalies well but offer limited language-level defect understanding, while instruction-tuned vision-language models can describe defects but do not natively produce pixel-level masks. We introduce GenAU, a Generalist vision-language framework for industrial Anomaly Understanding that unifies image-level detection, pixel-level segmentation, multi-type anomaly detection, and defect analysis in a single instruction-following model. GenAU augments a vision-language model with two segmentation tokens, [SEG_defect] and [SEG_normal], whose hidden states act as language-grounded queries over multi-scale visual features for pixel-level localization; the image-level score fuses this map with the decoder's textual normal/defect decision, while the language decoder produces structured defect-aware responses. Trained with a joint language-modeling and segmentation objective, GenAU covers all four tasks within one architecture and recipe, adding zero-shot multi-type detection and language-grounded defect analysis at a quantified cost to detection and segmentation. Across cross-dataset benchmarks, GenAU attains the strongest image-level detection among CLIP-based zero-shot methods on VisA and Real-IAD, with segmentation approaching but not surpassing specialized CLIP baselines.

Abstract

Explore similar work

GenAU: Language-Grounded Industrial Anomaly Understanding with Vision-Language Models

MOPDA: Mixed-Trajectory On-Policy Distillation for Language-Guided Industrial Anomaly Detection

UniScale: Arbitrary-Scale Industrial Anomaly Generation