SemiFA: An Agentic Multi-Modal Framework for Autonomous Semiconductor Failure Analysis Report Generation

Shivam Chand Kaushik

cs.CVApr 14, 2026

SemiFA: An Agentic Multi-Modal Framework for Autonomous Semiconductor Failure Analysis Report Generation

Authors: Shivam Chand Kaushik

Organizations: School of Artificial Intelligence and Data Engineering (SAIDE), Indian Institute of Technology Jodhpur, India

Abstract

Semiconductor failure analysis (FA) requires engineers to examine inspection images, correlate equipment telemetry, consult historical defect records, and write structured reports, a process that can consume several hours of expert time per case. We present SemiFA, an agentic multi-modal framework that autonomously generates structured FA reports from semiconductor inspection images in under one minute. SemiFA decomposes FA into a four-agent LangGraph pipeline: a DefectDescriber that classifies and narrates defect morphology using DINOv2 and LLaVA-1.6, a RootCauseAnalyzer that fuses SECS/GEM equipment telemetry with historically similar defects retrieved from a Qdrant vector database, a SeverityClassifier that assigns severity and estimates yield impact, and a RecipeAdvisor that proposes corrective process adjustments. A fifth node assembles a PDF report. We introduce SemiFA-930, a dataset of 930 annotated semiconductor defect images paired with structured FA narratives across nine defect classes, drawn from procedural synthesis, WM-811K, and MixedWM38. Our DINOv2-based classifier achieves 92.1% accuracy on 140 validation images (macro F1 = 0.917), and the full pipeline produces complete FA reports in 48 seconds on an NVIDIA A100-SXM4-40 GB GPU. A GPT-4o judge ablation across four modality conditions demonstrates that multi-modal fusion improves root cause reasoning by +0.86 composite points (1-5 scale) over an image-only baseline, with equipment telemetry as the more load-bearing modality. To our knowledge, SemiFA is the first system to integrate SECS/GEM equipment telemetry into a vision-language model pipeline for autonomous FA report generation.

Figures & tables

Fig. 1: Pipeline architecture of the SemiFA system. Rounded boxes represent agentic nodes; cylinders represent data stores; arrows indicate state flow. All agentic nodes share LLaVA-1.6 via a thread-safe ModelRegistry singleton.

SemiFA-930 Class	WM-811K Label	Source
scratch	—	Synthetic only
particle_contam.	—	Synthetic only
edge_crack	—	Synthetic only
center_cluster	Center	WM-811K
local_cluster	Loc	WM-811K
ring_pattern	Donut, Edge-Ring	WM-811K + MixedWM38

TABLE I: Label taxonomy mapping from WM-811K to SemiFA-930 . Three classes (scratch, particle_contamination, edge_crack) have no WM-811K equivalent and are provided exclusively through procedural synthesis.

Class	Synthetic	WM-811K	MixedWM38	Total
scratch	112	0	0	112
particle_contam.	106	0	0	106
edge_crack	100	0	0	100
center_cluster	0	106	0	106
local_cluster	0	100	0	100
ring_pattern	0	54	40	94

TABLE II: Image count per defect class and data source in SemiFA-930 .

Defect Class	Train	Val	Physical Mechanism
scratch	94	18	Mechanical contact
particle_contam.	96	10	Contamination event
edge_crack	84	16	Dicing / handling
center_cluster	93	13	CMP / CVD bowl
local_cluster	79	21	Plasma non-uniformity
ring_pattern	71	23	Spin-coat / edge-bead

TABLE III: Defect class distribution in SemiFA-930 (930 total: 790 train, 140 val). Each class includes domain knowledge about the physical root cause mechanism.

Fig. 2: Representative examples from each defect class in SemiFA-930 . Each cell shows a $256\times 256$ wafer map with its severity badge (top-left) and root cause annotation (bottom). Spatial patterns are clearly distinguishable, though some classes (e.g., local_cluster vs. random_defects) present challenging overlap.

Defect Class	Prec.	Recall	F1
scratch	1.000	0.722	0.839
particle_contam.	1.000	1.000	1.000
edge_crack	0.933	0.875	0.903
center_cluster	1.000	1.000	1.000
local_cluster	0.913	1.000	0.955
ring_pattern	1.000	0.957	0.978

TABLE IV: Per-class classification results on the SemiFA-930 validation set (140 images). DINOv2-base (frozen) + MLP head.

Method	Accuracy	Macro F1	Trainable Params
CLIP ViT-B/32 zero-shot	20.7%	0.173	0
ResNet-50 (e2e fine-tuned)	82.9%	0.839	23.5M
DINOv2 + MLP (ours)	90.0%	0.898	214K

TABLE V: Comparison of visual encoder approaches on SemiFA-930 validation split (140 images). All models trained and evaluated on identical data splits.

Pipeline Node	Median (s)	Fraction
DefectDescriber	22.0	45.5%
RootCauseAnalyzer	11.9	24.6%
SeverityClassifier	5.5	11.4%
RecipeAdvisor	6.5	13.3%
ReportGenerator	2.5	5.2%
Total	48.4	100%

TABLE VI: Per-node latency breakdown (single pipeline execution, NVIDIA A100-SXM4-40 GB, PyTorch 2.1.0, LLaVA-1.6 in 4-bit NF4).

Condition	Spec.	Action.	Ground.	Composite
Full (Retrieval + Telemetry)	4.2	2.4	4.2	3.60
No Retrieval (Telemetry only)	4.2	2.8	4.2	3.73
No Telemetry (Retrieval only)	3.8	2.2	4.0	3.33
Baseline (Description only)	3.0	2.2	3.0	2.74

TABLE VII: Multi-modal fusion ablation (5 cases, GPT-4o automated judge, scores 1–5). Spec. = specificity; Action. = actionability; Ground. = grounding. Composite = mean of three dimensions.

\Delta

= composite relative to Full SemiFA .

Case	Equipment	Defect Class	Conf.	Sev.	Root Cause (Top-1)	Recommendation (Primary)
1	EQ-INSP-01	scratch	0.91	MAJOR	Linear track across 12 die consistent with mechanical contact; vacuum chuck pressure anomaly logged 45 min prior to inspection	Inspect and replace vacuum chuck gasket; verify chuck flatness within $\pm$ 2 $\mu$ m specification
2	EQ-CVD-03	center_cluster	0.84	MAJOR	Defect concentration within 25% radius correlates with CVD chamber temperature gradient; similar pattern in lot LOT-2024-052	Recalibrate showerhead spacing; increase edge zone heater setpoint by 3 ∘ C
3	EQ-ETCH-07	ring_pattern	0.79	MAJOR	Annular band at 60–75% radius matches spin-coat edge-bead removal failure; RF impedance alarm logged during prior etch step	Verify edge-bead removal nozzle position; clean solvent delivery line
4	EQ-DICE-12	edge_crack	0.88	MAJOR	Peripheral die damage consistent with dicing blade wear; blade lifetime at 94% of scheduled replacement interval	Replace dicing blade; reduce dicing feed rate by 10% for remaining lots
5	EQ-LITHO-02	no_defect	0.97	NONE	No defect pattern detected; uniform yield map with $>$ 98% good die; all equipment parameters within specification	No corrective action required; continue standard monitoring

TABLE VIII: Five qualitative case studies demonstrating end-to-end SemiFA report generation. For brevity, only the top-ranked root cause hypothesis and primary recommendation are shown. Full reports contain 2–3 hypotheses and 3–5 recommendations each.

Fault Diagnosis

Fault diagnosis identifies, localizes, and explains failures in engineered and computational systems using sensor data, logs, models, and domain knowledge.

See all 148 papers

WaferSAGE: Large Language Model-Powered Wafer Defect Analysis via Synthetic Data Generation and Rubric-Guided Reinforcement Learning
Apr 30, 2026Ke Xu, Zhongyuan LianVLM Evaluation Visual Question Answering
Shanghai Huahong Grace Semiconductor Manufacturing Corporation, Shanghai, 201203, China · Dept. of Automation, School of Information Science and Engineering, East China University of Science and Technology, Shanghai, 200237, China
SePArate: Segmenting Patterns from Defects in Wafer Manufacturing Using Weak Supervision
Aug 31, 2026Dain Kwon, Changmin Shin, Sunjong Park +5Weakly Supervised Image Segmentation Semiconductor Manufacturing
Seoul National University · SK Hynix
DeFA: Dependency-Guided Failure Attribution for LLM Agents
Oct 1, 2026Bo Deng, Xinlei Zheng, Yi Wei +6Agent Failure Analysis LLM Agent Evaluation
Qwen DianJin Team, Alibaba Cloud Computing · Beihang University

Apr 30, 2026J/K move · Enter open · S save

WaferSAGE: Large Language Model-Powered Wafer Defect Analysis via Synthetic Data Generation and Rubric-Guided Reinforcement Learning

Ke Xu, Zhongyuan Lian

Shanghai Huahong Grace Semiconductor Manufacturing Corporation, Shanghai, 201203, China · Dept. of Automation, School of Information Science and Engineering, East China University of Science and Technology, Shanghai, 200237, China

VLM Evaluation Visual Question Answering Semiconductor Manufacturing Synthetic Data Generation Industrial Anomaly Detection RL for VLMs

Open paper AskPDF arXiv

We present WaferSAGE, a framework for wafer defect visual question answering using small vision-language models. To address data scarcity in semiconductor manufacturing, we propose a three-stage synthesis pipeline incorporating structured rubric generation for precise evaluation. Starting from limited labeled wafer maps, we employ clustering-based cleaning to filter label noise, then generate comprehensive defect descriptions using vision-language models, which are converted into structured evaluation rubrics criteria. These rubrics guide the synthesis of VQA pairs, ensuring coverage across defect type identification, spatial distribution, morphology, and root cause analysis. Our dual assessment framework aligns rule-based metrics with LLM-Judge scores via Bayesian optimization, enabling reliable automated evaluation. Through curriculum-based reinforcement learning with Group Sequence Policy Optimization (GSPO) and rubric-aligned rewards, our 4B-parameter Qwen3-VL model achieves a 6.493 LLM-Judge score, closely approaching Gemini-3-Flash (7.149) while enabling complete on-premise deployment. We demonstrate that small models with domain-specific training can surpass proprietary large models in specialized industrial visual understanding, offering a viable path for privacy-preserving, cost-effective deployment in semiconductor manufacturing.

Abstract

Fault Diagnosis

Explore similar work

WaferSAGE: Large Language Model-Powered Wafer Defect Analysis via Synthetic Data Generation and Rubric-Guided Reinforcement Learning

SePArate: Segmenting Patterns from Defects in Wafer Manufacturing Using Weak Supervision

DeFA: Dependency-Guided Failure Attribution for LLM Agents