cs.CVOct 30, 2026

Beyond CLIP: Calibrated (Few-Shot) Defect Detection With System One Decision Models

Authors: Fadel M. Megahed, Ying-Ju Chen

Organizations: Farmer School of Business, Miami University, Oxford, OH 4050, USA · College of Arts and Sciences, University of Dayton, Dayton, OH 45469, USA

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Cameras and vision sensors are widely used for product and process inspection in manufacturing environments. Traditionally, this process required thousands of images of both nominal and defective products to train such models. Recent research shows that vision transformer (ViT) models, such as CLIP, can reduce this requirement to 50-100 images per nominal and defective class. However, collecting even 50 defective images can be difficult in high-yield or low-volume manufacturing processes. In this paper, we examine the utility of two recently proposed structured decision models (Jev and Clef-Flash) in applications using as few as k=2 reference images per class at inference. In the process, we provide three main contributions: (a) a framework for k-shot learning of these models for image classification/inspection applications; (b) an agentic approach for automatically refining the text descriptions that guide image classification in these decision models; and (c) an approach to calibrate the resulting defect probabilities to improve their reliability for inspection decisions. We test these contributions in five manufacturing case studies. Using two reference images per class, both Jev and Clef-Flash outperform CLIP's test accuracies with the same references (0.75-1.00 [Jev] and 0.71-0.99 [Clef-Flash] vs 0.67-0.84 [CLIP]). Furthermore, we show that our agentic refinement of text descriptions significantly improves accuracy across 6 of 8 model-application combinations, and our calibration approach reduces the Brier score in 13 of the 15 model-application configurations. To support the reproducibility of our work, we provide our replication code and have created the R "decisions" package for evaluating additional models. We also provide a publicly available web app that allows practitioners to inspect their own images with no code.

Figures & tables

Figure 1: Compression of CLIP-predicted defect probabilities across the manufacturing case studies of [ 7 ] .
Figure 2: Overview of our k−k- shot learning framework for using decision models (e.g., Jev and Clef-Flash ) to inspect manufactured parts.
Figure 3: An example of the two nominal and two defective reference images per manufacturing case study, used by the decision models and reference-matched CLIP.
 Learning SetPartition of LLEvaluation
Case StudyLLR2R_{2}DDCCTest
Pans102––78
Extrusion5024825200
Textiles5024825200
Pipe staples5024817100
Microstructure5024825450
Table 1: Image allocation across the five case studies. Counts are per class, except for the test set, which includes both classes.
Table 2: Classification performance, inference time, and computational requirements across the five manufacturing case studies. Results are reported after Platt calibration. Bold values indicate the best performance for each case study.
Figure 4: Paired accuracy differences (95% bootstrap confidence intervals) for Jev and Clef-Flash : (a) versus CLIP, (b) refined versus generic descriptions, and (c) two versus zero reference images per class.
Candidate 1 , development AUC 0.59
NominalNo visible manufacturing defects; the part appears conforming.
DefectiveAt least one visible manufacturing defect or anomaly.
Candidate 2 , development AUC 0.74
NominalA close-up of a 3D printing nozzle and freshly extruded plastic; the extruded material forms a smooth, even layer of steady width that lies flat on the print surface.
DefectiveA close-up of a 3D printing nozzle and extruded plastic where the amount of material is wrong: either a thin, stringy or broken strand with dark gaps showing through, or a bulging, blobby, overly wide mass of plastic.
Candidate 3 , development AUC 0.78
Table 3: The descriptions that the Jev description loop kept for extrusion, with Git-style track changes.
Figure 5: Reliability diagrams comparing raw (gray) and Platt-calibrated (purple) defect probabilities for Jev , Clef-Flash , and CLIP across three sample case studies (see supplementary materials for full results). Values indicate test ECE before and after calibration.
More in this topic

Prompt Learning for VLMs

See all 38 papers

Explore similar work

CardsList
Jul 12, 2026

Answer-Conditioned Chain-of-Thought Distillation for Few-Shot Industrial Vision with Small VLMs

Deploying AI-based visual inspection in manufacturing is hard because requirements change often, new defect types appear, and large labeled datasets are rarely available. We propose answer-conditioned chain-of-thought (CoT) distillation for rapidly adapting small vision-language models (VLMs) to new industrial tasks using minimal labeled data. A frontier VLM receives each training image along with its correct label and generates a justified visual explanation. A 3B-parameter model is then fine-tuned on these reasoning-augmented examples via LoRA. By conditioning on correct answers, we ensure all training reasoning is directed toward the correct conclusion, which is critical because frontier models score as low as 24.1% on our hardest task. We validate on four industrial classification tasks spanning three image modalities using only 18 to 30 labeled images per task. Across 4 seeds per task (32 training runs), our method outperforms direct fine-tuning on all 16 seed-task combinations, with mean improvements of +1.7 to +4.4 percentage points. A controlled equal-budget experiment confirms the improvement comes from reasoning quality, not additional training steps. An unconditioned baseline demonstrates that with out answer-conditioning, wrong reasoning degrades performance by 17.8 percentage points. On weld radiograph classification, the fine-tuned 3B model outperforms GPT-4.1 by 10.0pp using just 24 training images.

Shubham Rao

Entropy AI Research Labs Private Limited

CoT DistillationVLM AdaptationVLM DistillationIndustrial InspectionFew-Shot Image Classification
AskPDF
Nov 2, 2026

VerdictAD: Few-Shot Anomaly Detection via Decision-Guided Refinement

Detecting defects in real-world industrial inspection often requires learning from only a few normal images per category. Recent few-shot anomaly detection methods have achieved strong performance by measuring feature differences between query images and normal references. However, visual differences do not always indicate genuine defects, as normal appearance variations can also produce high anomaly scores. Conversely, structural or logical anomalies may not produce strong local feature differences, making them difficult to detect. To address this limitation, we introduce VerdictAD, a training-free framework that refines existing anomaly detectors through visual decisions. Rather than replacing existing detectors, VerdictAD adopts a typed decision model to assess the query image and its local regions against a few normal references. By directly producing structured decision probabilities without text generation, the model provides complementary evidence for refining anomaly responses at different spatial scales. Building on this capability, we introduce two modules: Global Verdict Refinement (GVR) uses image-level decisions to refine the overall anomaly score, while Local Verdict Refinement (LVR) aggregates regional decisions to refine the anomaly map. Combined with a simple DINOv2-based detector, VerdictAD achieves competitive performance in one-shot and few-shot anomaly detection. More importantly, as a plug-and-play framework, it consistently improves existing anomaly detectors across MVTec AD, VisA, and MVTec LOCO AD, achieving strong results across one-shot and few-shot settings. Our code and experimental logs are publicly available at https://github.com/sgstq0811/VerdictAD-Jev-AD.

Shangquan Sun, Wenjun Luo, Yuanzhi Wang +4

Nanyang Technological University · Michigan State University · University of Michigan

Anomaly DetectionFew-Shot Anomaly DetectionIndustrial Anomaly Detection
AskPDF
Aug 22, 2026

Trustworthy Visual Quality Inspection under Data Scarcity in Manufacturing

Automated visual inspection in manufacturing aims to replace slow and inconsistent manual checks, but its economic value depends on whether its decisions can be trusted enough to automate routine inspection while reserving human expertise for ambiguous cases. In production-line settings, defective samples are scarce, since the process is optimized to produce good parts, which limits any learning-based inspector trained on real data alone. Compounding this, defect decisions emitted as hard labels with no confidence estimate carry an asymmetric cost: a false reject wastes a good product, while a false accept may increase the risk of undetected defects progressing through the production process. We address both problems by mitigating data scarcity through the generation of synthetic defective samples with a diffusion model, and meeting the need for confidence-aware decisions with a Bayesian classifier that defers ambiguous units to human review rather than misclassifying them. These components are embedded in a staged pipeline of successive, complementary checks. We evaluate how synthetic augmentation affects classification and localization on a test set of real defects, and examine the system's trustworthiness at three points: the decision, the synthetic data, and the pipeline structure. This work-in-progress reports preliminary results suggesting that diffusion-generated defects, combined with uncertainty-aware classification, can lower the cost of reaching a trustworthy, deployable inspection model under data scarcity.

Panagiotis Sapoutzoglou, Jessy Ribaira, Martin Kanounnikoff +3

National Technical University of Athens, Athens, Greece · German Aerospace Center (DLR), Germany · Philips, High Tech Campus 52, 5656 AG Eindhoven, The Netherlands +1

Industrial InspectionIndustrial Anomaly DetectionSynthetic Data Augmentation
AskPDF