cs.CVNov 16, 2026

Deciding Without Pointing: A Case Study of Whether a Decision Model Localizes the Anomalies It Detects

Authors: Hsiang-Hung Lee

Organizations: Politecnico di Milano, Milan, Italy

Abstract

Decision models can answer a typed question about an image with a probability per option, without generating text. In our case study of Clef-flash (post-trained Qwen3.5-9B backbone) on industrial inspection benchmarks, we ask whether it localizes the anomalies it detects. We obtained uneven results with zero-shot detection: per-category image AUROC 0.924 on MVTec AD and 0.668 on MVTec AD 2. On the former, the decision head's routing attention produces 0.391 mean per-image pixel AUROC, and hiding the image tokens from it leaves detection unchanged. The multimodal LLM's backbone attention does point at the defect, however it falls below a center prior when we change question. It falls below chance if we remove textual context in the question, partly due to backbone attention leaving the object, while the head's map hardly changes. The backbone's states carry the location when probed with normal images, on the test-split images, however, the decision depends on the defect region. A larger model, Clef, that comes with a post-trained Qwen3.8-27B backbone, gives no useful map in the decision head either. We found that it's possible to recover anomaly location by asking question specifically on an image region, retraining the head or reading the backbone, but none matches dedicated few-shot localizers.

Explore similar work

CardsList
  1. VerdictAD: Few-Shot Anomaly Detection via Decision-Guided Refinement

    Nov 2, 2026Shangquan Sun, Wenjun Luo, Yuanzhi Wang +4Anomaly DetectionFew-Shot Anomaly Detection

  2. TED:Text-Axis Evidence Decomposition for Prompted Anomaly Localization

    Sep 30, 2026JinYoung Kim, Geonho Kim, GiJeong Park +2Domain GeneralizationAnomaly Localization

  3. Reasoning-Driven Anomaly Detection and Localization with Image-Level Supervision

    Mar 28, 2026Yizhou Jin, Yuezhu Feng, Jinjin Zhang +3weakly supervised anomaly detectionAnomaly Localization