Do Words Matter? What Text Contributes to Few and Full-Shot Vision-Language Anomaly Detection

Arda Baris Basaran; Mesut Ceylan; Elgar Fleisch; Filipe Barata

cs.CVOct 25, 2026

Do Words Matter? What Text Contributes to Few and Full-Shot Vision-Language Anomaly Detection

Authors: Arda Baris Basaran, Mesut Ceylan, Elgar Fleisch, Filipe Barata

Organizations: ETH Zurich · University of St. Gallen

Abstract

Vision-language anomaly detectors describe normal and abnormal appearance with text, and their gains are commonly credited to the semantic content of these descriptions. This attribution is plausible in the zero-shot setting, but few- and full-shot detectors interpose a trained visual adapter, and in that setting it has not been isolated. Changing a description changes both its meaning and its geometry, so accuracy comparisons cannot separate the two. We ask what a fixed pair of text embeddings, or anchors, contributes to such a detector: semantic content, effective scale, or joint anchor geometry. For a single anchor pair, the answer is effective scale alone. The score reads the anchors only through their difference, and a trained adapter can rotate to follow any replacement direction, leaving that direction, and with it the linguistic content, unidentifiable. We prove that replacement anchors with the same difference-vector length permit exactly the same probability maps and attainable losses. Across seven medical datasets, three vision backbones, both supervision regimes, and three adapter families, scale-matched random and orthogonal anchors track correct prompts; breaking the scale match opens a gap that restoring it closes without text. The same pattern holds in MVFA and when MadCLIP's learned prompts are replaced by a text-free learned scale. When one adapter serves many categories, the answer expands to their joint arrangement. Rotating all category axes together changes nothing, as predicted. For the tested CLIP prompts, these axes are nearly collinear, and meaning-free rotations that spread them apart improve performance on both industrial benchmarks. Even under frozen zero-shot transfer, a random anchor frame transfers as well as the prompt frame: coordinate consistency, not meaning. What these detectors read from their prompts is scale and arrangement, not semantic content.

Figures & tables

Anchor condition	VisA		MVTec-AD
Anchor condition	AC	AS	AC	AS
Correct prompts	$86.2{\pm}0.1$	$91.7{\pm}0.1$	$96.2{\pm}0.1$	$93.6{\pm}0.1$
Shared rotation	$86.3{\pm}0.1$	$91.6{\pm}0.0$	$96.2{\pm}0.1$	$93.6{\pm}0.1$
Independent rotations	$91.8{\pm}0.1$	$96.1{\pm}0.1$	$97.4{\pm}0.1$	$96.6{\pm}0.0$

Table 3: Full-shot industrial results (%), mean

\pm

sd over three training seeds (and geometry seeds where applicable). AC and AS are image and pixel AUROC, category-macro means over the

T

categories; Appendix Table 22 adds PRO, AUPR, and F1max.

Appendix figures & tables14 assets

Supplementary material from the paper’s appendix.

Appendix

Target dataset	Target domain	Donor prompt bank
BraTS2021	brain MRI	BUSI (breast ultrasound)
LiverCT	liver CT	OCT2017 (retinal OCT)
RESC	retinal OCT	CheXpert (chest radiograph)
OCT2017	retinal OCT	Kaggle BrainMRI (brain MRI)
BUSI	breast ultrasound	RESC (retinal OCT)
CheXpert	chest radiograph	BraTS2021 (brain MRI)

Appendix

Table 4: Fixed target-to-donor mapping for the mismatched-dataset prompt-bank intervention. The donor contributes its complete normal and abnormal description banks; no donor prompt is rewritten to mention the target dataset.

Normal	Abnormal
An axial T2-FLAIR brain MRI slice of the brain showing absence of any annotated glioma region, including edema, necrotic or non-enhancing tumor core, and enhancing tumor.	An axial T2-FLAIR brain MRI slice of the brain showing annotated brain tumor tissue.
A clinical axial T2-FLAIR brain MRI slice of the brain showing absence of any annotated glioma region, including edema, necrotic or non-enhancing tumor core, and enhancing tumor.	A clinical axial T2-FLAIR brain MRI slice of the brain showing a tumor-bearing axial brain slice.
A diagnostic axial T2-FLAIR brain MRI slice of the brain demonstrating absence of any annotated glioma region, including edema, necrotic or non-enhancing tumor core, and enhancing tumor.	A diagnostic axial T2-FLAIR brain MRI slice of the brain demonstrating glioma tissue.
This axial T2-FLAIR brain MRI slice of the brain shows absence of any annotated glioma region, including edema, necrotic or non-enhancing tumor core, and enhancing tumor.	This axial T2-FLAIR brain MRI slice of the brain shows a whole-tumor region.
This axial T2-FLAIR brain MRI slice of the brain demonstrates absence of any annotated glioma region, including edema, necrotic or non-enhancing tumor core, and enhancing tumor.	This axial T2-FLAIR brain MRI slice of the brain demonstrates peritumoral edema.
The brain on this axial T2-FLAIR brain MRI slice shows absence of any annotated glioma region, including edema, necrotic or non-enhancing tumor core, and enhancing tumor.	The brain on this axial T2-FLAIR brain MRI slice shows edema-associated T2-FLAIR hyperintensity.

Appendix

Table 5: Exact prompts for BraTS2021.

Normal	Abnormal
An axial liver CT slice of the liver showing absence of any annotated primary or metastatic liver tumor on the slice.	An axial liver CT slice of the liver showing an annotated liver tumor.
A clinical axial liver CT slice of the liver showing absence of any annotated primary or metastatic liver tumor on the slice.	A clinical axial liver CT slice of the liver showing a neoplastic liver lesion.
A diagnostic axial liver CT slice of the liver demonstrating absence of any annotated primary or metastatic liver tumor on the slice.	A diagnostic axial liver CT slice of the liver demonstrating a primary liver tumor.
This axial liver CT slice of the liver shows absence of any annotated primary or metastatic liver tumor on the slice.	This axial liver CT slice of the liver shows hepatocellular carcinoma.
This axial liver CT slice of the liver demonstrates absence of any annotated primary or metastatic liver tumor on the slice.	This axial liver CT slice of the liver demonstrates cholangiocarcinoma.
The liver on this axial liver CT slice shows absence of any annotated primary or metastatic liver tumor on the slice.	The liver on this axial liver CT slice shows a metastatic liver lesion.

Appendix

Table 6: Exact prompts for LiverCT.

Normal	Abnormal
A retinal OCT B-scan of the retina showing absence of any annotated retinal-edema lesion, including subretinal fluid and pigment epithelial detachment.	A retinal OCT B-scan of the retina showing an annotated retinal-edema lesion.
A clinical retinal OCT B-scan of the retina showing absence of any annotated retinal-edema lesion, including subretinal fluid and pigment epithelial detachment.	A clinical retinal OCT B-scan of the retina showing retinal edema.
A diagnostic retinal OCT B-scan of the retina demonstrating absence of any annotated retinal-edema lesion, including subretinal fluid and pigment epithelial detachment.	A diagnostic retinal OCT B-scan of the retina demonstrating an edematous retinal region.
This retinal OCT B-scan of the retina shows absence of any annotated retinal-edema lesion, including subretinal fluid and pigment epithelial detachment.	This retinal OCT B-scan of the retina shows edema-associated retinal thickening.
This retinal OCT B-scan of the retina demonstrates absence of any annotated retinal-edema lesion, including subretinal fluid and pigment epithelial detachment.	This retinal OCT B-scan of the retina demonstrates subretinal fluid.
The retina on this retinal OCT B-scan shows absence of any annotated retinal-edema lesion, including subretinal fluid and pigment epithelial detachment.	The retina on this retinal OCT B-scan shows a localized pocket of subretinal fluid.

Appendix

Table 7: Exact prompts for RESC.

Normal	Abnormal
A retinal OCT B-scan of the retina showing a disease-free retinal appearance with no choroidal neovascularization, diabetic macular edema, or drusen.	A retinal OCT B-scan of the retina showing choroidal neovascularization.
A clinical retinal OCT B-scan of the retina showing a disease-free retinal appearance with no choroidal neovascularization, diabetic macular edema, or drusen.	A clinical retinal OCT B-scan of the retina showing CNV-related neovascular change.
A diagnostic retinal OCT B-scan of the retina demonstrating a disease-free retinal appearance with no choroidal neovascularization, diabetic macular edema, or drusen.	A diagnostic retinal OCT B-scan of the retina demonstrating a choroidal neovascular membrane.
This retinal OCT B-scan of the retina shows a disease-free retinal appearance with no choroidal neovascularization, diabetic macular edema, or drusen.	This retinal OCT B-scan of the retina shows diabetic macular edema.
This retinal OCT B-scan of the retina demonstrates a disease-free retinal appearance with no choroidal neovascularization, diabetic macular edema, or drusen.	This retinal OCT B-scan of the retina demonstrates DME-related retinal thickening.
The retina on this retinal OCT B-scan shows a disease-free retinal appearance with no choroidal neovascularization, diabetic macular edema, or drusen.	The retina on this retinal OCT B-scan shows diabetic macular edema with intraretinal fluid.

Appendix

Table 8: Exact prompts for OCT2017.

Normal	Abnormal
A grayscale breast ultrasound image of breast tissue showing absence of any focal breast mass labeled benign or malignant.	A grayscale breast ultrasound image of breast tissue showing a focal breast mass.
A clinical grayscale breast ultrasound image of breast tissue showing absence of any focal breast mass labeled benign or malignant.	A clinical grayscale breast ultrasound image of breast tissue showing an annotated breast lesion.
A diagnostic grayscale breast ultrasound image of breast tissue demonstrating absence of any focal breast mass labeled benign or malignant.	A diagnostic grayscale breast ultrasound image of breast tissue demonstrating a localized breast nodule.
This grayscale breast ultrasound image of breast tissue shows absence of any focal breast mass labeled benign or malignant.	This grayscale breast ultrasound image of breast tissue shows a benign breast mass.
This grayscale breast ultrasound image of breast tissue demonstrates absence of any focal breast mass labeled benign or malignant.	This grayscale breast ultrasound image of breast tissue demonstrates a benign breast lesion.
Breast tissue on this grayscale breast ultrasound image shows absence of any focal breast mass labeled benign or malignant.	Breast tissue on this grayscale breast ultrasound image shows a benign breast nodule.

Appendix

Table 9: Exact prompts for BUSI.

Normal	Abnormal
A chest radiograph of the chest showing the No Finding state with none of the twelve evaluated disease observations.	A chest radiograph of the chest showing enlarged cardiomediastinum.
A clinical chest radiograph of the chest showing the No Finding state with none of the twelve evaluated disease observations.	A clinical chest radiograph of the chest showing cardiomegaly.
A diagnostic chest radiograph of the chest demonstrating the No Finding state with none of the twelve evaluated disease observations.	A diagnostic chest radiograph of the chest demonstrating lung opacity.
This chest radiograph of the chest shows the No Finding state with none of the twelve evaluated disease observations.	This chest radiograph of the chest shows a lung lesion.
This chest radiograph of the chest demonstrates the No Finding state with none of the twelve evaluated disease observations.	This chest radiograph of the chest demonstrates pulmonary edema.
The chest on this chest radiograph shows the No Finding state with none of the twelve evaluated disease observations.	The chest on this chest radiograph shows pulmonary consolidation.

Appendix

Table 10: Exact prompts for CheXpert.

Normal	Abnormal
A 2D brain MRI image of the brain showing absence of any visible brain tumor or intracranial mass.	A 2D brain MRI image of the brain showing a visible brain tumor.
A clinical 2D brain MRI image of the brain showing absence of any visible brain tumor or intracranial mass.	A clinical 2D brain MRI image of the brain showing an intracranial tumor.
A diagnostic 2D brain MRI image of the brain demonstrating absence of any visible brain tumor or intracranial mass.	A diagnostic 2D brain MRI image of the brain demonstrating a tumor-bearing brain MRI appearance.
This 2D brain MRI image of the brain shows absence of any visible brain tumor or intracranial mass.	This 2D brain MRI image of the brain shows a focal intracranial mass.
This 2D brain MRI image of the brain demonstrates absence of any visible brain tumor or intracranial mass.	This 2D brain MRI image of the brain demonstrates a neoplastic brain lesion.
The brain on this 2D brain MRI image shows absence of any visible brain tumor or intracranial mass.	The brain on this 2D brain MRI image shows a space-occupying brain lesion.

Appendix

Table 11: Exact prompts for Kaggle BrainMRI.

Normal state phrase	Abnormal state phrase
{}	damaged {}
flawless {}	broken {}
perfect {}	{} with flaw
unblemished {}	{} with defect
{} without flaw	{} with damage
{} without defect	␣disease {}

Appendix

Table 12: Exact state phrases used to construct every industrial prompt bank. The object placeholder {} is replaced by the noun in Table 14 . The leading space in the sixth abnormal phrase is retained from the reference implementation.

	Template
1	a cropped photo of the {}.
2	a cropped photo of a {}.
3	a close-up photo of a {}.
4	a close-up photo of the {}.
5	a bright photo of a {}.
6	a bright photo of the {}.

Appendix

Table 13: Exact photo templates used for the industrial prompt ensemble. The state placeholder {} is replaced by one of the state phrases in Table 12 .

Dataset	Object nouns
MVTec-AD	bottle; cable; capsule; carpet; grid; hazelnut; leather; metal nut; pill; screw; tile; toothbrush; transistor; wood; zipper
VisA	candle; capsules; cashew; chewing gum; fryum; macaroni 1; macaroni 2; printed circuit board 1; printed circuit board 2; printed circuit board 3; printed circuit board 4; pipe fryum

Appendix

Table 14: Exact object nouns substituted into the industrial prompt templates. Numbered VisA categories deliberately retain their index.

Quantity	BraTS 2021	Liver CT	RESC	OCT 2017	BUSI	CheXpert	Kaggle BrainMRI
$\rho_{\mathrm{correct}}$	$0.9255$	$0.9711$	$0.9524$	$0.9408$	$0.9771$	$0.9241$	$0.9742$
$c$	$0.2730$	$0.1700$	$0.2183$	$0.2432$	$0.1513$	$0.2755$	$0.1608$
$\lambda_{\mathrm{eff}}^{\mathrm{train}}$	$38.61$	$24.04$	$30.87$	$34.40$	$21.40$	$38.97$	$22.74$
$\lambda_{\mathrm{eff}}^{\mathrm{eval}}$	$7.72$	$4.81$	$6.17$	$6.88$	$4.28$	$7.79$	$4.55$

Appendix

Table 15: Correct-prompt geometry for the seven medical datasets. The reported values determine both the Gram-matched random construction and the rescaling of matched orthogonal anchors.

MVTec-AD					VisA
Category	$\rho_{t}$	$c_{t}$	$\lambda_{\mathrm{eff},t}^{\mathrm{train}}$	$\lambda_{\mathrm{eff},t}^{\mathrm{eval}}$	Category	$\rho_{t}$	$c_{t}$	$\lambda_{\mathrm{eff},t}^{\mathrm{train}}$	$\lambda_{\mathrm{eff},t}^{\mathrm{eval}}$
bottle	$0.9754$	$0.1567$	$22.17$	$4.43$	candle	$0.9743$	$0.1604$	$22.68$	$4.54$
cable	$0.9771$	$0.1514$	$21.41$	$4.28$	capsules	$0.9742$	$0.1605$	$22.70$	$4.54$
capsule	$0.9715$	$0.1688$	$23.87$	$4.77$	cashew	$0.9771$	$0.1514$	$21.41$	$4.28$
carpet	$0.9767$	$0.1526$	$21.58$	$4.32$	chewing gum	$0.9801$	$0.1410$	$19.95$	$3.99$
grid	$0.9735$	$0.1629$	$23.03$	$4.61$	fryum	$0.9701$	$0.1730$	$24.47$	$4.89$

Appendix

Table 16: Per-category correct-prompt geometry for the unified industrial detector. Each category’s shared- and independent-rotation controls preserve its displayed

\rho_{t}

c_{t}

, and

\lambda_{\mathrm{eff},t}

; independent rotations change only cross-category off-diagonal geometry. The participation-ratio effective rank of the correct-prompt difference Gram matrix

{\bm{M}}

(Section 6.4 ) is

r_{\mathrm{eff}}=1.79

for MVTec-AD and

1.78

for VisA. Shared rotations preserve

{\bm{M}}

and hence

r_{\mathrm{eff}}

; independent rotations raise it to

14.28

–

14.29

of 15 (MVTec-AD) and about

11.10

of 12 (VisA) across the three geometry seeds.

Family	Parameterization	Closed	Reason
linear	${\bm{y}}=\widetilde{\bm{W}}^{\top}\tilde{\bm{z}}$ , $\widetilde{\bm{W}}$ free	yes	absorb ${\bm{Q}}$ into $\widetilde{\bm{W}}$
MLP	${\bm{y}}={\bm{W}}_{2}\,\mathrm{GELU}({\bm{W}}_{1}{\bm{z}})$	yes	absorb ${\bm{Q}}$ into ${\bm{W}}_{2}$
Transformer	${\bm{y}}={\bm{U}}_{\mathrm{out}}T({\bm{U}}_{\mathrm{in}}Z+E_{\mathrm{pos}})$	yes	absorb ${\bm{Q}}$ into ${\bm{U}}_{\mathrm{out}}$

Appendix

Table 18: Rotation closure of the three adapter families.

Multimodal Anomaly Detection

Multimodal anomaly detection identifies unusual patterns by combining complementary signals such as images, depth, video, text, and sensor data.

See all 97 papers

Mar 9, 2026

VisualAD: Language-Free Zero-Shot Anomaly Detection via Vision Transformer

Zero-shot anomaly detection (ZSAD) requires detecting and localizing anomalies without access to target-class anomaly samples. Mainstream methods rely on vision-language models (VLMs) such as CLIP: they build hand-crafted or learned prompt sets for normal and abnormal semantics, then compute image-text similarities for open-set discrimination. While effective, this paradigm depends on a text encoder and cross-modal alignment, which can lead to training instability and parameter redundancy. This work revisits the necessity of the text branch in ZSAD and presents VisualAD, a purely visual framework built on Vision Transformers. We introduce two learnable tokens within a frozen backbone to directly encode normality and abnormality. Through multi-layer self-attention, these tokens interact with patch tokens, gradually acquiring high-level notions of normality and anomaly while guiding patches to highlight anomaly-related cues. Additionally, we incorporate a Spatial-Aware Cross-Attention (SCA) module and a lightweight Self-Alignment Function (SAF): SCA injects fine-grained spatial information into the tokens, and SAF recalibrates patch features before anomaly scoring. VisualAD achieves state-of-the-art performance on 13 zero-shot anomaly detection benchmarks spanning industrial and medical domains, and adapts seamlessly to pretrained vision backbones such as the CLIP image encoder and DINOv2. Code: https://github.com/7HHHHH/VisualAD

Yanning Hou, Peiyuan Li, Zirui Liu +4

College of Intelligence Science and Technology, National University of Defense Technology, Changsha, China · School of Artificial Intelligence, Anhui University, Hefei, China · State Key Laboratory of Opto-Electronic Information Acquisition and Protection Technology, Anhui University, Hefei, China

Zero-Shot Anomaly Detection Vision Foundation Model Adaptation Medical Image Anomaly Detection

Ask PDF

Sep 7, 2026

Proximity-CLIP: Text-Guided Semantic Proximity Learning for Zero-Shot Anomaly Detection

Vision-language models offer a promising approach for zero-shot anomaly detection (ZSAD). However, due to object-centric bias, normal and anomalous text prototypes exhibit a high semantic overlap. While enforcing strict orthogonality between them improves discriminability, mapping highly contiguous visual inputs onto drastically orthogonal prototypes introduces a geometric dilemma, disrupting the pre-trained structural continuity. To address this problem, we propose Proximity-CLIP, a framework that visually calibrates the semantic margin to guide visual adaptation. First, we introduce a visually-calibrated semantic proximity learning mechanism that uses a bounded dynamic regularization to learn an appropriate semantic margin, ensuring discriminative separation while preserving structural alignment. Second, we design an Anomaly Query Module (AQM) driven by these text priors. Using the calibrated anomalous prototype as a semantic query, the AQM actively retrieves localized defect cues from contextual visual patches, mitigating the dilution of subtle anomalies during global pooling. Extensive experiments demonstrate that Proximity-CLIP outperforms current state-of-the-art methods across multiple ZSAD benchmarks with minimal architectural modifications.

Manwen Yang, Leqian Ding, Yu Guo +1

School of Software Engineering, Xi’an Jiaotong University · State Key Laboratory of Human-Machine Hybrid Augmented Intelligence, Institute of Artificial Intelligence and Robotics, Xi’an Jiaotong University

VLM Adaptation Anomaly Localization Zero-Shot Anomaly Detection

Ask PDF

Oct 4, 2026

Anchor and Adapt: Asymmetric Prompt Adaptation for Few-Shot Industrial Anomaly Detection

In few-shot industrial anomaly detection, the few normal target images provide no direct defect supervision, making anomaly prompts difficult to learn from these samples alone. Some vision-language methods therefore use manually specified descriptions to supply explicit anomaly semantics. However, constructing these descriptions requires product-specific effort, and their effectiveness depends on prompt selection. We propose Anchor and Adapt, a two-stage prompt learning framework that separates the acquisition of anomaly semantics from adaptation to target normal appearance. Stage I learns transferable normal and abnormal anchors from annotated auxiliary data. Stage II keeps these anchors fixed and adapts an additional normal branch using the few target normal samples. The inherited and adapted normal branches jointly characterize target normality, with text-anchor regularization encouraging consistency with the generic normal prior and separation from the abnormal anchors. This design retains learned anomaly knowledge while reducing dependence on category-specific anomaly templates, without requiring synthetic anomaly generation. Cross-dataset experiments between MVTec-AD and VisA under 1-, 2-, and 4-shot settings demonstrate competitive detection and localization performance. Controlled ablations assess the roles of transferred anchors, asymmetric adaptation, dual-normal representations, and anchor regularization.

Mengyang Zhao, Teng Fu, Haiyang Yu +3

Shanghai Key Lab of Intelligent Information Processing and College of Computer Science and Artificial Intelligence, Fudan University, Shanghai 200433, China

Prompt Learning for VLMs Few-Shot Anomaly Detection Industrial Anomaly Detection VLM Adaptation Anomaly Localization

Ask PDF

Abstract

Multimodal Anomaly Detection

Explore similar work

VisualAD: Language-Free Zero-Shot Anomaly Detection via Vision Transformer

Proximity-CLIP: Text-Guided Semantic Proximity Learning for Zero-Shot Anomaly Detection

Anchor and Adapt: Asymmetric Prompt Adaptation for Few-Shot Industrial Anomaly Detection