Do Words Matter? What Text Contributes to Few and Full-Shot Vision-Language Anomaly Detection

Arda Baris Basaran; Mesut Ceylan; Elgar Fleisch; Filipe Barata

cs.CVOct 25, 2026

Do Words Matter? What Text Contributes to Few and Full-Shot Vision-Language Anomaly Detection

Authors: Arda Baris Basaran, Mesut Ceylan, Elgar Fleisch, Filipe Barata

Organizations: ETH Zurich · University of St. Gallen

Abstract

Vision-language anomaly detectors describe normal and abnormal appearance with text, and their gains are commonly credited to the semantic content of these descriptions. This attribution is plausible in the zero-shot setting, but few- and full-shot detectors interpose a trained visual adapter, and in that setting it has not been isolated. Changing a description changes both its meaning and its geometry, so accuracy comparisons cannot separate the two. We ask what a fixed pair of text embeddings, or anchors, contributes to such a detector: semantic content, effective scale, or joint anchor geometry. For a single anchor pair, the answer is effective scale alone. The score reads the anchors only through their difference, and a trained adapter can rotate to follow any replacement direction, leaving that direction, and with it the linguistic content, unidentifiable. We prove that replacement anchors with the same difference-vector length permit exactly the same probability maps and attainable losses. Across seven medical datasets, three vision backbones, both supervision regimes, and three adapter families, scale-matched random and orthogonal anchors track correct prompts; breaking the scale match opens a gap that restoring it closes without text. The same pattern holds in MVFA and when MadCLIP's learned prompts are replaced by a text-free learned scale. When one adapter serves many categories, the answer expands to their joint arrangement. Rotating all category axes together changes nothing, as predicted. For the tested CLIP prompts, these axes are nearly collinear, and meaning-free rotations that spread them apart improve performance on both industrial benchmarks. Even under frozen zero-shot transfer, a random anchor frame transfers as well as the prompt frame: coordinate consistency, not meaning. What these detectors read from their prompts is scale and arrangement, not semantic content.

Figures & tables

Anchor condition	VisA		MVTec-AD
Anchor condition	AC	AS	AC	AS
Correct prompts	$86.2{\pm}0.1$	$91.7{\pm}0.1$	$96.2{\pm}0.1$	$93.6{\pm}0.1$
Shared rotation	$86.3{\pm}0.1$	$91.6{\pm}0.0$	$96.2{\pm}0.1$	$93.6{\pm}0.1$
Independent rotations	$91.8{\pm}0.1$	$96.1{\pm}0.1$	$97.4{\pm}0.1$	$96.6{\pm}0.0$

Table 3: Full-shot industrial results (%), mean

\pm

sd over three training seeds (and geometry seeds where applicable). AC and AS are image and pixel AUROC, category-macro means over the

T

categories; Appendix Table 22 adds PRO, AUPR, and F1max.

Appendix figures & tables14 assets

Supplementary material from the paper’s appendix.

Appendix

Target dataset	Target domain	Donor prompt bank
BraTS2021	brain MRI	BUSI (breast ultrasound)
LiverCT	liver CT	OCT2017 (retinal OCT)
RESC	retinal OCT	CheXpert (chest radiograph)
OCT2017	retinal OCT	Kaggle BrainMRI (brain MRI)
BUSI	breast ultrasound	RESC (retinal OCT)
CheXpert	chest radiograph	BraTS2021 (brain MRI)

Appendix

Table 4: Fixed target-to-donor mapping for the mismatched-dataset prompt-bank intervention. The donor contributes its complete normal and abnormal description banks; no donor prompt is rewritten to mention the target dataset.

Normal	Abnormal
An axial T2-FLAIR brain MRI slice of the brain showing absence of any annotated glioma region, including edema, necrotic or non-enhancing tumor core, and enhancing tumor.	An axial T2-FLAIR brain MRI slice of the brain showing annotated brain tumor tissue.
A clinical axial T2-FLAIR brain MRI slice of the brain showing absence of any annotated glioma region, including edema, necrotic or non-enhancing tumor core, and enhancing tumor.	A clinical axial T2-FLAIR brain MRI slice of the brain showing a tumor-bearing axial brain slice.
A diagnostic axial T2-FLAIR brain MRI slice of the brain demonstrating absence of any annotated glioma region, including edema, necrotic or non-enhancing tumor core, and enhancing tumor.	A diagnostic axial T2-FLAIR brain MRI slice of the brain demonstrating glioma tissue.
This axial T2-FLAIR brain MRI slice of the brain shows absence of any annotated glioma region, including edema, necrotic or non-enhancing tumor core, and enhancing tumor.	This axial T2-FLAIR brain MRI slice of the brain shows a whole-tumor region.
This axial T2-FLAIR brain MRI slice of the brain demonstrates absence of any annotated glioma region, including edema, necrotic or non-enhancing tumor core, and enhancing tumor.	This axial T2-FLAIR brain MRI slice of the brain demonstrates peritumoral edema.
The brain on this axial T2-FLAIR brain MRI slice shows absence of any annotated glioma region, including edema, necrotic or non-enhancing tumor core, and enhancing tumor.	The brain on this axial T2-FLAIR brain MRI slice shows edema-associated T2-FLAIR hyperintensity.

Appendix

Table 5: Exact prompts for BraTS2021.

Normal	Abnormal
An axial liver CT slice of the liver showing absence of any annotated primary or metastatic liver tumor on the slice.	An axial liver CT slice of the liver showing an annotated liver tumor.
A clinical axial liver CT slice of the liver showing absence of any annotated primary or metastatic liver tumor on the slice.	A clinical axial liver CT slice of the liver showing a neoplastic liver lesion.
A diagnostic axial liver CT slice of the liver demonstrating absence of any annotated primary or metastatic liver tumor on the slice.	A diagnostic axial liver CT slice of the liver demonstrating a primary liver tumor.
This axial liver CT slice of the liver shows absence of any annotated primary or metastatic liver tumor on the slice.	This axial liver CT slice of the liver shows hepatocellular carcinoma.
This axial liver CT slice of the liver demonstrates absence of any annotated primary or metastatic liver tumor on the slice.	This axial liver CT slice of the liver demonstrates cholangiocarcinoma.
The liver on this axial liver CT slice shows absence of any annotated primary or metastatic liver tumor on the slice.	The liver on this axial liver CT slice shows a metastatic liver lesion.

Appendix

Table 6: Exact prompts for LiverCT.

Normal	Abnormal
A retinal OCT B-scan of the retina showing absence of any annotated retinal-edema lesion, including subretinal fluid and pigment epithelial detachment.	A retinal OCT B-scan of the retina showing an annotated retinal-edema lesion.
A clinical retinal OCT B-scan of the retina showing absence of any annotated retinal-edema lesion, including subretinal fluid and pigment epithelial detachment.	A clinical retinal OCT B-scan of the retina showing retinal edema.
A diagnostic retinal OCT B-scan of the retina demonstrating absence of any annotated retinal-edema lesion, including subretinal fluid and pigment epithelial detachment.	A diagnostic retinal OCT B-scan of the retina demonstrating an edematous retinal region.
This retinal OCT B-scan of the retina shows absence of any annotated retinal-edema lesion, including subretinal fluid and pigment epithelial detachment.	This retinal OCT B-scan of the retina shows edema-associated retinal thickening.
This retinal OCT B-scan of the retina demonstrates absence of any annotated retinal-edema lesion, including subretinal fluid and pigment epithelial detachment.	This retinal OCT B-scan of the retina demonstrates subretinal fluid.
The retina on this retinal OCT B-scan shows absence of any annotated retinal-edema lesion, including subretinal fluid and pigment epithelial detachment.	The retina on this retinal OCT B-scan shows a localized pocket of subretinal fluid.

Appendix

Table 7: Exact prompts for RESC.

Normal	Abnormal
A retinal OCT B-scan of the retina showing a disease-free retinal appearance with no choroidal neovascularization, diabetic macular edema, or drusen.	A retinal OCT B-scan of the retina showing choroidal neovascularization.
A clinical retinal OCT B-scan of the retina showing a disease-free retinal appearance with no choroidal neovascularization, diabetic macular edema, or drusen.	A clinical retinal OCT B-scan of the retina showing CNV-related neovascular change.
A diagnostic retinal OCT B-scan of the retina demonstrating a disease-free retinal appearance with no choroidal neovascularization, diabetic macular edema, or drusen.	A diagnostic retinal OCT B-scan of the retina demonstrating a choroidal neovascular membrane.
This retinal OCT B-scan of the retina shows a disease-free retinal appearance with no choroidal neovascularization, diabetic macular edema, or drusen.	This retinal OCT B-scan of the retina shows diabetic macular edema.
This retinal OCT B-scan of the retina demonstrates a disease-free retinal appearance with no choroidal neovascularization, diabetic macular edema, or drusen.	This retinal OCT B-scan of the retina demonstrates DME-related retinal thickening.
The retina on this retinal OCT B-scan shows a disease-free retinal appearance with no choroidal neovascularization, diabetic macular edema, or drusen.	The retina on this retinal OCT B-scan shows diabetic macular edema with intraretinal fluid.

Appendix

Table 8: Exact prompts for OCT2017.

Normal	Abnormal
A grayscale breast ultrasound image of breast tissue showing absence of any focal breast mass labeled benign or malignant.	A grayscale breast ultrasound image of breast tissue showing a focal breast mass.
A clinical grayscale breast ultrasound image of breast tissue showing absence of any focal breast mass labeled benign or malignant.	A clinical grayscale breast ultrasound image of breast tissue showing an annotated breast lesion.
A diagnostic grayscale breast ultrasound image of breast tissue demonstrating absence of any focal breast mass labeled benign or malignant.	A diagnostic grayscale breast ultrasound image of breast tissue demonstrating a localized breast nodule.
This grayscale breast ultrasound image of breast tissue shows absence of any focal breast mass labeled benign or malignant.	This grayscale breast ultrasound image of breast tissue shows a benign breast mass.
This grayscale breast ultrasound image of breast tissue demonstrates absence of any focal breast mass labeled benign or malignant.	This grayscale breast ultrasound image of breast tissue demonstrates a benign breast lesion.
Breast tissue on this grayscale breast ultrasound image shows absence of any focal breast mass labeled benign or malignant.	Breast tissue on this grayscale breast ultrasound image shows a benign breast nodule.

Appendix

Table 9: Exact prompts for BUSI.

Normal	Abnormal
A chest radiograph of the chest showing the No Finding state with none of the twelve evaluated disease observations.	A chest radiograph of the chest showing enlarged cardiomediastinum.
A clinical chest radiograph of the chest showing the No Finding state with none of the twelve evaluated disease observations.	A clinical chest radiograph of the chest showing cardiomegaly.
A diagnostic chest radiograph of the chest demonstrating the No Finding state with none of the twelve evaluated disease observations.	A diagnostic chest radiograph of the chest demonstrating lung opacity.
This chest radiograph of the chest shows the No Finding state with none of the twelve evaluated disease observations.	This chest radiograph of the chest shows a lung lesion.
This chest radiograph of the chest demonstrates the No Finding state with none of the twelve evaluated disease observations.	This chest radiograph of the chest demonstrates pulmonary edema.
The chest on this chest radiograph shows the No Finding state with none of the twelve evaluated disease observations.	The chest on this chest radiograph shows pulmonary consolidation.

Appendix

Table 10: Exact prompts for CheXpert.

Normal	Abnormal
A 2D brain MRI image of the brain showing absence of any visible brain tumor or intracranial mass.	A 2D brain MRI image of the brain showing a visible brain tumor.
A clinical 2D brain MRI image of the brain showing absence of any visible brain tumor or intracranial mass.	A clinical 2D brain MRI image of the brain showing an intracranial tumor.
A diagnostic 2D brain MRI image of the brain demonstrating absence of any visible brain tumor or intracranial mass.	A diagnostic 2D brain MRI image of the brain demonstrating a tumor-bearing brain MRI appearance.
This 2D brain MRI image of the brain shows absence of any visible brain tumor or intracranial mass.	This 2D brain MRI image of the brain shows a focal intracranial mass.
This 2D brain MRI image of the brain demonstrates absence of any visible brain tumor or intracranial mass.	This 2D brain MRI image of the brain demonstrates a neoplastic brain lesion.
The brain on this 2D brain MRI image shows absence of any visible brain tumor or intracranial mass.	The brain on this 2D brain MRI image shows a space-occupying brain lesion.

Appendix

Table 11: Exact prompts for Kaggle BrainMRI.

Normal state phrase	Abnormal state phrase
{}	damaged {}
flawless {}	broken {}
perfect {}	{} with flaw
unblemished {}	{} with defect
{} without flaw	{} with damage
{} without defect	␣disease {}

Appendix

Table 12: Exact state phrases used to construct every industrial prompt bank. The object placeholder {} is replaced by the noun in Table 14 . The leading space in the sixth abnormal phrase is retained from the reference implementation.

	Template
1	a cropped photo of the {}.
2	a cropped photo of a {}.
3	a close-up photo of a {}.
4	a close-up photo of the {}.
5	a bright photo of a {}.
6	a bright photo of the {}.

Appendix

Table 13: Exact photo templates used for the industrial prompt ensemble. The state placeholder {} is replaced by one of the state phrases in Table 12 .

Dataset	Object nouns
MVTec-AD	bottle; cable; capsule; carpet; grid; hazelnut; leather; metal nut; pill; screw; tile; toothbrush; transistor; wood; zipper
VisA	candle; capsules; cashew; chewing gum; fryum; macaroni 1; macaroni 2; printed circuit board 1; printed circuit board 2; printed circuit board 3; printed circuit board 4; pipe fryum

Appendix

Table 14: Exact object nouns substituted into the industrial prompt templates. Numbered VisA categories deliberately retain their index.

Quantity	BraTS 2021	Liver CT	RESC	OCT 2017	BUSI	CheXpert	Kaggle BrainMRI
$\rho_{\mathrm{correct}}$	$0.9255$	$0.9711$	$0.9524$	$0.9408$	$0.9771$	$0.9241$	$0.9742$
$c$	$0.2730$	$0.1700$	$0.2183$	$0.2432$	$0.1513$	$0.2755$	$0.1608$
$\lambda_{\mathrm{eff}}^{\mathrm{train}}$	$38.61$	$24.04$	$30.87$	$34.40$	$21.40$	$38.97$	$22.74$
$\lambda_{\mathrm{eff}}^{\mathrm{eval}}$	$7.72$	$4.81$	$6.17$	$6.88$	$4.28$	$7.79$	$4.55$

Appendix

Table 15: Correct-prompt geometry for the seven medical datasets. The reported values determine both the Gram-matched random construction and the rescaling of matched orthogonal anchors.

MVTec-AD					VisA
Category	$\rho_{t}$	$c_{t}$	$\lambda_{\mathrm{eff},t}^{\mathrm{train}}$	$\lambda_{\mathrm{eff},t}^{\mathrm{eval}}$	Category	$\rho_{t}$	$c_{t}$	$\lambda_{\mathrm{eff},t}^{\mathrm{train}}$	$\lambda_{\mathrm{eff},t}^{\mathrm{eval}}$
bottle	$0.9754$	$0.1567$	$22.17$	$4.43$	candle	$0.9743$	$0.1604$	$22.68$	$4.54$
cable	$0.9771$	$0.1514$	$21.41$	$4.28$	capsules	$0.9742$	$0.1605$	$22.70$	$4.54$
capsule	$0.9715$	$0.1688$	$23.87$	$4.77$	cashew	$0.9771$	$0.1514$	$21.41$	$4.28$
carpet	$0.9767$	$0.1526$	$21.58$	$4.32$	chewing gum	$0.9801$	$0.1410$	$19.95$	$3.99$
grid	$0.9735$	$0.1629$	$23.03$	$4.61$	fryum	$0.9701$	$0.1730$	$24.47$	$4.89$

Appendix

Table 16: Per-category correct-prompt geometry for the unified industrial detector. Each category’s shared- and independent-rotation controls preserve its displayed

\rho_{t}

c_{t}

, and

\lambda_{\mathrm{eff},t}

; independent rotations change only cross-category off-diagonal geometry. The participation-ratio effective rank of the correct-prompt difference Gram matrix

{\bm{M}}

(Section 6.4 ) is

r_{\mathrm{eff}}=1.79

for MVTec-AD and

1.78

for VisA. Shared rotations preserve

{\bm{M}}

and hence

r_{\mathrm{eff}}

; independent rotations raise it to

14.28

–

14.29

of 15 (MVTec-AD) and about

11.10

of 12 (VisA) across the three geometry seeds.

Family	Parameterization	Closed	Reason
linear	${\bm{y}}=\widetilde{\bm{W}}^{\top}\tilde{\bm{z}}$ , $\widetilde{\bm{W}}$ free	yes	absorb ${\bm{Q}}$ into $\widetilde{\bm{W}}$
MLP	${\bm{y}}={\bm{W}}_{2}\,\mathrm{GELU}({\bm{W}}_{1}{\bm{z}})$	yes	absorb ${\bm{Q}}$ into ${\bm{W}}_{2}$
Transformer	${\bm{y}}={\bm{U}}_{\mathrm{out}}T({\bm{U}}_{\mathrm{in}}Z+E_{\mathrm{pos}})$	yes	absorb ${\bm{Q}}$ into ${\bm{U}}_{\mathrm{out}}$

Appendix

Table 18: Rotation closure of the three adapter families.

Multimodal Anomaly Detection

Multimodal anomaly detection identifies unusual patterns by combining complementary signals such as images, depth, video, text, and sensor data.

See all 97 papers

VisualAD: Language-Free Zero-Shot Anomaly Detection via Vision Transformer
Mar 9, 2026Yanning Hou, Peiyuan Li, Zirui Liu +4Zero-Shot Anomaly Detection Vision Foundation Model Adaptation
College of Intelligence Science and Technology, National University of Defense Technology, Changsha, China · School of Artificial Intelligence, Anhui University, Hefei, China · State Key Laboratory of Opto-Electronic Information Acquisition and Protection Technology, Anhui University, Hefei, China
Proximity-CLIP: Text-Guided Semantic Proximity Learning for Zero-Shot Anomaly Detection
Sep 7, 2026Manwen Yang, Leqian Ding, Yu Guo +1VLM Adaptation Anomaly Localization
School of Software Engineering, Xi’an Jiaotong University · State Key Laboratory of Human-Machine Hybrid Augmented Intelligence, Institute of Artificial Intelligence and Robotics, Xi’an Jiaotong University
Anchor and Adapt: Asymmetric Prompt Adaptation for Few-Shot Industrial Anomaly Detection
Oct 4, 2026Mengyang Zhao, Teng Fu, Haiyang Yu +3Prompt Learning for VLMs Few-Shot Anomaly Detection
Shanghai Key Lab of Intelligent Information Processing and College of Computer Science and Artificial Intelligence, Fudan University, Shanghai 200433, China

Mar 9, 2026J/K move · Enter open · S save

VisualAD: Language-Free Zero-Shot Anomaly Detection via Vision Transformer

Yanning Hou, Peiyuan Li, Zirui Liu, Yitong Wang+3

College of Intelligence Science and Technology, National University of Defense Technology, Changsha, China · School of Artificial Intelligence, Anhui University, Hefei, China · State Key Laboratory of Opto-Electronic Information Acquisition and Protection Technology, Anhui University, Hefei, China

Zero-Shot Anomaly Detection Vision Foundation Model Adaptation Medical Image Anomaly Detection

Open paper AskPDF arXiv

Zero-shot anomaly detection (ZSAD) requires detecting and localizing anomalies without access to target-class anomaly samples. Mainstream methods rely on vision-language models (VLMs) such as CLIP: they build hand-crafted or learned prompt sets for normal and abnormal semantics, then compute image-text similarities for open-set discrimination. While effective, this paradigm depends on a text encoder and cross-modal alignment, which can lead to training instability and parameter redundancy. This work revisits the necessity of the text branch in ZSAD and presents VisualAD, a purely visual framework built on Vision Transformers. We introduce two learnable tokens within a frozen backbone to directly encode normality and abnormality. Through multi-layer self-attention, these tokens interact with patch tokens, gradually acquiring high-level notions of normality and anomaly while guiding patches to highlight anomaly-related cues. Additionally, we incorporate a Spatial-Aware Cross-Attention (SCA) module and a lightweight Self-Alignment Function (SAF): SCA injects fine-grained spatial information into the tokens, and SAF recalibrates patch features before anomaly scoring. VisualAD achieves state-of-the-art performance on 13 zero-shot anomaly detection benchmarks spanning industrial and medical domains, and adapts seamlessly to pretrained vision backbones such as the CLIP image encoder and DINOv2. Code: https://github.com/7HHHHH/VisualAD

Abstract

Multimodal Anomaly Detection

Explore similar work

VisualAD: Language-Free Zero-Shot Anomaly Detection via Vision Transformer

Proximity-CLIP: Text-Guided Semantic Proximity Learning for Zero-Shot Anomaly Detection

Anchor and Adapt: Asymmetric Prompt Adaptation for Few-Shot Industrial Anomaly Detection