cs.CVMar 20, 2026
SaveFB-CLIP: Fine-Grained Zero-Shot Anomaly Detection with Foreground-Background Disentanglement
Organizations: Xi’an Institute of Optics and Precision Mechanics, Chinese Academy of Sciences · University of Chinese Academy of Sciences · Xi’an Jiaotong University · Zhongnan Hospital of Wuhan University
Abstract
Fine-grained anomaly detection is crucial in industrial and medical applications, but labeled anomalies are often scarce, making zero-shot detection challenging. While vision-language models like CLIP offer promising solutions, they struggle with foreground-background feature entanglement and coarse textual semantics. We propose FB-CLIP, a framework that enhances anomaly localization via multi-strategy textual representations and foreground-background separation. In the textual modality, it combines End-of-Text features, global-pooled representations, and attention-weighted token features for richer semantic cues. In the visual modality, multi-view soft separation along identity, semantic, and spatial dimensions, together with background suppression, reduces interference and improves discriminability. Semantic Consistency Regularization (SCR) aligns image features with normal and abnormal textual prototypes, suppressing uncertain matches and enlarging semantic gaps. Experiments show that FB-CLIP effectively distinguishes anomalies from complex backgrounds, achieving accurate fine-grained anomaly detection and localization under zero-shot settings.
Figures & tables
Figure 1: Framework comparison. (a) Existing methods use a single text feature for uniform alignment, leading to coarse semantics. (b) FB-CLIP employs multi-text features and Semantic Consistency Regularization (SCR) to enhance alignment, while foreground–background separation and background suppression enable fine-grained, noise-robust anomaly representation.
Figure 2: (a) Visualization of the responses from the original CLIP model, corresponding to Fig. 1(c) in AnomalyCLIP [ 66 ] . (b) Entanglement of foreground, background, and anomalous regions. Separating foreground and background clarifies the distinction between anomalous and normal points, improving accuracy.
Figure 3: Overview of FB-CLIP for zero-shot anomaly detection. (a) Multi-Strategy Text Feature Fusion (MSTFF) generates task-aware text embeddings (see Section 3.1 ); (b) Multi-View Foreground-Background Enhancement (MVFBE) separates and enhances foreground and background features (see Section 3.2 ); (c) Background Suppression reduces residual background interference (see Section 3.3 ); (d) Semantic Consistency Regularization (SCR) enforces confident and discriminative visual-text alignment. These components jointly enable fine-grained anomaly detection without labeled abnormal samples (see Section 3.4 ).
Figure 4: Structure of the proposed MVFBE module. The module first generates a soft foreground mask from multiple anomaly indicators. Three complementary views are applied: Identity (ID) preserves original features, Semantic (SEM) models foreground richness and background stability, and Spatial (SPA) captures fine-grained local structures. Features are then integrated across layers and refined with foreground-background attention to produce discriminative anomaly representations. Details in Appendix A , B
| Domain | Metric | Dataset | WinCLIP [ 27 ] | VAND [ 8 ] | AnomalyCLIP [ 66 ] | AdaCLIP [ 6 ] | AA-CLIP [ 37 ] | AF-CLIP [ 14 ] | FAPrompt [ 67 ] | FB-CLIP |
| CVPR’2023 | CVPR’2023 | ICLR’2024 | ECCV’2024 | CVPR’2025 | MM’2025 | ICCV’2025 | Ours | |||
| Industrial | image-level (AUROC, AP) | MVTec | (91.8, 96.5) | (86.1, 93.5) | (91.5, 96.2) | (92.1, 96.7 ) | (90.5, 94.9) | ( 92.9 , 96.8 ) | (91.9, 95.7) | ( 92.4 ,96.6) |
| VisA | (78.1, 81.2) | (78.0, 81.4) | (82.1, 85.4) | (86.3, 89.2) | (84.6, 82.2) | ( 88.5 , 90.0 ) | (84.6, 86.8) | ( 89.5 , 90.7 ) | ||
| MPDD | (63.6, 69.9) | (78.0, 81.4) | (77.0, 82.0) | (76.8, -) | (-, -) | (-, -) | ( 80.1 , 83.9 ) | ( 79.1 , 86.2 ) | ||
| BTAD | (68.2, 70.9) | (73.5, 68.6) | (88.3, 87.3) | (92.4, 96.2) | ( 93.8 , 97.9 ) | ( 94.3 , 95.2) | (92.2, 92.5) | (93.2, 96.4 ) | ||
| DAGM | (91.8, 79.5) | (94.4, 83.8) | (97.5, 92.3) | (98.3, 93.7) | (93.9, 84.5) | (98.7, 94.6) | ( 98.8 , 95.3 ) | ( 99.0 , 96.5 ) |
Table 1: Zero-shot anomaly detection performance on industrial and medical domains. The image-level performance AUROC(%), AP(%) and pixel-level performance AUROC(%), PRO(%) are reported. Red: best , blue: second best.
| Method | MVTec | VisA | ||||||
| Base | MSTFF | MVFBE | BS | SCR | Image (AUROC, AP) | Pixel (AUROC, PRO) | Image (AUROC, AP) | Pixel (AUROC, PRO) |
| ✓ | (67.1, 84.0) | (65.2, 32.9) | (45.7, 56.2) | (88.4, 71.8) | ||||
| ✓ | ✓ | (82.0, 91.4) | (84.0, 68.0) | (78.3, 80.5) | (93.2, 81.4) | |||
| ✓ | ✓ | (92.2, 96.5) | (91.7, 84.5) | (88.4, 89.7) | (96.0, 90.8) | |||
| ✓ | ✓ | (44.8, 72.7) | (70.7, 38.3) | (48.3, 56.6) | (84.1, 58.3) | |||
| ✓ | ✓ | (65.4, 81.3) | (75.7, 45.5) | (63.8, 69.0) | (88.2, 65.5) | |||
Table 2: Ablation study of different modules for zero-shot anomaly detection. Deeper colors correspond to higher average performance. The columns MSTFF , MVFBE , BS , and SCR correspond to the four improvement strategies proposed in Figure 3 .
Figure 5: Overall visualization of four image sets demonstrating the effects of the proposed modules. (a) Multi-View Foreground-Background Enhancement (MVFBE) results, highlighting the enhancement of tokens across multiple views. (b) Background Subtraction (BS) process, showing how irrelevant background tokens are removed to emphasize anomalies. Together, these visualizations illustrate the progressive improvement in anomaly representation and localization.
Figure 6: Generation process of the foreground–background mask. Multiple complementary anomaly indicators are first aggregated to estimate the foreground–background probability of each token.
Figure 7: Visualization results of anomaly localization across multiple categories using different CLIP-based methods.
| Strategies | MVTec | VisA | ||||
|---|---|---|---|---|---|---|
| EOT | GP | Attn | Image-level (AUROC, AP) | Pixel-level (AUROC, PRO) | Image-level (AUROC, AP) | Pixel-level (AUROC, PRO) |
| ✓ | (67.1, 84.0) | (65.2, 32.9) | (45.7, 56.2) | (88.4, 71.8) | ||
| ✓ | (62.6, 81.9) | (63.3, 31.9) | (51.3, 56.6) | (83.7, 67.5) | ||
| ✓ | (80.9, 90.8) | (79.0, 55.7) | (78.5, 80.7 ) | (92.7, 80.3) | ||
| ✓ | ✓ | (81.1, 90.9) | (83.1, 66.4) | (78.6, 80.7) | (92.7, 80.4) | |
| ✓ | ✓ | ✓ | (82.0, 91.4) | (84.0, 68.0) | (78.3, 80.5) | (93.2, 81.4) |
Table 3: Ablation study on Multi-Strategy Text Feature Fusion (MSTFF) across two datasets ( MVTec and VisA ). EOT: End-of-Text; GP: Global Pooling; Attn: Attention-weighted.
| Method | MVTec | VisA | |||||
|---|---|---|---|---|---|---|---|
| ID | SEM | SPA | FB-Attn | Img (AUROC, AP) | Pix (AUROC, PRO) | Img (AUROC, AP) | Pix (AUROC, PRO) |
| ✓ | (82.0, 91.4) | (84.0, 68.0) | (78.3, 80.5) | (93.2, 81.4) | |||
| ✓ | (83.0, 92.2) | (77.3, 48.6) | (80.8, 82.9) | (92.1, 77.1) | |||
| ✓ | (83.0, 91.9) | (80.8, 55.6) | (79.3, 81.7) | (91.0, 73.9) | |||
| ✓ | ✓ | (81.4, 91.1) | (84.8, 67.1) | (79.1, 81.5) | (93.4, 81.7) | ||
| ✓ | ✓ | (82.0, 91.3) | (83.1, 64.3) | (77.8, 80.5) | (93.3, 80.9) | ||
Table 4: Ablation study with modular configuration using four components: ID , SEM , SPA and FB-Attention ,. A checkmark (✓) indicates that the component is enabled.
| MVTec | VisA | |||
|---|---|---|---|---|
| I-(AUROC, AP) | P-(AUROC, PRO) | I-(AUROC, AP) | P-(AUROC, PRO) | |
| L | (92.4, 96.6) | (91.8, 85.4) | (89.3, 90.5) | (96.3, 91.4) |
| L/2 | (92.4, 96.6) | (91.9, 85.7) | (89.5, 90.7) | (96.3, 91.4) |
Table 5: Ablation study on the number of tokens for background representation. Performance is reported on MVTec and VisA datasets, including image-level (I) AUROC and AP, as well as pixel-level (P) AUROC and PRO metrics.
| Objects | Pixel | Image | ||
|---|---|---|---|---|
| AUROC | AUPRO | AUROC | AP | |
| Class1 | 91.8 | 82.1 | 94.1 | 82.1 |
| Class2 | 99.7 | 99.0 | 99.9 | 99.8 |
| Class3 | 97.0 | 96.3 | 100.0 | 100.0 |
| Class4 | 94.0 | 80.5 | 99.5 | 97.1 |
| Class5 | 99.3 | 97.6 | 100.0 | 99.9 |
Table 6: Performance on the DAGM_KAGGLEUPLOAD dataset.
| Objects | Pixel | Image | ||
|---|---|---|---|---|
| AUROC | AUPRO | AUROC | AP | |
| bracket_black | 97.9 | 93.0 | 79.0 | 88.5 |
| bracket_brown | 94.1 | 86.4 | 57.8 | 73.9 |
| bracket_white | 99.6 | 97.5 | 91.2 | 90.6 |
| connector | 96.9 | 88.6 | 83.6 | 79.1 |
| metal_plate | 94.4 | 87.0 | 66.9 | 86.8 |
Table 7: Performance on the MPDD dataset (class-wise results).
| Objects | Pixel | Image | ||
|---|---|---|---|---|
| AUROC | AUPRO | AUROC | AP | |
| 01 | 94.5 | 78.9 | 95.4 | 98.4 |
| 02 | 95.4 | 66.3 | 85.5 | 97.7 |
| 03 | 97.5 | 94.9 | 98.6 | 93.1 |
| Mean | 95.8 | 80.0 | 93.2 | 96.4 |
Table 8: Performance on the BTAD dataset (class-wise results).
| Objects | Pixel | Image | ||
|---|---|---|---|---|
| AUROC | AUPRO | AUROC | AP | |
| Blotchy_099 | 99.2 | 95.3 | 99.9 | 100.0 |
| Fibrous_183 | 99.4 | 97.7 | 100.0 | 100.0 |
| Marbled_078 | 99.0 | 96.0 | 99.6 | 99.9 |
| Matted_069 | 98.2 | 83.1 | 97.2 | 99.3 |
| Mesh_114 | 97.3 | 86.7 | 92.0 | 96.8 |
Table 9: Performance on the DTD dataset (class-wise results).
| Objects | Pixel | Image | ||
|---|---|---|---|---|
| AUROC | AUPRO | AUROC | AP | |
| bottle | 93.4 | 86.7 | 96.3 | 98.8 |
| cable | 79.4 | 69.2 | 73.9 | 84.1 |
| capsule | 96.2 | 91.8 | 93.3 | 98.6 |
| carpet | 99.4 | 94.6 | 100.0 | 100.0 |
| grid | 97.6 | 80.7 | 99.0 | 99.7 |
Table 10: Pixel- and image-level anomaly detection performance on MVTec dataset using FB-CLIP.
| Objects | Pixel | Image | ||
|---|---|---|---|---|
| AUROC | AUPRO | AUROC | AP | |
| candle | 98.8 | 96.3 | 92.1 | 93.5 |
| capsules | 97.3 | 91.0 | 94.2 | 97.0 |
| cashew | 96.4 | 97.1 | 94.3 | 97.5 |
| chewinggum | 99.4 | 94.0 | 98.6 | 99.4 |
| fryum | 95.9 | 93.6 | 97.0 | 98.7 |
Table 11: Cross-domain zero-shot performance from MVTec to VISA dataset.
| Objects | Pixel | Image | ||
|---|---|---|---|---|
| AUROC | AUPRO | AUROC | AP | |
| pcb | 95.8 | 82.0 | 69.2 | 76.9 |
| phone_battery | 77.7 | 94.9 | 85.1 | 84.1 |
| sim_card_set | 99.8 | 98.5 | 95.6 | 96.3 |
| switch | 88.4 | 79.5 | 69.4 | 75.0 |
| terminalblock | 97.9 | 93.1 | 86.2 | 88.8 |
Table 12: Cross-domain zero-shot performance on Real-IAD dataset.
| Infer Metric | AF-CLIP | FAPrompt | FB-CLIP |
|---|---|---|---|
| Time | 112 ms | 232 ms | 215 ms |
| Memory | 2.5G | 2.6G | 3.3G |
Table 13: Comparison with recent SOTA methods in the inference process.
Figure 8: Visualization results of inaccurate anomaly localization using the FB-CLIP method.
Figure 9: Visualization of the same object from different viewpoints in the Real-IAD dataset.
More in this topic
Zero-Shot Anomaly Detection
Zero-shot anomaly detection finds unusual patterns in unseen domains without task-specific labeled examples.
See all 85 papers