Retrieval-Augmented Generation (RAG) systems are essential to contemporary AI applications, allowing large language models to obtain external knowledge via vector similarity search. Nevertheless, these systems encounter a significant security flaw: hubness - items that frequently appear in the top-k retrieval results for a disproportionately high number of varied queries. These hubs can be exploited to introduce harmful content, alter search rankings, bypass content filtering, and decrease system performance. We introduce hubscan, an open-source security scanner that evaluates vector indices and embeddings to identify hubs in RAG systems. Hubscan presents a multi-detector architecture that integrates: (1) robust statistical hubness detection utilizing median/Median Absolute Deviation (MAD)-based z-scores, (2) cluster spread analysis to assess cross-cluster retrieval patterns, (3) stability testing under query perturbations, and (4) domain-aware and modality-aware detection for category-specific and cross-modal attacks. Our solution accommodates several vector databases (FAISS, Pinecone, Qdrant, Weaviate) and offers versatile retrieval techniques, including vector similarity, hybrid search, and lexical matching with reranking capabilities. We evaluate hubscan on Food-101, MS-COCO, and FiQA adversarial hubness benchmarks constructed using state-of-the-art gradient-optimized and centroid-based hub generation methods. Hubscan achieves 90% recall at a 0.2% alert budget and 100% recall at 0.4%, with adversarial hubs ranking above the 99.8th percentile. In testing, domain-scoped scanning recovered 100% of targeted attacks that evaded global detection. Production validation on 1M real web documents from MS MARCO demonstrates significant score separation between clean documents and adversarial content.
Figures & tables
Fig. 1 : Adversarial Hubness Detector detection pipeline overview showing the multi-stage process from input to verdict assignment.
Fig. 2 : Key detection metrics and their interpretation: Hub z-score measures statistical anomaly, cluster entropy captures cross-cluster spread, stability indicates robustness to perturbations, and combined scores provide holistic risk assessment.
Fig. 3 : Three detection modes: Global detection analyzes all queries together, Domain-Aware detection groups queries by semantic domains, and Modality-Aware detection handles cross-modal attacks.
Fig. 4 : Cross-modal hub detection: A hub (red star) positioned at the intersection of text and image modalities appears in top- k results for queries from both modalities, exploiting the modality boundary.
Fig. 5 : Hubness score distribution for normal documents vs. planted adversarial hubs on Food-101 and MS-COCO. Normal items cluster near zero while adversarial hubs are extreme outliers (z-score > 20). At a 0.1% alert budget, all planted hubs rank above the 99.8th percentile, achieving 100% recall with minimal false positives.
Benchmark
Docs
Queries
Hubs
Dim
Domains
Food-101 [ 2 ]
5,010
25,000
10
512
101
MS-COCO [ 7 ]
5,008
25,000
10
512
40
FiQA [ 9 ]
5,005
648
5
1024
—
TABLE I : Evaluation Benchmark Statistics
Dataset
Attack
Budget
K
Prec.
Recall
Food-101
Universal
0.2%
10
90%
90%
Universal
0.4%
20
50%
100%
MS-COCO
Universal
0.2%
10
90%
90%
Universal
0.4%
20
50%
100%
FiQA
Universal
0.1%
5
100%
100%
TABLE II : Detection Performance (Full Configuration)
Domain
≤ 2%
5%
10%
20%
30%
Food-101
1.00
1.00
1.00
0.99
0.96
MS-COCO
1.00
1.00
1.00
1.00
1.00
FiQA
1.00
0.99
0.95
0.87
0.80
CodeSearchNet
1.00
0.92
0.76
0.67
0.62
PubMed
1.00
0.96
0.87
0.75
0.69
Average
1.00
0.97
0.92
0.86
0.82
TABLE III : AUC-ROC by Adversarial Corpus Fraction (Hubness Z-Score Detector)
Metric
Clean MS MARCO
Adversarial
Separation
99 th pct.
2.3
13–17
5.8 ×
99.9 th pct.
5.0
13–17
2.7 ×
TABLE IV : Score Distribution: Clean vs. Adversarial (1M Scale)
Faculty of Information Technology, University of Science, Ho Chi Minh City, Vietnam · Vietnam National University, Ho Chi Minh City, Vietnam · Virginia Tech +6