ProGRank: Probe-Gradient Reranking to Defend Dense-Retriever RAG from Corpus Poisoning

Xiangyu Yin; Yi Qi; Chih-Hong Cheng

cs.AIMar 24, 2026

ProGRank: Probe-Gradient Reranking to Defend Dense-Retriever RAG from Corpus Poisoning

Authors: Xiangyu Yin, Yi Qi, Chih-Hong Cheng

Organizations: Chalmers University of Technology, Sweden · University of Leeds, United Kingdom · Carl von Ossietzky University of Oldenburg, Germany

Abstract

Retrieval-Augmented Generation (RAG) improves large language model applications by grounding generation in retrieved evidence, but also introduces corpus poisoning as a new attack surface. In this setting, an adversary injects or edits passages so that they enter the Top- $K$ results for target queries and influence downstream generation. Existing defences often rely on content filtering, auxiliary models, or generator-side reasoning, which complicates deployment. We propose ProGRank, a post hoc, training-free retriever-side defence for dense-retriever RAG. ProGRank stress-tests each query--passage pair under mild randomized perturbations, extracts probe gradients from a small fixed parameter subset, and derives two instability signals: representational consistency and dispersion risk. It then combines these signals with a score gate for reranking. ProGRank preserves the original passage content, requires no retraining, and supports a surrogate-based variant when the deployed retriever is unavailable. Experiments across datasets, retrievers, attacks, and retrieval-stage and end-to-end settings show that ProGRank improves robustness and maintains a favorable robustness--utility trade-off, including under adaptive evasive attacks.

Figures & tables

Figure 1: Undefended ranking versus ProGRank. ProGRank derives consistency and dispersion-risk penalties from randomized probe gradients and applies them through a score gate to suppress poisoned Top- $K$ exposure.

Method	Post hoc and training-free	No auxiliary LM inference	Preserves original passage content	No generator-side calls
GRADA [ 39 ]	✓	✓	✓	✓
RAGPart [ 18 ]	✓	✓	✗	✓
RAGMask [ 18 ]	✓	✓	✗	✓
GMTP [ 12 ]	✓	✗	✗	✓
RAGuard [ 13 ]	✗	✗	✗	✗
ProGRank (ours)	✓	✓	✓	✓

Table 1: Comparison of representative defences against corpus poisoning in dense-retriever RAG. Columns highlight deployment properties relevant to our positioning.

Figure 2: Overall retrieval-stage results, comparing Poison Hit Rate and Poison Recall Rate under $s_{\theta}(q,p)$ and $\tilde{s}_{R,\theta,\vartheta}(q,p)$ .

Figure 3: Distribution of reranking signals for clean and poisoned passages. From left to right: $P_{\mathrm{dr}}^{R,\theta,\vartheta}(q,p)$ , $P_{\mathrm{rep}}^{R,\theta,\vartheta}(q,p)$ , the applied score correction $\tilde{s}_{R,\theta,\vartheta}(q,p)-s_{\theta}(q,p)$ , and the gate value $w_{\theta}(q,p)$ .

Figure 4: Rank shift induced by ProGRank. Upward and downward rank shifts are reported separately for poisoned and clean passages as Poison Up , Poison Down , Clean Up , and Clean Down . We use $P\downarrow$ and $C\downarrow$ as shorthand for Poison Down and Clean Down , respectively.

Figure 5: Ablation of the reranking objective. We compare the full score in Eq. ( 13 ) against ablated variants obtained by removing $w_{\theta}(q,p)$ , $P_{\mathrm{dr}}^{R,\theta,\vartheta}(q,p)$ , $P_{\mathrm{rep}}^{R,\theta,\vartheta}(q,p)$ , and their combinations.

Figure 6: Ablation summary across four factors: perturbation repeat $R$ , probe layer, perturbation type, and retriever backbone.

Dataset	Poison Method	Baseline	GRADA	GMTP	RAGuard	Ours
HotpotQA	PoisonedRAG	0.940 $\pm$ 0.239 0.960 $\pm$ 0.197 0.010 $\pm$ 0.100	0.080 $\pm$ 0.273 0.080 $\pm$ 0.273 0.230 $\pm$ 0.423	0.090 $\pm$ 0.288 0.030 $\pm$ 0.171 0.100 $\pm$ 0.302	0.350 $\pm$ 0.479 0.360 $\pm$ 0.480 0.330 $\pm$ 0.473	0.032 $\pm$ 0.316 0.000 $\pm$ 0.000 0.300 $\pm$ 0.483
	LIAR-RAG	0.739 $\pm$ 0.449 0.826 $\pm$ 0.388 0.000 $\pm$ 0.000	0.000 $\pm$ 0.000 0.043 $\pm$ 0.209 0.391 $\pm$ 0.499	0.000 $\pm$ 0.000 0.000 $\pm$ 0.000 0.261 $\pm$ 0.449	0.348 $\pm$ 0.487 0.340 $\pm$ 0.485 0.435 $\pm$ 0.507	0.000 $\pm$ 0.000 0.000 $\pm$ 0.000 0.400 $\pm$ 0.516
	Joint-GCG	0.966 $\pm$ 0.183 0.949 $\pm$ 0.222 0.017 $\pm$ 0.130	0.085 $\pm$ 0.281 0.119 $\pm$ 0.326 0.237 $\pm$ 0.429	0.102 $\pm$ 0.305 0.034 $\pm$ 0.183 0.136 $\pm$ 0.345	0.441 $\pm$ 0.501 0.430 $\pm$ 0.498 0.288 $\pm$ 0.457	0.000 $\pm$ 0.000 0.000 $\pm$ 0.000 0.469 $\pm$ 0.500
	Macro Avg	0.882 $\pm$ 0.290 0.912 $\pm$ 0.269 0.009 $\pm$ 0.077	0.055 $\pm$ 0.185 0.081 $\pm$ 0.269 0.286 $\pm$ 0.450	0.064 $\pm$ 0.198 0.021 $\pm$ 0.118 0.166 $\pm$ 0.365	0.380 $\pm$ 0.489 0.377 $\pm$ 0.488 0.351 $\pm$ 0.479	0.011 $\pm$ 0.105 0.000 $\pm$ 0.000 0.390 $\pm$ 0.500
NQ	PoisonedRAG	0.612 $\pm$ 0.490 0.653 $\pm$ 0.478 0.286 $\pm$ 0.454	0.031 $\pm$ 0.173 0.061 $\pm$ 0.241 0.531 $\pm$ 0.502	0.020 $\pm$ 0.142 0.092 $\pm$ 0.290 0.255 $\pm$ 0.438	0.276 $\pm$ 0.449 0.290 $\pm$ 0.454 0.367 $\pm$ 0.485	0.000 $\pm$ 0.000 0.000 $\pm$ 0.000 0.596 $\pm$ 0.422
	LIAR-RAG	0.522 $\pm$ 0.505 0.696 $\pm$ 0.465 0.130 $\pm$ 0.341	0.022 $\pm$ 0.147 0.043 $\pm$ 0.206 0.630 $\pm$ 0.488	0.000 $\pm$ 0.000 0.087 $\pm$ 0.285 0.326 $\pm$ 0.474	0.261 $\pm$ 0.444 0.270 $\pm$ 0.448 0.500 $\pm$ 0.506	0.000 $\pm$ 0.000 0.000 $\pm$ 0.000 0.200 $\pm$ 0.422

Table 2: Downstream generation results at Top-

5

. Each cell reports substring-based ASR (top), LLM-judged ASR (middle), and substring-based ACC (bottom). For each row, the lowest substring-based ASR and LLM-judged ASR, as well as the highest ACC, are highlighted in bold .

Dataset	PoisonedRAG	LIAR-RAG	Joint-GCG	Macro Avg
HotpotQA	0.060 $\pm$ 0.287 0.018 $\pm$ 0.012 0.284 $\pm$ 0.465	0.026 $\pm$ 0.009 0.008 $\pm$ 0.011 0.381 $\pm$ 0.497	0.028 $\pm$ 0.015 0.029 $\pm$ 0.010 0.451 $\pm$ 0.481	0.030 $\pm$ 0.118 0.016 $\pm$ 0.013 0.372 $\pm$ 0.476
NQ	0.024 $\pm$ 0.010 0.030 $\pm$ 0.014 0.579 $\pm$ 0.403	0.012 $\pm$ 0.008 0.022 $\pm$ 0.012 0.186 $\pm$ 0.405	0.030 $\pm$ 0.011 0.024 $\pm$ 0.016 0.333 $\pm$ 0.401	0.016 $\pm$ 0.009 0.034 $\pm$ 0.015 0.365 $\pm$ 0.398
MS MARCO	0.028 $\pm$ 0.013 0.011 $\pm$ 0.010 0.382 $\pm$ 0.492	0.017 $\pm$ 0.301 0.010 $\pm$ 0.014 0.383 $\pm$ 0.499	0.027 $\pm$ 0.012 0.034 $\pm$ 0.017 0.521 $\pm$ 0.487	0.049 $\pm$ 0.097 0.019 $\pm$ 0.012 0.429 $\pm$ 0.503

Table 3: Downstream generation results of ProGRank under ProGRank-based evasive attacks.

Method	HotpotQA (s/query)	NQ (s/query)	MS MARCO (s/query)	Mean (s/query)
GRADA	0.91	1.14	0.63	0.90
GMTP	14.71	17.82	11.25	14.60
RAGuard	110.17	148.74	95.61	118.17
Ours	4.68	4.92	4.58	4.73

Table 4: Measured end-to-end latency (seconds per query).

Dataset (Metric)	Baseline	GRADA	GMTP	RAGuard	Ours
HotpotQA (F1)	0.440 $\pm$ 0.026	0.251 $\pm$ 0.130	0.300 $\pm$ 0.078	0.431 $\pm$ 0.013	0.304 $\pm$ 0.009
NQ (EM)	0.175 $\pm$ 0.035	0.150 $\pm$ 0.070	0.100 $\pm$ 0.000	0.175 $\pm$ 0.035	0.225 $\pm$ 0.106
MS MARCO (ROUGE-L F1)	0.235 $\pm$ 0.017	0.251 $\pm$ 0.045	0.245 $\pm$ 0.012	0.235 $\pm$ 0.018	0.202 $\pm$ 0.101

Table 5: Clean utility reported by F1 (

\uparrow

), EM (

\uparrow

), ROUGE-L F1 (

\uparrow

). The highest value is highlighted in bold for each row.

RAG Security

Retrieval-augmented generation security studies how to protect retrieval-based language systems from attacks, data leakage, and unsafe outputs.

See all 104 papers

PRA-RAG: Provably Robust Aggregation in Retrieval-Augmented Generation against Retrieval Corruption
May 8, 2026Xue Tan, Yi Zheng, Chang Huo +7Retrieval-Augmented Generation RAG Poisoning Attacks
School of Computer Science, Fudan University, Shanghai, China · Institute of Big Data, Fudan University, Shanghai, China · Department of Computer Science, Worcester Polytechnic Institute, MA, USA
Semantic Chameleon: Corpus-Dependent Poisoning Attacks and Defenses in RAG Systems
Mar 10, 2026Scott ThorntonRAG Poisoning Attacks RAG Security
Independent Researcher
RAGuard: A Layered Defense Framework for Retrieval-Augmented Generation Systems Against Data Poisoning
Jul 28, 2026Pushkal Kumar, Tucker Nielson, Tanish Kolhe +2RAG Poisoning Attacks RAG Security

May 8, 2026J/K move · Enter open · S save

PRA-RAG: Provably Robust Aggregation in Retrieval-Augmented Generation against Retrieval Corruption

Xue Tan, Yi Zheng, Chang Huo, Yunruo Zhang+6

School of Computer Science, Fudan University, Shanghai, China · Institute of Big Data, Fudan University, Shanghai, China · Department of Computer Science, Worcester Polytechnic Institute, MA, USA

Retrieval-Augmented Generation RAG Poisoning Attacks RAG Security Certified Robustness RAG Robustness

Open paper AskPDF arXiv

Retrieval-Augmented Generation (RAG) enhances Large Language Models (LLMs) by incorporating external knowledge, effectively mitigating their inherent knowledge limitations. However, RAG remains vulnerable to poisoning attacks that manipulate retrieved texts to mislead model outputs. Existing defense mechanisms often lack theoretical robustness guarantees and perform unreliably when the LLM has limited knowledge of the retrieved content. In this work, we propose PRA-RAG, a provably robust retrieval aggregation algorithm designed to defend against poisoning attacks on retrieved texts. PRA-RAG samples multiple combinations of retrieved texts and utilizes geometric structures in the embedding space to identify a robust subset, from which a stable aggregated representation is derived. We provide theoretical bounds on the maximum impact of poisoned retrieved content and establish a quantitative measure of RAG's robustness. Experiments across multiple benchmarks and RAG architectures demonstrate that PRA-RAG reduces the attack success rate to as low as 1% while maintaining an accuracy of 71%, significantly outperforming representative state-of-the-art methods.

Abstract

RAG Security

Explore similar work

PRA-RAG: Provably Robust Aggregation in Retrieval-Augmented Generation against Retrieval Corruption

Semantic Chameleon: Corpus-Dependent Poisoning Attacks and Defenses in RAG Systems

RAGuard: A Layered Defense Framework for Retrieval-Augmented Generation Systems Against Data Poisoning