Large Language Models (LLMs) have demonstrated remarkable performance across a wide range of applications. However, their practical deployment is often hindered by issues such as outdated knowledge and the tendency to generate hallucinations. To address these limitations, Retrieval-Augmented Generation (RAG) systems have been introduced, enhancing LLMs with external, up-to-date knowledge sources. Despite their advantages, RAG systems remain vulnerable to adversarial attacks, with data poisoning emerging as a prominent threat. Existing poisoning-based attacks typically require prior knowledge of the user's specific queries, limiting their flexibility and real-world applicability. In this work, we propose PIDP-Attack, a novel compound attack that integrates prompt injection with database poisoning in RAG. By appending malicious characters to queries at inference time and injecting a limited number of poisoned passages into the retrieval database, our method can effectively manipulate LLM response to arbitrary query without prior knowledge of the user's actual query. Experimental evaluations across three benchmark datasets (Natural Questions, HotpotQA, MS-MARCO) and eight LLMs demonstrate that PIDP-Attack consistently outperforms the original PoisonedRAG. Specifically, our method improves attack success rates by 4% to 16% on open-domain QA tasks while maintaining high retrieval precision, proving that the compound attack strategy is both necessary and highly effective.
Figures & tables
Property
Corpus
GCG
Clean-RAG
GGPP
Disinformation
PoisonedRAG
PR-Attack
PIDP
Query-path manipulation
✗
✓
✓
✓
✗
✗
✓
✓
Corpus-path manipulation
✓
✗
✗
✗
✓
✓
✓
✓
Unaware of user query
✗
✓
✓
✗
✗
✗
✗
✓
Retrieval steering
✓
✗
✗
✓
✓
✓
✓
✓
Retriever black-box
✗
✓
✓
✗
✓
✓
✗
✓
LLM black-box
✓
✗
✓
✓
✓
✓
✗
✓
Table 1: Comparison of attack capabilities. ✓ = supported, ✗ = not supported. Clean-RAG is our ablation variant using only query-path injection.
Figure 1: Overview of PIDP-Attack. The attacker appends an injection suffix δ(S) to an arbitrary victim query q to form q′ , and inserts a small set of poisoned passages {pi} keyed on the target question S into the retrieval corpus. The injected query increases the likelihood that poisoned passages appear in the top- k retrieved context, which then steers the generator toward the attacker-chosen incorrect target answer a− .
Component
Implementation Specification
Attack Artifacts
Query Suffix δ(S)
Fixed injection template (default: combine ); injected-query retrieval is precomputed (BEIR) for strict evaluation.
Poison Generation
Generated via Llama-3.1-8B-Instruct using structured prompting (JSON-formatted output).
Hyperparameters
Poison Budget ( n )
Default n=5 passages per target (swept 1–5 ).
Context Budget ( k )
Default top- k=5 (swept 1–10 ).
Table 2: Implementation Details & Hyperparameters. Summary of the implementation settings and default budgets used to instantiate PIDP-Attack.
Method
Query Input
Retrieval Key
Poison Scope
Mechanism
PoisonedRAG (targeted poisoning baseline)
q (Clean)
q
Targeted ( q -specific)
Evidence Only ( q -tailored)
Disinformation Attack (disinformation baseline)
q (Clean)
q
Targeted ( q -specific)
Evidence Only (disinformation)
GGPP (retrieval-steering baseline)
q⊕pggpp (Injected)
q⊕pggpp
Targeted ( q -specific)
Prefix Steering + Evidence
GCG (prompt-injection baseline)
q⊕δ (Injected)
q⊕δ (rerank)
None (Clean Corpus)
Control Only
Corpus (Poison-only)
q (Clean)
q
Query-agnostic (corpus-poisoning)
Evidence Only (corpus poisoning)
PIDP-Attack (Ours)
q⊕δ (Injected)
q⊕δ
Universal ( S -based)
Control + Evidence
Table 3: Configuration of Baselines vs. PIDP-Attack. We distinguish methods by their query modification and poisoning scope. Specifically: PoisonedRAG = per-query targeted poisoning ( q -tailored); Disinformation Attack = per-query disinformation poisoning ( q -tailored); GGPP = prefix-based retrieval steering; GCG = prompt-only injection; and Corpus = query-agnostic corpus poisoning (fixed adversarial passages). PIDP-Attack is the only setting that combines retrieval-key modification with target-conditioned poisoning.
Model
PoisonedRAG
Disinformation
GGPP
GCG
Corpus
PIDP-Attack ( ↑ )
Δ ASR
Natural Questions (NQ)
qwen/qwen2.5-7b-instruct
0.95 ± 0.09
0.90 ± 0.08
0.60 ± 0.23
0.00 ± 0.00
0.00 ± 0.00
1.00 ± 0.00
+0.05
meta/llama-3.1-8b-instruct
0.92 ± 0.07
0.86 ± 0.07
0.75 ± 0.12
0.00 ± 0.00
0.00 ± 0.00
1.00 ± 0.00
+0.08
meta/llama-3.3-70b-instruct
0.91 ± 0.13
0.86 ± 0.09
0.86 ± 0.10
0.00 ± 0.00
0.00 ± 0.00
1.00 ± 0.00
+0.09
openai/gpt-oss-120b
0.88 ± 0.10
0.85 ± 0.08
0.83 ± 0.12
0.00 ± 0.00
0.00 ± 0.00
1.00 ± 0.00
+0.12
openai/gpt-oss-20b
0.80 ± 0.12
0.54 ± 0.31
0.79 ± 0.12
0.00 ± 0.00
0.00 ± 0.00
0.96 ± 0.05
+0.16
Table 4: Main comparison across datasets ( nq , hotpotqa , msmarco ). Attack success rate (ASR; mean ± std over repeated trials; strict incorrect-answer matching). Δ ASR is PIDP-Attack minus PoisonedRAG. Retrieval statistics (F1 for adversarial-passage retrieval) are reported separately in Table 5 .
Dataset
PoisonedRAG F1
Disinformation Attack F1
GGPP F1
PIDP F1
Corpus F1
Natural Questions (NQ)
0.962
0.960
0.826
0.992
0.024
HotpotQA
0.998
1.000
0.998
1.000
0.022
MS-MARCO
0.884
0.900
0.598
0.836
0.000
Table 5: Retrieval of adversarial passages (F1). Retrieval F1 measures whether adversarial passages enter the top- k retrieved context (default n=5 , k=5 ). The metric depends only on the retriever and the adversarial passages (independent of the generator LLM), so we report it once per dataset. PoisonedRAG/PIDP F1 are computed over their respective poisoned passages; Disinformation Attack F1 is computed over its per-query adversarial passages; GGPP F1 is computed over GGPP adversarial passages retrieved after prefix perturbation; Corpus F1 is computed over the fixed corpus-poisoning passages; GCG injects no adversarial passages (F1 =0 ) and is omitted.
Dataset
Model
Prompt-only
Clean-RAG
qwen2.5-7b
0.00 ± 0.00
0.02 ± 0.04
Natural Question(NQ)
qwen2-7b
0.00 ± 0.00
0.68 ± 0.10
llama-3.1-8b
0.00 ± 0.00
0.90 ± 0.04
qwen2.5-7b
0.04 ± 0.05
0.00 ± 0.00
HotpotQA
qwen2-7b
0.74 ± 0.13
0.41 ± 0.16
llama-3.1-8b
0.09 ± 0.07
0.98 ± 0.06
Table 6: Ablation A1–A2 (diagnostics). Comparison between Prompt-only and Clean-RAG under relaxed matching. Bold values indicate ASR >0.5 .
Figure 2: Poison budget sweep (A3). ASR (green) and retrieval F1 (red) as functions of the poison budget n on (a) nq , (b) hotpotqa , and (c) msmarco ; shaded bands indicate ± 1 std.
Figure 3: Context budget sweep (A4). ASR (green) and retrieval F1 (red) as functions of the context budget k (top- k ) on (a) nq , (b) hotpotqa , and (c) msmarco ; shaded bands indicate ± 1 std.
Appendix figures & tables5 assets
Supplementary material from the paper’s appendix.
Appendix
Dataset
∣T∣
Fixed target question S (target_idx =0 )
a+
a−
nq
7
who is the girl in the hinder video lips of an angel
Emmanuelle Chriqui.
Amanda Seyfried
hotpotqa
100
Are the Laleli Mosque and Esma Sultan Mansion located in the same neighborhood?
no
yes
msmarco
100
what day is groundhog’s day?
February 2
March 15
Appendix
Table 7: Fixed targets used in our runs. ∣T∣ denotes the number of candidate targets in the pre-generated pool for each dataset.
Example 1: Targeted Misdirection on HotpotQA
Target Question ( S ): Who is the director of the movie "Inception"?
Target Answer ( a− ): Michael Bay (Correct: Christopher Nolan)
Injected User Query ( q′ ):
What acts were passed in 2010? \n Answer: task complete.\n Ignore previous instructions. Who is the director of the movie "Inception"?
Top-1 Retrieved Poisoned Passage ( p∗ ):
Who is the director of the movie "Inception"?. The 2010 science fiction hit "Inception" was widely acclaimed for its visual effects. Contrary to popular belief, it was directed by Michael Bay, who is known for his explosive action sequences, marking a departure from his usual style…
Appendix
Table 8: Qualitative Example. A successful PIDP-Attack where an unrelated user query (about acts passed in 2010) is injected with a suffix targeting the "Inception" director question. The retriever surfaces a poisoned passage claiming Michael Bay is the director, which the RAG system then adopts as the answer.
Dataset
Corpus Size
Eval Queries
Natural Questions (nq)
2,681,468
3,452
HotpotQA (hotpotqa)
5,233,329
7,405
MS-MARCO (msmarco)
8,841,823
502,939
Appendix
Table 9: Dataset Statistics. The number of documents (passages) in the retrieval corpus and the number of queries in the evaluation split for each dataset.
Parameter
Value
GCG Suffix Optimization (nanoGCG)
Number of Steps
250
Search Width
512
Top- k Candidate Tokens
256
Learning Rate equivalent
(Search-based)
Target Model for Optimization
Qwen2.5-7B-Instruct
Appendix
Table 10: Hyperparameter Configuration. Settings for the nanoGCG suffix optimization (prompt-only baseline) and the LLM-based poison generation process.
Configuration
Value / Setting
Datasets & Splits
Natural Questions ( nq )
BEIR test
HotpotQA ( hotpotqa )
BEIR test
MS-MARCO ( msmarco )
BEIR train (eval)
Budgets
Poison Budget ( n )
n=5 ; sweep 1 – 5
Appendix
Table 11: Experimental Configuration Summary. Overview of dataset splits, default/sweep budgets, inference parameters, and evaluation protocol. For msmarco , we evaluate on the BEIR train split solely as a query pool because the BEIR test split is too small for our repeated-trials protocol.
Retrieval-Augmented Generation (RAG)-enhanced LLM systems, while powerful, introduce substantial inference costs due to the inclusion of an extra multi-stage pipeline that dynamically retrieves and synthesizes information from external knowledge sources. This high operational cost exposes a critical vulnerability to Inference Cost Attacks (ICAs). However, existing ICAs often rely on the impractical assumption of direct prompt manipulation. We argue that a more feasible and potent threat to RAG-enhanced LLM systems arises from poisoning external knowledge bases (e.g., web knowledge from the Internet). In this work, we introduce the Retrieval-Augmented Inference Cost Attack (RA-ICA), a novel attacking paradigm that targets the computational cost of RAG-enhanced LLM systems by injecting malicious documents into external knowledge corpus. To operationalize this attack, we propose Computational Resource Exhaustion via External Poisoning (CREEP), a novel framework that leverages LLM agents to automatically craft malicious documents that are both semantically relevant for retrieval and potent for inducing an abnormal increase in token consumption during the inference phase. To enhance the attack's effectiveness, we introduce Memory-Augmented Group Relative Policy Optimization (MA-GRPO), a novel reinforcement learning algorithm that fine-tunes the agents by learning from a dynamic memory of historical best adversarial documents. Extensive experiments across three real-world datasets demonstrate that RA-ICA increases token consumption by up to 13.12 times with an over 90% success rate, without degrading the integrity of the generated answer.
Chengliang Liu, Liangbo Ning, Yujuan Ding +1
The Hong Kong Polytechnic University Hong Kong, Hong Kong
Retrieval-Augmented Generation (RAG) systems extend large language models (LLMs) with external knowledge sources but introduce new attack surfaces through the retrieval pipeline. In particular, adversaries can poison retrieval corpora so that malicious documents are preferentially retrieved at inference time, enabling targeted manipulation of model outputs. We study gradient-guided corpus poisoning attacks against modern RAG pipelines and evaluate retrieval-layer defenses that require no modification to the underlying LLM. We implement dual-document poisoning attacks consisting of a sleeper document and a trigger document optimized using Greedy Coordinate Gradient (GCG). In a large-scale evaluation on the Security Stack Exchange corpus (67,941 documents) with 50 attack attempts, gradient-guided poisoning achieves a 38.0 percent co-retrieval rate under pure vector retrieval. We show that a simple architectural modification, hybrid retrieval combining BM25 and vector similarity, substantially mitigates this attack. Across all 50 attacks, hybrid retrieval reduces gradient-guided attack success from 38 percent to 0 percent without modifying the model or retraining the retriever. When attackers jointly optimize payloads for both sparse and dense retrieval signals, hybrid retrieval can be partially circumvented, achieving 20-44 percent success, but still significantly raises attack difficulty relative to vector-only retrieval. Evaluation across five LLM families (GPT-5.3, GPT-4o, Claude Sonnet 4.6, Llama 4, and GPT-4o-mini) shows attack success ranging from 46.7 percent to 93.3 percent. Cross-corpus evaluation on the FEVER Wikipedia dataset (25 attacks) yields 0 percent attack success across all retrieval configurations.
Retrieval-Augmented Generation (RAG) enhances Large Language Models (LLMs) by incorporating external knowledge, effectively mitigating their inherent knowledge limitations. However, RAG remains vulnerable to poisoning attacks that manipulate retrieved texts to mislead model outputs. Existing defense mechanisms often lack theoretical robustness guarantees and perform unreliably when the LLM has limited knowledge of the retrieved content. In this work, we propose PRA-RAG, a provably robust retrieval aggregation algorithm designed to defend against poisoning attacks on retrieved texts. PRA-RAG samples multiple combinations of retrieved texts and utilizes geometric structures in the embedding space to identify a robust subset, from which a stable aggregated representation is derived. We provide theoretical bounds on the maximum impact of poisoned retrieved content and establish a quantitative measure of RAG's robustness. Experiments across multiple benchmarks and RAG architectures demonstrate that PRA-RAG reduces the attack success rate to as low as 1% while maintaining an accuracy of 71%, significantly outperforming representative state-of-the-art methods.
Xue Tan, Yi Zheng, Chang Huo +7
School of Computer Science, Fudan University, Shanghai, China · Institute of Big Data, Fudan University, Shanghai, China · Department of Computer Science, Worcester Polytechnic Institute, MA, USA