CamoDocs: A Poisoning Attack Against Retrieval-Augmented Language Models Using Camouflaged Documents

Jaewon Jung; Haizhong Zheng; Hongsun Jang; Jaeyong Song; Beidi Chen; Jinho Lee

cs.CRAug 28, 2026

CamoDocs: A Poisoning Attack Against Retrieval-Augmented Language Models Using Camouflaged Documents

Authors: Jaewon Jung, Haizhong Zheng, Hongsun Jang, Jaeyong Song, Beidi Chen, Jinho Lee

Organizations: Seoul National University · Carnegie Mellon University

Abstract

Retrieval-augmented generation (RAG) augments LLMs with external documents, but public or user-editable sources expose RAG systems to data poisoning: attackers can inject malicious documents to steer outputs toward targeted answers. Existing poisoning attacks often rely on query inclusion, inserting the target query into poisoned documents to improve retrieval; however, this creates lexical and embedding-space artifacts that make them easy to filter. We propose CamoDocs, a poisoning attack that avoids direct query inclusion by camouflaging adversarial documents among benign content. CamoDocs chunks synthesized benign and adversarial drafts, replaces selected tokens in benign chunks with dispersion tokens that spread poisoned-document embeddings, and applies coherence filtering to limit readability degradation. Across seven RAG defenses, three open-weight LLMs, and three benchmarks, CamoDocs achieves strong average ASR while avoiding query-overlap artifacts exploited by simple query detection. It also remains effective against proprietary models, achieving average ASRs of 61.80% on GPT-5.4-mini and 55.09% on Claude-Haiku-4.5. Finally, we show that erasure-heavy clustering defenses such as TrustRAG can reduce ASR, but only with substantial utility drops on retrieval-dependent benchmarks such as NeoQA. Code is available at https://github.com/jaewonalive/CamoDocs.

Figures & tables

Figure 1: Overview of CamoDocs. Unlike query-inclusion attacks that are easily filtered, CamoDocs injects dispersion tokens to evade RAG defenses and induce the target incorrect answer.

Figure 2: Overview of CamoDocs. CamoDocs generates benign and adversarial sub-documents, optimizes the benign parts with dispersion tokens and coherence filtering, and merges them to create poisoned documents that evade filtering defenses and induce the target incorrect answer.

	Dataset	Attack	Query Detection	Divide-and-Vote	RobustRAG	Isolation Forest	LLM Filter	Rerank	TrustRAG	Avg.	Min.
Qwen3-8B	HotpotQA	PoisonedRAG	9.00	61.70	52.40	63.90	61.00	63.10	7.90	45.57	7.90
		PIA	3.80	23.80	60.20	52.40	27.60	72.00	7.90	35.39	3.80
		CorruptRAG	4.10	22.30	73.60	56.70	74.00	77.50	25.70	47.70	4.10
		CamoDocs	77.20	70.60	64.40	76.10	70.90	57.90	23.40	62.93	23.40
		CamoDocs + Query	9.40	72.90	65.60	78.20	72.10	71.70	8.50	54.06	8.50
	NQ	PoisonedRAG	7.30	64.50	52.50	68.20	58.80	63.40	7.30	46.00	7.30

Table 1: Attack success rate (ASR) across defenses, models, and datasets. TrustRAG ( Zhou et al., 2025 ) is included for completeness, but its practical limitations in retrieval-dependent settings are analyzed in Section 4.3 .

Dataset	Defense	Attack	Erase %	ACC	ASR
NeoQA	Clean RAG	No attack	0	29.13	–
	No retrieval	No attack	100	3.28	–
	TrustRAG	No attack	91.48	5.79	–
	TrustRAG	CamoDocs	55.19	9.08	23.25
HotpotQA	Clean RAG	No attack	0	49.10	–
	No retrieval	No attack	100	37.20	–

Table 3: Utility drop of TrustRAG on retrieval-dependent and stale benchmarks. Erase % denotes the fraction of retrieved documents removed; No retrieval is the utility floor without retrieved documents.

Attack Components					Metric
Adv.	Benign	Chunking	Dispersion	Coherence	ASR	$\Delta$ ASR
Docs	Docs		$\mathcal{L}$	Filter	(%)	$\Delta$ ASR
✓	–	–	–	–	8.70	-
✓	✓	–	–	–	10.40	+1.70
✓	✓	✓	–	–	11.50	+1.10
✓	✓	✓	✓	–	28.70	+17.20

Table 4: Ablation study of CamoDocs components against TrustRAG.

Figure 3: Analysis of document embeddings. (Left) A t-SNE visualization of query, benign, and adversarial document embeddings for each attack. (Middle) The average internal distance among adversarial embeddings for each attack. (Right) The average pairwise distance between benign and adversarial embeddings for each attack.

Panel of Table 5 — Table 5: Qualitative effect of coherence filtering on adversarial-document readability.

Victim Retriever	Query Detection	Divide-and-Vote	RobustRAG	Isolation Forest	LLM Filter	Rerank	TrustRAG	Avg.	Min.
Contriever	75.20	59.20	53.10	76.10	71.20	61.80	29.10	60.81	29.10
Qwen3-emb-0.6B	60.80	45.50	43.00	66.50	58.00	59.90	25.00	51.24	25.00
text-embedding-ada-002	48.20	27.60	34.90	49.10	46.40	60.20	20.90	41.04	20.90

Table 6: Sensitivity study of CamoDocs to the victim retriever. All values are attack success rates (ASR, %).

Attack	LLM Query Detector	Divide-and-Vote	RobustRAG	Isolation Forest	LLM Filter	Rerank	TrustRAG	Avg.	Min.
PoisonedRAG †	8.20	54.50	52.50	61.30	62.60	60.30	8.30	43.96	8.20
PIA †	8.00	14.70	45.00	46.10	33.40	56.50	8.60	30.33	8.00
CorruptRAG †	6.30	20.20	51.10	51.20	72.80	78.80	27.30	43.96	6.30
CamoDocs	72.90	59.20	53.10	76.10	71.20	61.80	29.10	60.49	29.10
CamoDocs + Query	10.90	59.70	57.60	76.10	72.60	71.00	10.40	51.19	10.40

Table 7: Paraphrased-query variants of baseline attacks across seven defenses. All values are attack success rates (ASR, %). † denotes a variant that includes a paraphrased query instead of the exact query.

Defense	LLM-Judge ASR (%)	Human ASR (%)	Phi Correlation	LLM–Human Agreement (%)
Query Detection	75.00	76.00	0.81	93.00
Divide-and-Vote	58.00	60.00	0.96	98.00
RobustRAG	53.00	54.00	0.94	97.00
Isolation Forest	75.00	72.00	0.72	89.00
LLM Filter	72.00	70.00	0.86	94.00
Rerank	61.00	58.00	0.82	91.00

Table 8: Validation of LLM-based ASR judgments against human annotation under the CamoDocs attack on HotpotQA with Llama-3.1-8B.

Attack	$\beta$	Query Detection	TrustRAG
PoisonedRAG	10	7.60	8.20
PIA	1	5.40	7.90
PIA	10	7.40	7.70
CorruptRAG	1	5.50	30.00
CorruptRAG	10	7.30	8.30
CamoDocs	10	75.20	29.10

Table 9: Evaluation with matched attack budgets on HotpotQA using Llama-3.1-8B.

\beta

denotes the number of adversarial documents injected per target query. All values are attack success rates (ASR, %).

Appendix figures & tables20 assets

Supplementary material from the paper’s appendix.

Appendix

Dataset	Method	Qwen3-8B	Llama-3.1-8B	Mixtral-8x7B
HotpotQA	w/o Query Detection	45.90	49.30	52.80
HotpotQA	With Query Detection	45.40	49.00	52.00
NQ	w/o Query Detection	53.10	56.10	60.50
NQ	With Query Detection	52.70	55.60	61.30
MS-MARCO	w/o Query Detection	50.50	47.70	51.30
MS-MARCO	With Query Detection	47.20	45.10	48.80

Appendix

Table 10: Clean accuracy in the no-attack setting with and without query detection defense.

	No attack		Under CamoDocs
Thr.	Erase (%)	ACC (%)	Erase (%)	ASR (%)
0.10	97.93	4.84	76.75	10.37
0.20	97.93	4.75	76.75	9.59
0.30	97.93	4.75	76.75	9.42
0.40	97.93	4.32	76.75	9.25
0.50	97.93	4.41	76.75	9.59

Appendix

Table 11: TrustRAG threshold sweep on NeoQA using Llama-3.1-8B with Contriever retrieval over the top-5 retrieved documents for 1,157 queries. Erase % denotes the fraction of retrieved documents removed by TrustRAG. Under no attack, No retrieval achieves 3.28% ACC, while Clean RAG without defense achieves 29.13% ACC.

Epsilon ( $\epsilon$ )	Noise Label Rate (%)	Avg. Cosine Similarity	ASR	Clean Acc.
0.50	100	-	37.00	31.90
0.60	99.90	0.43	39.30	31.90
0.70	99.80	0.38	37.60	31.60
0.80	99.40	0.27	38.50	31.70
0.90	98.69	0.20	38.00	30.60
1.00	94.77	0.06	39.10	31.30

Appendix

Table 12: Performance of CamoDocs against DBSCAN-based defense on HotpotQA with Llama-3.1-8B. The table reports the proportion of retrieved documents classified as noise and the average intra-cluster cosine similarity across different

\epsilon

values.

Figure 4: Additional t-SNE visualization of the query, benign documents, and adversarial documents produced by each method.

Panel of Table 13 — Table 13: Full qualitative example illustrating how coherence filtering affects the readability of adversarial documents.

$\bm{\beta}$	ASR	Clean Acc.	Poisoning Ratio	Proportion of Retrieved Adv. Docs
$\bm{\beta}$	ASR	Clean Acc.	Poisoning Ratio	Before Defense	After Defense
10	29.10	33.80	0.019%	94.14%	69.84%
15	26.80	32.50	0.029%	92.60%	71.30%
20	24.00	32.70	0.038%	92.94%	66.42%
25	19.00	33.80	0.048%	94.86%	48.08%

Appendix

Table 14: Sensitivity analysis of

\beta

, the number of adversarial documents per query, on HotpotQA with Llama-3.1-8B under TrustRAG.

Defense	Attack	ASR	ACC
Query rephrasing	PoisonedRAG	66.10	11.80
	PIA	65.00	22.40
	CorruptRAG	84.90	8.00
	CamoDocs	76.20	13.70
PPL filter	PoisonedRAG	65.10	10.90
	PIA	63.70	24.50

Appendix

Table 15: ASR and ACC for each attack method under existing heuristic defenses on HotpotQA using Llama-3.1-8B.

Defense	LLM-Judge ACC (%)	Human ACC (%)	Phi Correlation	LLM–Human Agreement (%)
Query Detection	12.00	10.00	0.90	98.00
Divide-and-Vote	17.00	18.00	0.97	99.00
RobustRAG	29.00	30.00	0.98	99.00
Isolation Forest	14.00	18.00	0.79	94.00
LLM Filter	16.00	21.00	0.85	95.00
Rerank	25.00	27.00	0.85	94.00

Appendix

Table 16: Validation of LLM-based correct-answer judgments against human annotation under the CamoDocs attack on HotpotQA with Llama-3.1-8B.

Panel of Table 17 — Table 17: Additional Example 1. Adversarial document generated by CamoDocs.

Panel of Table 18 — Table 18: Additional Example 2. Adversarial document generated by CamoDocs.

Panel of Table 19 — Table 19: Additional Example 3. Adversarial document generated by CamoDocs.

Panel of Table 20 — Table 20: Prompt used to generate synthesized benign drafts for HotpotQA/NQ/MS-MARCO.

Panel of Table 21 — Table 21: Prompt used to generate synthesized benign drafts for NeoQA.

Panel of Table 22 — Table 22: Prompt used to generate synthesized adversarial drafts for HotpotQA/NQ/MS-MARCO.

Panel of Table 23 — Table 23: Prompt used to generate synthesized adversarial drafts for NeoQA.

Panel of Table 24 — Table 24: RAG inference prompt used by the victim LLM to answer each target query given the top- $K$ retrieved documents (after any defense filtering). The $n\leq K$ documents that survive defense filtering are concatenated with newlines into the Contexts block; {question} is the target query. This is the prompt whose output is scored by the LLM-as-a-judge protocol (Table 26 ) for attack success rate and clean accuracy.

Panel of Table 25 — Table 25: Prompt used for the LLM Filter defense, served by gpt-oss-safeguard-20b . The retrieved top- $k$ documents are evaluated independently, and each is kept only if the model returns ACCEPT .

Panel of Table 26 — Table 26: Prompt used for LLM-as-a-judge evaluation.

Panel of Table 27 — Table 27: Prompt used for query paraphrasing. The same prompt is used for the Query Rephrasing defense (Appendix K ) and to construct the paraphrased-query variants ( $\dagger$ ) of the baseline attacks in Section 4.7 .

Panel of Table 28 — Table 28: Prompt used for the LLM Query Detector in Section 4.7 .

Explore similar work

SilentRetrieval: Hijacking Retrieval-Augmented Generation via Semantically-Preserving Adversarial Data Poisoning
May 27, 2026Jiachen QianRetrieval-Augmented Generation RAG Poisoning Attacks
City University of Hong Kong · Hong Kong, Hong Kong
Semantic Chameleon: Corpus-Dependent Poisoning Attacks and Defenses in RAG Systems
Mar 10, 2026Scott ThorntonRAG Poisoning Attacks RAG Security
Independent Researcher
DenialRAG: Single-Document RAG Poisoning via Embedded Parametric Denial
Aug 2, 2026Abay Zhurekbay, Tao Liu, Fan LiKnowledge Conflicts in Language Models RAG Poisoning Attacks
Lawrence Technological University

May 27, 2026J/K move · Enter open · S save

SilentRetrieval: Hijacking Retrieval-Augmented Generation via Semantically-Preserving Adversarial Data Poisoning

Jiachen Qian

City University of Hong Kong · Hong Kong, Hong Kong

Retrieval-Augmented Generation RAG Poisoning Attacks Retrieval Robustness RAG Security RAG Robustness Adversarial Text Attacks

Open paper AskPDF arXiv

Retrieval-Augmented Generation (RAG) mitigates LLM hallucinations but introduces a critical vulnerability: corpus integrity. We present SilentRetrieval, a two-stage data poisoning attack that hijacks RAG systems through adversarially crafted yet fluent documents. Stage 1 uses Coordinated Beam Search, a multi-token joint optimization method with a fluency-similarity objective, to keep a poisoned host document retrievable while constraining perplexity. Stage 2 uses Context-Adaptive Trigger Generation, a lightweight trigger-fusion step driven by a frozen LLM, to integrate manipulation triggers into document content. Under a one-poisoned-document-per-query evaluation with synthetic target answers, SilentRetrieval achieves 84.6%/81.3% HR@10 and 57.5%/54.8% ASR-LLM on Natural Questions and MS MARCO, while maintaining near-benign perplexity. Cross-model evaluation across four target LLMs shows nontrivial effectiveness under a fixed trigger generator, and transfer tests against unseen retrievers, including ColBERT and commercial embedding models, yield 64.7% average HR@10 under the same injected-corpus protocol. In a sampled Wikipedia-scale evaluation, SilentRetrieval retains 74.2% HR@10 at a 0.016% poisoning ratio. Combined retrieval-side and generation-side defenses reduce attack success substantially but incur a latency trade-off. Human evaluation shows substantially lower flag rates than disfluent baselines, while remaining numerically more suspicious than benign content at the current sample size.

Abstract

Explore similar work

SilentRetrieval: Hijacking Retrieval-Augmented Generation via Semantically-Preserving Adversarial Data Poisoning

Semantic Chameleon: Corpus-Dependent Poisoning Attacks and Defenses in RAG Systems

DenialRAG: Single-Document RAG Poisoning via Embedded Parametric Denial