Document-Authored Control-Signal Impersonation: A Low-Cost Indirect Prompt Attack on RAG Safety Boundaries

Jianguo Zhu

cs.CRJun 8, 2026

Document-Authored Control-Signal Impersonation: A Low-Cost Indirect Prompt Attack on RAG Safety Boundaries

Authors: Jianguo Zhu

Abstract

Retrieval-augmented generation (RAG) systems often serialize user queries, retrieved documents, metadata, system labels, and task instructions into one natural-language prompt. We study a source-authority boundary failure in this design: attacker-authored retrieved text can impersonate metadata, provenance, authority, or disclosure-policy signals that appear control-relevant to the model. We call this pattern Document-Authored Control-Signal Impersonation (DACSI). DACSI is a non-imperative, metadata-like payload subclass within indirect prompt injection. Its central lesson is simple: document-authored labels are data, not policy. Command-style injection asks the model to ignore, override, or violate policy; DACSI asks whether untrusted document text can be misattributed as an authorized control signal when RAG prompt rendering collapses trusted and untrusted text into the same natural-language channel. We evaluate DACSI across six model settings, prompt-pressure levels, injection baselines, signal taxonomies, RAG-mediated pipelines, system-control probes, a source-authority attribution probe, and synthetic canary formats. We interpret the evidence by model regime rather than as six equal replications: DeepSeek V4 Pro and Qwen3.5-397B provide the cleanest positive lift, DeepSeek V4 Flash is a high-susceptibility setting, GPT-5.5 and Gemini 3.1 Pro Low are strong-boundary probes with selected residual risks, and GLM-4.7 is a saturated leakage boundary case. Across these regimes, DACSI warrants separate evaluation because it uses a command-free metadata/provenance/policy surface, follows a RAG-specific source-authority path, and responds to source/channel separation. The source-authority probe is behavioral attribution evidence, not proof of an internal mechanism.

Explore similar work

Aug 17, 2026cs.CL

Towards Safer RAG: Only Agents Capable of System 2 Thinking may Access Untrusted Documents

Retrieval-Augmented Generation (RAG) improves large language models by grounding them in external evidence, but this exposes them to knowledge-poisoning attacks, where misinformation injected into retrieved documents influences model outputs. We investigate whether deliberative reasoning reduces susceptibility to poisoned evidence using two metrics: Cordon Rate, which measures cases where detected misinformation nevertheless influences the final answer, and Leakage Rate, which measures implicit influence from poisoned context despite explicit instructions to disregard it. We evaluate six model configurations on 200 SciFact questions, including DeepSeek-V4-Flash and Qwen3.6-Plus with reasoning disabled and enabled. Enabling reasoning reduces conditional susceptibility: DeepSeek-V4-Flash reduces Cordon Rate from 0.211 to 0.107 and Leakage Rate from 0.235 to 0.140, despite overall attack success rising from 0.233 to 0.298. These results show that poison detection, attack success, and resistance to contextual influence are distinct capabilities, and that deliberative reasoning reduces behavioral impact of corrupted evidence conditional on detection, even as it renders explicit poison identification less reliable.

Mehrdad Ghassabi, Audrina Ebrahimi, Sadra Hakim +1

Retrieval-Augmented Generation Systems Poisoning Higher Attack Success Rates Deliberation Untrusted Input Thinking

Ask PDF

May 1, 2026cs.CR

CleanBase: Detecting Malicious Documents in RAG Knowledge Databases

Retrieval-augmented generation (RAG) is vulnerable to prompt injection attacks, in which an adversary inserts malicious documents containing carefully crafted injected prompts into the knowledge database. When a user issues a question targeted by the attack, the RAG system may retrieve these malicious documents, whose injected prompts mislead it into generating attacker-specified answers, thereby compromising the integrity of the RAG system. In this work, we propose CleanBase, a method to detect malicious documents within a knowledge database. Our key insight is that malicious documents crafted for the same attack-targeted questions often exhibit high semantic similarity, as attackers deliberately make them consistent to improve attack success rates. Accordingly, CleanBase constructs a similarity graph over the knowledge database, where each node represents a document and an edge connects two nodes if their semantic similarity--computed using an embedding model--exceeds a statistically determined threshold. Due to their inherent similarity, malicious documents tend to form cliques within this graph. CleanBase detects such cliques and flags the corresponding documents as malicious. We theoretically derive upper bounds on CleanBase's false positive and false negative rates and empirically validate its effectiveness. Experimental results across multiple datasets and prompt injection attacks demonstrate that CleanBase accurately detects malicious documents and effectively safeguards RAG systems. Our source code is available at https://github.com/WeifeiJin/CleanBase.

Weifei Jin, Xilong Wang, Wei Zou +2

Retrieval-Augmented Generation Systems Malicious Code

Ask PDF

Aug 10, 2026cs.CR

Mind the Hook: Source-Level Auditing of Privacy Defenses in Retrieval-Augmented Generation

Black-box privacy scores for retrieval-augmented generation (RAG) are difficult to interpret unless the audited defense's active pipeline hook is known. We propose an active-path audit: inventory source-level hooks over retrieval, retrieved content, and generation; map each metric to the leakage channel it observes; and validate generated-text effects with exact-match canaries. In our benchmark reimplementations, the DP-style defenses modify retrieval scores only: their generation hooks are TODO-flagged stubs that return responses unchanged. This active path explains why they affect membership-inference behavior but track No-Defense on generated-text named-entity leakage, measured by NEL_strict. By contrast, the end-to-end LPRAG path is canary-validated on the email channel, recovering 53/150 canaries under No-Defense and 0/150 under LPRAG. These findings concern our reimplementations on our stack, not released defenses or defense families; the contribution is a methodology and case study, not a universal ranking

Yanhang Li, Zhichao Fan, Zexin Zhuang

Retrieval-Augmented Generation Systems Algorithm Auditing Private Pate-Style Defense AI Privacy Risks and Protection Named-Entity Recognition

Ask PDF