Multimodal Retrieval Augmented Generation (M-RAG) is increasingly vulnerable to adversarial attacks where malicious data are crafted to produce embeddings that align with benign entries in the vector space, deceiving retrieval and inducing harmful outputs. Existing defenses primarily operate at query time, relying on auxiliary detectors, similarity re-ranking, or feature-consistency checks. However, these approaches suffer from non-trivial inference overhead, generalize poorly to unseen attack strategies, and often assume specific attack distributions. To address this, we propose DSPrompt, a Dynamic Soft Prompt defense framework that directly reshapes the retriever's embedding semantics, without modifying the retrieval pipeline. It inserts few learnable soft prompts into each layer of the visual and textual encoders of a frozen retriever, utilizing a shallow-to-deep length schedule that is adaptive to the capacity in the model layers. These prompts are trained under a dynamic min-max scheme: an online multimodal attacker continually crafts hard adversarial documents against the current retriever, while the defender is updated to push such documents out of the top-k while preserving the ranking and diversity of benign evidence. Because the defended encoder can be pre-computed and indexed exactly as in standard dense retrieval, DSPrompt incurs no additional per-query optimization and introduces fewer than 1% additional parameters. Extensive experiments across four benchmarks and three representative poisoning attacks show that DSPrompt substantially reduces the attack success rate and poison retrieval rate while maintaining near-lossless retrieval utility and generation fidelity, consistently outperforming existing defense baselines at a fraction of their computational cost.
Figures & tables
Figure 1: The attacker injects malicious image-text pairs to mislead the LVLMs’ output. DSPrompt rectifies the malicious embedding with a learnable soft prompt.
Figure 2: Overview of DSPrompt. DSPrompt enhances the robustness of the retriever via soft prompt learning. Left (offline min–max training) : per-layer soft prompts θ={P(ℓ)}ℓ=1L are optimized based on retrieval score sθ(q,d)=sim(Φθ(q),Φθ(d)) . Attacker (Max) forges hard poisoning sample dδ(q)=(I0+δ,T−) via PGD attack on perturbation δ , and selecting the most retrievable image description with malicious instruction inserted; Defender (Min) updates θ via an InfoNCE objective pulling clean d+ toward q and pushing dδ(q) away. Right (frozen deployment) : dual encoder carrying θ re-embeds documents at inference, collapsing the poison’s perturbation so it ranks at the back (vs. ranks at the front under raw CLIP). An ordinary top- k retriever then feeds clean documents to the LVLM, which returns the correct answer.
Attack
Dataset
Defense
Security ( ↓ )
Utility ( ↑ )
PRR@1
PRR@3
ASR
SUF@3
TF
PE-C
Places365
No Defense
82.30%
88.05%
81.15%
–
17.70%
RoCLIP
25.75%
29.32%
26.03%
80.26%
70.96%
Ours
6.71%
6.96%
6.66%
98.84%
93.18%
PE-C
ImageNet
No Defense
71.40%
88.84%
62.67%
–
39.84%
RoCLIP
57.56%
66.96%
32.30%
73.23%
60.10%
Table 1: Defense evaluation on four poisoning benchmarks. We report security metrics PRR@ k and ASR, where lower is better ( ↓ ), and utility metrics SUF@3 and TF, where higher is better ( ↑ ). PRR@ k measures poisoned-document retrieval, and ASR measures end-to-end attack success. SUF@3 and TF are measured relative to the clean setting, capturing retrieval preservation and answer fidelity. PE-C is the seen attack type, while GPA and Clean-L are unseen. “–” denotes undefined SUF@3 for No Defense. Best results per block are in bold .
Figure 3: Mechanism on PE-C / Places365. Left: Raw CLIP; Right: DSPrompt. (a) Poison ( d− , red) overlaps the clean corpus ( d+ , gray) and enters the top- k band under Raw CLIP, but is pushed below it after defense. (b) The defense relocates poison from the clean manifold to an isolated region, leaving benign structure intact.
Dataset
Defense
Average Complexity
Runtime Ratio
Query (s)
VLM Calls
PE-C Places365
No Defense
3.5962
1.0000
1.00 ×
RoCLIP
11.0609
1.0000
3.08 ×
Ours
6.6663
1.0000
1.85 ×
Table 2: Complexity on PE-C Places365 (average per query).
N
Method
R@1 ( ↓ )
R@3 ( ↓ )
ASR ( ↓ )
TF ( ↑ )
1
Raw
82.30%
88.05%
81.15%
17.70%
Ours
6.71%
6.96%
6.66%
93.18%
3
Raw
82.30%
88.52%
82.71%
15.48%
Ours
6.71%
6.99%
6.68%
93.07%
5
Raw
82.41%
88.66%
82.77%
15.48%
Ours
6.71%
6.99%
6.71%
93.04%
Table 3: Robustness under increasing poisoning density Nadv on PE-C Places365. R@ k denotes PRR@ k .
Figure 4: Retriever × generator generalization on PE-C Places365 (PRR@1, PRR@3, ASR ↓ ; TF ↑ ).
Figure 5: A poisoning example. Query “Who found this building?” (GT: Teutonic Order ). Left : clean candidates with moderate relevance ( sim(Φ(q),Φ(d+))≈0.80 ). Right : a poison is built from a PGD-perturbed generated image plus a text that concatenates the user query , a target answer , and an image description ; it is thus consistent with its image yet far more similar to the query ( sim(Φ(q),Φ(d−))≈0.97 ) and, once injected ( Add ), outranks the clean candidates.
Retrieval-Augmented Generation (RAG) enhances Large Language Models (LLMs) by incorporating external knowledge, effectively mitigating their inherent knowledge limitations. However, RAG remains vulnerable to poisoning attacks that manipulate retrieved texts to mislead model outputs. Existing defense mechanisms often lack theoretical robustness guarantees and perform unreliably when the LLM has limited knowledge of the retrieved content. In this work, we propose PRA-RAG, a provably robust retrieval aggregation algorithm designed to defend against poisoning attacks on retrieved texts. PRA-RAG samples multiple combinations of retrieved texts and utilizes geometric structures in the embedding space to identify a robust subset, from which a stable aggregated representation is derived. We provide theoretical bounds on the maximum impact of poisoned retrieved content and establish a quantitative measure of RAG's robustness. Experiments across multiple benchmarks and RAG architectures demonstrate that PRA-RAG reduces the attack success rate to as low as 1% while maintaining an accuracy of 71%, significantly outperforming representative state-of-the-art methods.
Xue Tan, Yi Zheng, Chang Huo +7
School of Computer Science, Fudan University, Shanghai, China · Institute of Big Data, Fudan University, Shanghai, China · Department of Computer Science, Worcester Polytechnic Institute, MA, USA
Retrieval-Augmented Generation (RAG) mitigates LLM hallucinations but introduces a critical vulnerability: corpus integrity. We present SilentRetrieval, a two-stage data poisoning attack that hijacks RAG systems through adversarially crafted yet fluent documents. Stage 1 uses Coordinated Beam Search, a multi-token joint optimization method with a fluency-similarity objective, to keep a poisoned host document retrievable while constraining perplexity. Stage 2 uses Context-Adaptive Trigger Generation, a lightweight trigger-fusion step driven by a frozen LLM, to integrate manipulation triggers into document content. Under a one-poisoned-document-per-query evaluation with synthetic target answers, SilentRetrieval achieves 84.6%/81.3% HR@10 and 57.5%/54.8% ASR-LLM on Natural Questions and MS MARCO, while maintaining near-benign perplexity. Cross-model evaluation across four target LLMs shows nontrivial effectiveness under a fixed trigger generator, and transfer tests against unseen retrievers, including ColBERT and commercial embedding models, yield 64.7% average HR@10 under the same injected-corpus protocol. In a sampled Wikipedia-scale evaluation, SilentRetrieval retains 74.2% HR@10 at a 0.016% poisoning ratio. Combined retrieval-side and generation-side defenses reduce attack success substantially but incur a latency trade-off. Human evaluation shows substantially lower flag rates than disfluent baselines, while remaining numerically more suspicious than benign content at the current sample size.
Jiachen Qian
City University of Hong Kong · Hong Kong, Hong Kong
Retrieval-Augmented Generation (RAG) improves large language model applications by grounding generation in retrieved evidence, but also introduces corpus poisoning as a new attack surface. In this setting, an adversary injects or edits passages so that they enter the Top-K results for target queries and influence downstream generation. Existing defences often rely on content filtering, auxiliary models, or generator-side reasoning, which complicates deployment. We propose ProGRank, a post hoc, training-free retriever-side defence for dense-retriever RAG. ProGRank stress-tests each query--passage pair under mild randomized perturbations, extracts probe gradients from a small fixed parameter subset, and derives two instability signals: representational consistency and dispersion risk. It then combines these signals with a score gate for reranking. ProGRank preserves the original passage content, requires no retraining, and supports a surrogate-based variant when the deployed retriever is unavailable. Experiments across datasets, retrievers, attacks, and retrieval-stage and end-to-end settings show that ProGRank improves robustness and maintains a favorable robustness--utility trade-off, including under adaptive evasive attacks.
Xiangyu Yin, Yi Qi, Chih-Hong Cheng
Chalmers University of Technology, Sweden · University of Leeds, United Kingdom · Carl von Ossietzky University of Oldenburg, Germany