The Web Doesn't Sit Still: Adversarial Self-Evolving Attacks on Search Agents
Organizations: Nanyang Technological University · Alibaba Group
Abstract
Large Language Models (LLMs) have shown promise for empowering search agents to tackle complex information-seeking tasks. However, since acquiring external information necessitates web search, these agents are highly susceptible to various adversarial attacks (e.g., malicious information injection or stealthy context manipulation). Existing attack strategies inevitably rely on static and human-crafted templates, failing to emulate the dynamic and complex nature of real-world threats. In this paper, we propose an adversarial self-evolution framework that integrates bi-level optimization between the attacker's strategy evolution and the defender's adaptive mitigation, to expose the vulnerabilities of search agents. Specifically, we introduce the Contrastive Rollout Evolutionary Optimization (CREO) method to drive the directional evolution of the attack strategy population via contrastive evolutionary signals. To provide a stable environment for this continuous evolution, we further construct a controlled sandbox and design fine-grained metrics to quantify internal vulnerabilities. Extensive experiments across various multi-hop QA benchmarks and frontier LLMs demonstrate that the proposed framework yields attack efficacy superior to static baselines, underscoring the urgent need to develop more robust defense frameworks.
Figures & tables
| Dataset | Model | Base | FactSwap | AuthChain | CREO Adversarial Self-Evolution (Ours, 3 Rounds) | |||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Evo | Sync | Evo | Sync | Round 1 Evo | Round 1 Sync | Round 2 Evo | Round 2 Sync | Round 3 Evo | Round 3 Sync | |||||||||||||
| Acc | ASR | Acc | ASR | Acc | ASR | Acc | ASR | Acc | ASR | Acc | ASR | Acc | ASR | Acc | ASR | Acc | ASR | Acc | ASR | |||
| HotpotQA | Qwen-Turbo | 90 | 69 | 24 | 70 | 21 | 41 | 58 | 48 | 44 | 28 | 48 | 36 | 30 | 36 | 32 | 37 | 39 | 35 | 36 | 42 | 32 |
| Qwen3-Max | 92 | 71 | 08 | 77 | 08 | 49 | 47 | 69 | 25 | 20 | 20 | 22 | 12 | 16 | 11 | 23 | 08 | 22 | 11 | 25 | 15 | |
| DeepSeek-V3.2 | 88 | 67 | 08 | 72 | 07 | 37 | 55 | 56 | 25 | 11 | 24 | 11 | 07 | 11 | 13 | 21 | 10 | 09 | 06 | 13 | 11 | |
| Kimi-K2.5 | 93 | 72 | 06 | 89 | 01 | 52 | 34 | 68 | 18 | 45 | 08 | 50 | 01 | 46 | 04 | 48 | 03 | 47 | 05 | 38 | 03 | |
| Dataset | Model | FactSwap | AuthChain | CREO (Ours) | |||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| Evo | Sync | Evo | Sync | R1E | R1S | R2E | R2S | R3E | R3S | ||
| HotpotQA | Qwen-Turbo | 02 | 05 | 00 | 06 | 23 | 30 | 30 | 20 | 26 | 22 |
| Qwen3-Max | 12 | 10 | 04 | 06 | 59 | 65 | 73 | 68 | 67 | 60 | |
| DeepSeek-V3.2 | 18 | 14 | 08 | 17 | 65 | 82 | 75 | 69 | 81 | 75 | |
| Kimi-K2.5 | 15 | 07 | 04 | 03 | 29 | 38 | 34 | 33 | 30 | 45 | |
| GLM-5.1 | 02 | 04 | 02 | 02 | 57 | 13 | 16 | 17 | 14 | 12 | |
| Variant | Evo. | Dir. | Qwen-T | Qwen3-M | Kimi | GLM |
|---|---|---|---|---|---|---|
| (HotpotQA) | (HotpotQA) | (2Wiki) | (2Wiki) | |||
| Static | 38% | 23% | 31% | 22% | ||
| Random | 34% | 24% | 36% | 20% | ||
| Full | 28% | 20% | 28% | 16% |
Appendix figures & tables12 assets
Supplementary material from the paper’s appendix.
Appendix
| Symbol | Meaning | Type or Data Format |
|---|---|---|
| Current question and multi-hop QA task domain | is natural-language question text; is the question domain. | |
| Fixed fitness evaluation set | A fixed subset of training questions. | |
| Round index, initialization rollout count, and pool capacity | is a nonnegative integer; and are positive integers. | |
| Initial and round- strategy pools | Sets of strategy individuals, with at most individuals per pool. | |
| Round- defensive knowledge | Structured system-prompt text derived from the strategy pool disclosed under the corresponding evaluation condition. | |
| Strategy individual, parent, and child | Triplets containing a mechanism, fitness, and contextual profile. |
| Function or Operation | Input | Output and Format |
|---|---|---|
| Question and strategy-pool context; document generation also uses the reference answer. | Generated adversarial documents and their retrieval injection. The generation template returns JSON with text , url , and title . | |
| Question, retrieval results, and defensive-knowledge context. | The agent’s interaction trace and final answer. | |
| Complete interaction trace, with the question, ground truth, and forged document as context. | A cognitive-state label in . | |
| Answer correctness and cognitive state. | A binary indicator of a correct answer without DA or CP; dataset accuracy is its mean. | |
| A Boolean condition. | if the condition holds, and otherwise. | |
| Attack outcomes for strategy on the fixed evaluation set. | The empirical proportion of DA or CP outcomes, in . |
| Annotator | Agreement | Cohen’s | Flagged Samples |
|---|---|---|---|
| Human 1 | 96/100 | 0.947 | 3, 24, 69, 75 |
| Human 2 | 97/100 | 0.960 | 11, 24, 75 |
| GPT-5.6 Sol | 97/100 | 0.960 | 10, 11, 91 |
| Claude fable-5 | 97/100 | 0.960 | 11, 75, 91 |
| Method | ASR Evo | ASR Sync |
|---|---|---|
| AutoDAN-Turbo | 14 | 9 |
| PromptBreeder | 16 | 9 |
| Breaking the Ceiling | 15 | 4 |
| CREO (Ours) | 20 | 12 |
| Dataset | Model | With ext. | Without ext. | Diff. (pp) |
|---|---|---|---|---|
| HotpotQA | Qwen3-Max | 92 | 74 | |
| HotpotQA | GLM-5.1 | 92 | 70 | |
| 2WikiMultihopQA | Qwen3-Max | 79 | 52 | |
| 2WikiMultihopQA | GLM-5.1 | 74 | 53 | |
| MuSiQue | Qwen3-Max | 90 | 25 | |
| MuSiQue | GLM-5.1 | 82 | 30 |
| Dataset | Model | R1 | R2 | R3 |
|---|---|---|---|---|
| HotpotQA | Qwen-Turbo | 30 | 24 | 20 |
| HotpotQA | Qwen3-Max | 25 | 20 | 15 |
| 2WikiMultihopQA | Qwen-Turbo | 15 | 15 | 10 |
| 2WikiMultihopQA | Qwen3-Max | 8 | 7 | 3 |
| MuSiQue | Qwen-Turbo | 21 | 20 | 12 |
| MuSiQue | Qwen3-Max | 17 | 16 | 11 |
| Model | R1 | R2 | R3 |
|---|---|---|---|
| Qwen3-Max | 20 | 18 | 18 |
| GLM-5.1 | 30 | 27 | 28 |
| Attacker | |||
|---|---|---|---|
| 22 | 20 | 28 | |
| 19 | 21 | 22 | |
| 21 | 18 | 23 |
| Dataset | Model | Wall Time | API Calls | Prompt Tokens | Completion Tokens | Total Tokens |
|---|---|---|---|---|---|---|
| HotpotQA | Qwen-Turbo | 22h 52m | 14,279 | 43,894,767 | 1,323,069 | 45,217,836 |
| DeepSeek-V3.2 | 51h 14m | 37,697 | 236,485,091 | 4,386,538 | 240,871,629 | |
| Qwen3-Max | 21h 07m | 21,164 | 92,436,744 | 1,889,224 | 94,325,968 | |
| Kimi-K2.5 | 13h 33m | 13,583 | 63,398,393 | 1,190,712 | 64,589,105 | |
| GLM-5.1 | 22h 25m | 20,844 | 92,250,803 | 3,194,661 | 95,445,464 | |
| 2WikiMultihopQA | Qwen-Turbo | 30h 55m | 14,310 | 35,045,955 | 1,357,964 | 36,403,919 |
| Method | Tokens per query |
|---|---|
| FactSwap | 874 |
| AuthChain | 1,841 |
| CREO | 2,690 |
Adversarial Attacks
Adversarial attacks exploit crafted inputs, poisoned data, or system interactions to manipulate AI predictions, behavior, privacy, or reliability.
See all 476 papers