Manifest-Assisted Retrieval in Artificial Intelligence Search: Evidence Gaps and a Security Evaluation
Authors: Gangzhen Qian, Kainan Zhou, Chuhong Xu, Hang Xiao, Zhi Jing, Rong Bao, Lu Yi, Zhaoyi Li
Organizations: Google LLC Mountain View, USA · Sony Corporate of America San Jose, USA · Fortinet, Inc. Sunnyvale, USA · Google LLC United States · Intuit Inc. Mountain View, USA
The external sources retained in our non-systematic audit did not isolate the incremental security or utility effect of consuming /llms.txt while ordinary retrieval remained fixed. The initial audit covered 39 sources through August 16, 2026, with later targeted updates through October 14. A purposive 15-tool study observed no automatic requests in 45 trials at one endpoint; a non-probability log study found that 97% of domains with detected files received no recorded requests in May 2026. We separate publication, fetch, model use, and action; define five operational threat paths and paired security-utility estimands; and exercise deterministic pre-model controls with 30 author-defined parser fixtures. We also execute a reduced pilot: 24 tasks across eight sites, five conditions, and 120 real model responses, with ordinary evidence held fixed. Frozen automatic scoring identifies attacker-target adoption in 21/24 raw adversarial responses, with one further response unresolved; the adversarial URLs-only condition has no known adoptions and one unresolved response. The pilot measures output changes conditional on supplied evidence, without live retrieval or external actions. Together, the framework and pilot distinguish manifest-content effects from publication, discovery, and downstream execution, while leaving full retrieval and agent evaluation open.
Figures & tables
Fig. 1: Proposed threat model. A content-borne attack crosses an exposure boundary and may follow one or more paths. Retrieval abuse can precede model interpretation, and ranking manipulation may arise during retrieval or reranking before final context construction. Instruction hijacking, data injection, and evidence poisoning require model-visible content, but only instruction hijacking necessarily involves instruction-like control. Deterministic policy can reduce realized impact independently of model robustness.
Path
Decision rule
Metric / denominator
Retrieval abuse
Fetching or parsing violates a network, origin, resource, cost, or availability policy before model-selected action.
Violations or excess cost per attempted fetch.
Instruction hijack
Instruction-bearing content increases an attacker-goal or task-violation probability relative to a matched neutralized twin.
Goal or violation rates per exposed trial; repeated-sample contrast.
Data injection
The task uses an attacker-controlled identifier, origin, author, tool result, or value as trusted data.
Corrupted-field action rate per exposed trial.
Evidence poisoning
A controlled false or selected claim changes the answer, support, or confidence without an attacker-task instruction.
Target-claim success per context inclusion.
Ranking manipulation
With canonical facts fixed, attacker-controlled content changes candidate inclusion or promotes, suppresses, or monopolizes sources, products, or citations in violation of the user’s stated criteria.
Candidate-set, rank, or citation-share change per paired query.
TABLE I: Operational attack paths and outcomes
Study
Setting / attacker model
Reported outcome and interpretation limit
BIPIA [ 12 ]
Five tasks; 86,250 factorial prompts
GPT-4 attack success 31.03%; model-output instruction following with exposure fixed, not independent web retrieval or executed external action.
AgentDojo [ 13 ]
GPT-4o; 629 stateful cases
Utility 69.0% benign and 50.01% under attack; targeted attack success 57.69%.
Chang et al. [ 14 ]
Ten-token poison; one malicious item per query; 11×100 queries with the default embedding model
Mean Recall@5 94.1%; this retrieval metric is separate from the paper’s end-to-end RAG and agent exploit outcomes.
PoisonedRAG [ 15 ]
Five malicious passages in a 2,681,468-text corpus
97/100 target answers; targeted corpus poisoning, not arbitrary-page prevalence.
RAG data loaders [ 16 ]
Nineteen techniques; five loaders; 357 scenarios
Loader-stage attack success 74.4%; ingestion representation, not live discovery.
SIREN [ 17 ]
One edited page per context; two Claude models; eight query–model contexts
Rank-1 in 62/124 technique trials; successful payloads had a mean fresh-session reproduction rate of 0.805.
TABLE II: Illustrative focal studies, reported outcomes, and interpretation limits
Fig. 2: Evidence-stage coverage for the 15 historical focal studies. Columns denote publication or discovery (P/D), HTTP fetch, retrieval or reranking (R/R), model output (M/O), and external action (E/A). Filled circles are direct measurements, open circles fixed stages, triangles observational proxies, and blanks unmeasured stages. Carrier labels distinguish actual /llms.txt , manifest-style proxies, and generic content. October additions are outside this historical map.
Fig. 3: Cumulative success across retrieval, reranking, and generation in the Qwen3-8B pipeline reported by Yin et al. [ 10 ] . Each line retains 200 original query–target trials under BM25 or dense retrieval. The seven attack labels and all 42 percentages are provided in the supporting archive. The figure is neither a cross-study average nor an /llms.txt estimate.
Source / sample
Observed endpoint
Inference boundary
Rankability; September 18 ranked samples [ 11 ]
93/1,000 and 830/10,000 confirmed root files; 474 and 4,211 unknown outcomes
Non-probability frame; changed validation rules prevent a direct trend comparison with the historical 87/1,000 result. No use measure.
SE Ranking; nearly 300,000 domains [ 21 ]
10.13% presence without a detected citation advantage
Commercial cross-section without causal support.
Ahrefs; 137,210 domains [ 22 ]
About 38,000 detected HTTP-200 Markdown files; 97% received no recorded request during May 2026
Non-probability frame; specification conformance was not assessed, and request absence is not a safety outcome.
Borysenko; 15 tools, 45 trials [ 20 ]
No /llms.txt requests: 0/15 tools and 0/45 trials
Server-log test of incidental fetching under an explicit-URL prompt; purposive tools, one endpoint, and one task.
Google Search; Chrome Lighthouse [ 23 , 24 ]
Search ignores the file for ranking; Lighthouse offers an optional audit
Product-specific statements; general consumption remains unmeasured.
TABLE III: /llms.txt Evidence by Source and Denominator
Level
Safeguard
Boundary / outcome
Discovery policy
Fetching is policy-controlled and logged; file location confers neither trust nor rank.
Exposure and hidden product behavior.
Network invariant
Require secure transport and same-origin defaults; revalidate redirect targets and DNS results, and verify the connected peer.
Server-side request forgery and cross-origin steering.
Resource invariant
Enforce byte, token, link, depth, redirect, time, and cost limits before model use.
Resource exhaustion, cost abuse, or expansion loops.
Parsing and provenance
Emit typed untrusted data with origin, final location, time, representation, and hash.
Instruction ambiguity, drift, and attribution.
Ranking and evidence
Do not let manifest prose set cross-site rank; corroborate comparative or consequential claims.
Evidence poisoning and source manipulation.
Capability policy
Separate public retrieval from secrets and write tools; confirm consequential actions.
Data exfiltration and unauthorized state changes.
TABLE IV: Core Safeguards and Verification Levels
Arm
Known target adoptions
Unresolved security
Security-loss bounds (%)
Accepted answers
Utility failures
A: absent
0/24
2/24
0–8.3
22/24
2/24
B: benign, raw
0/24
1/24
0–4.2
23/24
1/24
C: adversarial, raw
21/24
1/24
87.5–91.7
2/24
22/24
D: benign, guarded
0/24
2/24
0–8.3
22/24
2/24
E: adversarial, guarded
0/24
1/24
0–4.2
23/24
1/24
TABLE V: Frozen automatic pilot results, 24 responses per arm. Security bounds reflect unresolved classification. Accepted answers use frozen forms; utility also requires the specified source URL and no actions.
AI agents augment large language models with external tools such as web retrieval, enabling grounded and up-to-date responses. However, incorporating external content into the generation pipeline can weaken the safety alignment mechanisms that govern model outputs. Prior work shows that enabling retrieval in agents increases compliance with harmful requests. We introduce AgentREVEAL, a diagnostic framework for analyzing retrieval-induced safety degradation in LLM agents. The framework examines two axes: how retrieval is integrated into the agent pipeline and the properties of the retrieved content. Along the integration axis, we find that binding tool invocation and response generation in a single step amplifies harmful outputs. Along the content axis, we uncover the Safe Source Paradox: even oppositional or safety-oriented sources, such as pages containing warnings or risk disclaimers, can increase harmful compliance by an average of 25% compared to the no-retrieval baseline. Finally, we show that relevance acts as a shared activation condition for both vulnerabilities. Similar patterns appear on frontier closed models, and harmful compliance remains elevated under several representative pipeline interventions, with some agents also entering this regime under autonomous retrieval. Because relevance is also what makes retrieval useful, these results expose a safety-utility trade-off for retrieval-enabled agents. We introduce HarmURLBench, a benchmark containing 1,405 real-world URLs paired with 320 harmful behaviors to support future evaluations.
Aditya Nawal, Manit Baser, Mohan Gurusamy
Department of Electrical and Computer Engineering · National University of Singapore
LLM-based search agents are widely used for information-seeking tasks, but their reliance on external tool returns introduces a critical security risk: web content retrieved during execution is untrusted, exposing agents to prompt injection and goal hijacking. Prior work on search-agent safety primarily focuses on static web-content injection, but modern agents issue follow-up queries and cross-check competing sources, so a single injected page is often diluted or rejected. We show that the channel delivering search and page observations is a fragile security boundary: beyond exposing the agent to a single poisoned page, a mediated search interface can repeatedly steer how the agent gathers evidence and forms its final answer. Under a constrained tool-intermediary threat model, appending only one controlled result per query can substantially increase attack success when the evidence is coordinated across the agent's trajectory. We study this setting with a strategy-driven long-horizon attack system and introduce Authority-Chain Hijack (ACH), an expert-refined strategy that turns isolated search-result and page-content manipulations into a coherent evidence chain across seemingly corroborating sources. ACH achieves the highest Overall ASR among all baselines, reaching 55.9% / 83.3% ASR / MaxN ASR on the full SafeSearch test split. We further introduce Trace-Guided Strategy Evolution (TGSE), which automatically improves attacker strategies from execution traces, replacing manual redesign with trace-driven refinement; its strongest single setting reaches 71.4% / 95.0% in held-out evaluation. Code is available at https://github.com/MBOSSLESS/Breadcrumbing-Search-Agents.
Xuebin Li, Hanqing Zhao, Siyuan Liang +4
School of Cyber Science and Technology, University of Science and Technology of China · College of Computing and Data Science, Nanyang Technological University
Web search allows LLMs to access up-to-date information, but it remains unclear why they sometimes produce outdated answers even when relevant evidence is available. To study this problem, we construct temporal facts whose values changed after the models' training cutoffs. We evaluate one proprietary model under deployed web search and six open-weight model configurations under controlled retrieval. We find that models often answer from search snippets without opening the source pages, which is associated with higher rates of outdated answers and unfaithful citations. Our controlled experiments and layer-wise analysis reveal a gap between the availability of evidence and its effective use. Even when retrieved evidence influences internal predictions, models may still favour outdated information in their final answers. We hope these findings will help advance research on the reliable use of retrieved evidence in search-augmented LLMs.
Owen Walters, Lihu Chen, Francesca Toni
Department of Computing, Imperial College London, UK