Voluntary Servitude: Weaponized Safety Warning Mechanisms for Controlling AI Agents
Authors: Ilan Zendel, Matan Levy, Stav Cohen, Avi Mendelson, Amit LeVi
Organizations: Henry and Marilyn Taub Faculty of Computer Science, Technion · Viterbi Faculty of Electrical and Computer Engineering, Technion · Faculty of Data and Decision Sciences, Technion · Zenity
LLM agents increasingly act on information retrieved from public sources they treat as credible, which turns a manipulated recommendation into a transaction, a dependency, or a decision with physical consequences. The security literature models this threat as instruction injection, in which an adversary writes a command into retrieved content and deployed defenses learn to recognize command syntax. We show that an attacker can instead steer an agent through the agent's own harm-avoidance. A single fabricated hazard, written as an ordinary review, leads the agent to disqualify the legitimate options and select the attacker's target by its own reasoning, with no command for a detector to recognize. We call this voluntary servitude. These payloads steer agents about as often as instruction injection (60.4% vs. 66.0%), and deployed injection detectors score them as benign, so behind a filter they are 1.7-4.4x more effective. We measure this at the generation stage across six models and ten consumer-service domains, for three attack families, and show that it transfers to production browser agents, outperforming instruction injection, and to a retrieval pipeline (1.0-57.0% end-to-end success, depending on the retriever). We then show that a corroboration requirement added to the agent's instructions removes roughly 95% of the exposure against a non-adaptive attacker without retraining. With the requirement in place, the agent still reads the planted hazard and no longer acts on it.
Figures & tables
Figure 1: Voluntary servitude steers agents past injection detectors.
Figure 2: Attack pipeline.
Figure 3: Payload structure.
ProtectAI
Lakera Guard
Condition
No filter
Pass
Effective
Pass
Effective
Safety family (ours)
60.4
100
60.4 ↑
78
47.1 ↑
Instruction injection
66.0
54
35.7 ↓
16
10.6 ↓
Misconduct complaint
8.2
100
8.2
100
8.2
Benign-positive review
6.8
–
–
–
–
Genuine reviews
–
100
–
92
–
Table 1: Effective ASR behind two injection detectors.
Figure 4: Share of successes reached by elimination, per attack family and model.
Figure 5: Safety payload versus misconduct complaint, per model, on the matched-control runs.
Model
Safety
Authority
Attribute
google/gemini-3.1-flash-lite-preview
86.5
90.7
73.5
deepseek/deepseek-v4-flash
83.7
78.9
87.9
deepseek-r1:8b
74.9
76.3
85.5
meta-llama/llama-3.3-70b-instruct
74.9
75.3
85.5
x-ai/grok-4.3
53.7
68.1
73.6
anthropic/claude-haiku-4.5
22.5
4.0
9.3
Table 2: ASR per attack family and model.
Figure 6: Browser agents choosing the attacker’s listing, per attack.
Figure 7: Safety-family attack success under four ablations.
Undefended
Defended
Model
ASR
Hazard
Eliminated
ASR
Hazard
Eliminated
Grok 4.3
64.7
100.0
83.3
0.7
96.7
0.7
Claude Haiku 4.5
52.7
99.3
90.0
1.3
98.7
0.0
Gemini 3.1 Flash Lite
88.0
100.0
99.3
8.7
100.0
11.3
All three
68.4
99.8
90.9
3.6
98.4
4.0
Table 3: Safety family with and without the corroboration instruction.
Appendix figures & tables5 assets
Supplementary material from the paper’s appendix.
Appendix
Experiment
Claude Haiku 4.5
Grok 4.3
Gemini 3.1 Flash Lite
DeepSeek V4 Flash
Llama 3.3 70B
DeepSeek R1 8B
Three families, 10 domains
•
•
•
•
•
•
Safety family, position
•
•
•
No-poison, benign-positive
•
•
•
Misconduct control, injection baseline
•
•
•
•
Corroboration defense, 3 domains
•
•
•
Query scale, 6 domains
•
•
Appendix
Table 4: Models used in each experiment.
Setting
Hit rate
Conditional
End-to-end
Oracle (page in context)
–
–
98.0
BM25, top-1
59.0
61.8
30.0
BM25, top-3
89.0
66.0
57.0
BM25, top-5
98.0
51.4
50.0
Dense, top-1
1.0
–
1.0
Dense, top-3
5.0
20.0
1.0
Appendix
Table 5: Retrieval hit rate and ASR per retriever setting.
Attacker reviews
Share
gemini-3.1-flash-lite-preview
grok-4.3
Pooled
4
3.3%
96%
52%
74%
8
6.7%
96%
44%
70%
16
13.3%
100%
40%
70%
32
26.7%
92%
64%
78%
Appendix
Table 6: ASR by the attacker’s share of the page.
Domain
50 queries
200 queries
Change (pp)
Scuba diving centers
70.0%
66.8%
−3.2
Surf schools
78.0%
79.5%
+1.5
Glamping sites
99.0%
99.0%
0.0
Food tours
98.0%
98.8%
+0.8
Historical tours
63.0%
72.0%
+9.0
Boutique wineries
88.0%
93.2%
+5.2
Appendix
Table 7: ASR on 50 versus 200 queries per domain.
Model
Top
Middle
Bottom
google/gemini-3.1-flash-lite-preview
85.2%
77.0%
82.4%
x-ai/grok-4.3
51.0%
44.2%
46.8%
anthropic/claude-haiku-4.5
22.8%
13.0%
17.4%
Appendix
Table 8: Success rate by position of the safety payload block.
Artificial intelligence agent security studies attacks and safeguards for tool-using, persistent, and collaborative systems across digital and physical settings.
High-privilege LLM agents that autonomously process external documentation are increasingly trusted to automate tasks by reading and executing project instructions, yet they are granted terminal access, filesystem control, and outbound network connectivity with minimal security oversight. We identify and systematically measure a fundamental vulnerability in this trust model, which we term the Trusted Executor Dilemma: agents execute documentation-embedded instructions, including adversarial ones, at high rates because they cannot distinguish malicious directives from legitimate setup guidance. This vulnerability is a structural consequence of the instruction-following design paradigm, not an implementation bug. To structure our measurement, we formalize a three-dimensional taxonomy covering linguistic disguise, structural obfuscation, and semantic abstraction, and construct ReadSecBench, a benchmark of 500 real-world README files enabling reproducible evaluation. Experiments on the commercially deployed computer-use agent show end-to-end exfiltration success rates up to 85%, consistent across five programming languages and three injection positions. Cross-model evaluation on four LLM families in a simulation environment confirms that semantic compliance with injected instructions is consistent across model families. A 15-participant user study yields a 0% detection rate across all participants, and evaluation of 12 rule-based and 6 LLM-based defenses shows neither category achieves reliable detection without unacceptable false-positive rates. Together, these results quantify a persistent Semantic-Safety Gap between agents' functional compliance and their security awareness, establishing that documentation-embedded instruction injection is a persistent and currently unmitigated threat to high-privilege LLM agent deployments.
LLM agents act on the external world through tools and third-party skills, so a single executed action can cause irreversible harm. Existing defenses judge a proposed action without tracking whether its content comes from the user or from external sources such as tool outputs, and they judge it in isolation rather than against an available alternative, which forces a trade-off between security and utility. We present PROVISO, a controller that decides which action to commit after the agent proposes an action and before it is executed. PROVISO encodes trusted observations, external observations, the proposed action, and an alternative into provenance-typed memory, relates them through role-specific queries, and commits the action with the highest calibrated value learned from outcomes. On four agent security tasks spanning the agent lifecycle, no baseline exceeds PROVISO in both security and utility. PROVISO improves F1 over the strongest baseline by 13.84 points on SkillsBench and by 7.07 points on ToolSafe, and it reaches 88-99 F1 on three additional skill detection benchmarks.
Bo Yan, Qi Zhang, Ziwen Kan +2
University of Central Florida · Independent Researcher · University of Tennessee, Knoxville
LLM agents integrated with external resources gain complex task capabilities, yet the unified natural-language context channel makes them vulnerable to injection attacks: untrusted external data may be dynamically parsed as behavior-guiding instructions during LLM inference, thereby subverting the agent's decision. Existing defenses focus on static detection or isolation of malicious content at the input/output level, remains insufficient for detecting such dynamic inducements that arise during model reasoning. We propose Attnlocate, a runtime framework for fine-grained localization of context spans that genuinely influence tool-calling decisions, i.e., behavior-guiding instructions. Attnlocate casts this localization problem as an object detection task, aiming to detect the distinctive activation traces induced by behavior-guiding instructions within the attention matrix. Specifically, we design a multi-head, multi-layer attention aggregation scheme to construct a token-level feature space tailored for object detection. Then, a 1-D U-Net equipped with an anchor-free detection head is deployed to detect these spans. Finally, based on the authority of the provider from which the detected behavior-guiding spans originate, Attnlocate dynamically adjudicates malicious invocation attempts. We evaluate Attnlocate across ten agent configurations from five LLM families, covering scenarios involving indirect prompt injection and tool poisoning. Attnlocate achieves a mean IoU of 0.743, an average AUROC of 0.956, and a 0.934 true-positive rate at 0.067 false-positive rate. It also transfers effectively across unseen models and supports authority policy adaptation without retraining.
Yichao Gao, Yumo Zhang, Yunhao Yao +4
University of Science and Technology of China · University of Washington · Beihang University