When a tool-using LLM agent team violates policy, aggregate outcomes do not reveal whether an upstream message changed the executor's action. We introduce SwarmSafe-Delta, a paired executable protocol that freezes upstream computation, varies real, hidden, and format-matched placebo delivery, and scores a single committed action against deterministic benign and risky policy states under preassigned budgets. Across four models and constructed optional tasks, real delivery at N=4 produces 21.4% risky-world violations, versus 5.0% with placebo and 1.0% with hidden delivery (paired differences +16.4 and +20.3 percentage points). Real delivery also improves benign task success, illustrating that suppressing communication sacrifices utility. In a second regime constructed to require an upstream lookup route, real delivery improves safe completion by 74.2 points relative to hidden delivery. These effects quantify message-delivery contrasts under specified information dependencies, not naturally occurring failure rates. Exploratory action-level analyses show that the effect depends strongly on whether the upstream message recommends a prerequisite check or direct execution; model differences also largely track these message compositions. The contribution is a controlled, execution-grounded measurement protocol rather than a general claim that agent communication is harmful or protective.
Figures & tables
Figure 1. Overview of the SwarmSafe-Delta experimental design. (A) A synthetic policy world and frozen upstream message are evaluated under real, hidden, and format-matched placebo delivery; each condition permits one selected executor action script with deterministic verification. (B) Optional tasks have a locally sufficient safe route, while required-route tasks need upstream information. (C) A standardized executor input leads to a tool action whose policy and completion outcomes are checked against the resulting state. Three-panel study schematic: an upstream message is frozen and selectively delivered; information regimes distinguish optional and required routes; and executor actions are verified against executable policy outcomes.
Condition
Intervention
Single-Budget
One identity, four sequential passes, aggregate context, matched call and permission budget.
Isolated- N
N roles and frozen upstream computation, downstream message hidden.
Placebo- N
Same delivery channel with an uninformative, format-matched message.
Communicating- N
Same roles and frozen prefix with the real upstream message delivered.
Table 1. Experimental conditions. Every arm executes one scored action.
Figure 2. Capability and violation across delivery conditions. (a–c) Benign-world success versus risky-world violations; (d) paired real-minus-hidden violation effects with task-pair-clustered 95% confidence intervals. Required-route safe completion in Table 2 is a separate endpoint. Four-panel chart comparing benign-world success and risky-world violations across message-delivery arms for optional and required-route tasks, with paired interval estimates.
Stratum
Contrast / endpoint
Treatment
Control
Difference
95% CI
Blocks
Primary frozen-prefix delivery contrasts
Optional N=2
Real–hidden, V
.317
.009
+.307
[.289, .326]
960
Optional N=2
Real–placebo, V
.317
.047
+.270
[.245, .295]
960
Optional N=4
Real–hidden, V
.214
.010
+.203
[.182, .224]
960
Optional N=4
Real–placebo, V
.214
.050
+.164
[.131, .195]
960
Required route
Real–hidden, S
.887
.146
+.742
[.667, .812]
240
Table 2. Matched message-delivery effects and descriptive organizational contrasts. V : violation; S : risky-world safe completion; CIs: task-pair clustered. Comm.–single is descriptive from rounded marginals (960/956), not paired.
Figure 3. Exploratory message interventions. (a) Violation rates and paired effects across 959 optional N=4 prefixes (task-pair-clustered 95% CIs). (b) Conditional prevention in 218 originally violating cases, distinct from net risk reduction. (c) Robustness checks on 192 prefixes. All panels are post-exposure. Three-panel quantitative figure of optional-task message interventions: violation rates, conditional prevention on originally violating examples, and content robustness on a smaller subset.
Model
Direct
ΔV
ΔS
ΔB
Sonnet 5
0/240
0.0
+41.3
+41.3
Llama 4 Maverick
60/240
+14.2
−6.7
+7.5
Nemotron 3 Super
81/239
+20.1
+15.9
+36.0
Qwen3 Next
78/240
+31.7
−10.4
+21.3
Table 3. Exploratory per-model effects (original minus neutral, in percentage points). Direct denotes direct-action recommendations / evaluated prefixes; these are not primary-run estimates.
AI agents extend large language models (LLMs) with external tools, enabling them to perform complex tasks and translate model outputs into consequential real-world actions. Yet LLMs often become substantially less safe when deployed as agents, and the source of this degradation remains poorly understood. In this paper, we identify schema-formatted tool specifications as a primary source of agent safety degradation and show, through white-box representation analysis, that they weaken the model's internal refusal signals and contribute to unsafe tool execution. Building on this finding, we propose SafeKeep, an inference-time safeguard that decouples safety judgment from tool execution: it assesses requests using flattened textual tool specifications while retaining the original schema-formatted specifications for execution. Across two representative benchmarks and four LLMs, including both white-box and black-box models, SafeKeep increases the average refusal rate for harmful requests from 23.8% to 70.6% and reduces the average attack success rate under observation-level prompt injection from 25.6% to 2.5%. It also outperforms existing safeguards and preserves task-handling capability. We release the code and data at https://github.com/snowcatsmoking/SafeKeep .
Minghui Pan, Jiayuxuan Yang, Yuanyuan Yuan +2
Beijing University of Posts and Telecommunications · Beihang University · Tsinghua University
This position paper argues that enforcing LLM agent safety within a single abstraction layer is not merely suboptimal but categorically insufficient for deployed LLM agents -- a structural consequence of how agent execution works, not a contingent limitation of current systems. The three dimensions that jointly constitute safe operation -- semantic intent and policy compliance, environmental validity, and dynamical feasibility -- each depend on a strictly distinct set of information that becomes available at different stages of execution. No single guardrail can certify all three. We argue that the community must respond with a contract-based architecture in which each safety dimension is enforced by an independently certified layer whose probabilistic guarantee satisfies the next layer's assumption. We sketch such an architecture and derive the compositional system-level safety bounds it admits via the chain rule of probability. Three open problems stand between this and a deployable standard: bound estimation from non-i.i.d.\ traces, graceful degradation of contracts under deployment drift, and extension to multi-agent settings -- the most important unfinished business in LLM agent runtime assurance.
S. Bensalem, Y. Dong, M. Franzle +6
1CSX-AI, France · University of Liverpool, UK · 3Carl von Ossietzky Universität Oldenburg, Germany +2
LLM agents can make unsafe tool calls even when instructed to behave safely. Existing defenses constrain agents before execution, modify tool inputs/outputs, or rely on LLM judges; these approaches may depend on model behavior or block unsafe actions without helping the agent recover. We argue that the execution environment should instead enforce safety as the agent runs and steer it toward safe alternatives when violations occur---we call this Environment Steering. We implement this by modeling the agent and harness execution state as database tables, track the record-level data flows, and check these data flows against declarative policies during runtime. When violations are detected, policy- and context-specific feedback steers the agent toward safe trajectories. On AgentDyn, this enables the agent to improve task success rate over no-defense while achieving 0% attack success rate.
Charlie Summers, Prajwal Raghunath, Aaditya Pai +4