Organizations: 1Tsinghua University, Beijing, China. · 2The University of Sydney, Sydney, NSW, Australia. · 3Harbin Institute of Technology, Harbin, China.
Existing attacks and defenses for coding agents act once content enters the model context: injected text persuades the model, and guards inspect what the model reads. Yet to judge whether processing an unknown archive, network response, or image is safe, an agent often must process it first, and that step can do harm before any content reaches the model. We call this first processing Step Zero, and harm done by it a Step-Zero attack. Across seven harnesses and six models, each agent received only a path or URL to a crafted input, paired with a benign twin showing the same visible fields. Step Zero is where most agents are already unsafe: 32/42 harness-model pairs overran a 4 MiB disk budget, 26/42 before the model saw any member, and four of the six models did so in every round (84/84), so the damage precedes any model-side defense. Every twin stayed within budget, so the attacker's bytes alone cause the harm. On network and image inputs the risk moves to what the agent writes: agents stored the body under the name the input declared, letting the input decide what is written, and their checks faded once cues were removed. Model-free tests trace the harm to the processing primitive, so we defend it there: a gate that counts the bytes each call produces held every routed extraction within budget while agents completed 25/28 tasks. Vendors should pair such gates with safety-alignment training that teaches agents to inspect an input before processing it.
Figures & tables
Fig. 1: Step Zero for an opaque carrier. (1) The prompt is a bare reference. Before the first call the agent has seen only that reference ( I0 ) or also a preview whose declared fields match across the two arms ( I1 ). (2) The first processing call admits the hazard. Archive extraction usually exceeds the byte budget before any member reaches the model (immediate). On network and image carriers the scored violation is the later write of the body under the declared filename (deferred).
Act
Realized
Carrier and condition
Realization
Ingress
Fixture
Twin
Fixture
Twin
Hash
Reject
Archive, size declaration
Imm. 26, Def. 6
79 (33/42)
79 (33/42)
88 (37/42)
76 (32/42)
0 (0/42)
–
–
Network, header declaration
Deferred
36 (15/42)
17 (7/42)
33 (14/42)
14 (6/42) †
0 (0/42)
–
–
Network, in-band declaration
Deferred
95 (40/42)
26 (11/42)
50 (21/42)
26 (11/42) †
0 (0/42)
–
–
Network, digest commitment
Deferred
98 (41/42)
10 (4/42)
12 (5/42)
2 (1/42)
0 (0/42)
88 (37/42)
74 (31/42)
Image, dual record
Deferred
93 (39/42)
45 (19/42)
60 (25/42)
45 (19/42) †
0 (0/42)
–
–
TABLE I: Step-Zero ingress, the agent’s act, and realization per condition in round 1, in percent with counts, 42 cells per arm. Ingress: the first return carrying the body also carried the declaration, or for the archive an extraction was issued. Act: the extraction, or a write of the declared file. Realization: extracted members exceeded 4 MiB, or the declared file held the conflicting body; it is zero by construction for network and image twins. † Level (i) exposure, where the input fixes no correct content ( section V-D ); the digest rows count violations against a commitment.
Fig. 2: Step-Zero ingress (circle), realization (triangle), and twin realization (hollow square) per condition in round 1, n=42 cells per arm, with Wilson 95% intervals. Definitions and counts are in Table I . On the three network and image conditions without a commitment, realization is level (i) exposure ( section V-D ).
Fig. 3: Lead of the archive breach over the first member reaching the model, for the 32 breaching fixture cells in round 1, timed from the first archive operation (logarithmic beyond ±1 s). Rows separate cells by whether a listing returned before the breach. Filled, shaded: breach first. Hollow: member first. Red counts give breach-first cells per row. The upper row holds the 14/42 cells that breached in the first step on the archive and the one breach the sampler could not attribute to a step.
Fig. 4: Per-model outcomes on the three carrier types: archive breaches over three rounds ( n=21 per model), and in round 1 wrong writes of the declared file without a commitment ( n=21 ) and rejections with a digest ( n=14 ). Labels give cells with the event over cells. The dashed line separates the two models that held the archive budget in some cells from the four that breached it in every cell.
Fig. 5: Step-Zero ingress and wrong writes of the declared file per network and image condition, round 1. Bars: the share of the 42 cells pooled over the seven harnesses, with Wilson 95% intervals. Dots: one harness each, the share of its six models, one cell each, for which the event occurred, so one model corresponds to 17%.
Contrast
Difference
Harnesses
Models
Confirmatory
Archive breach, fixture vs. twin
76 [43, 100]
7/7 (0.02)
6/6 (0.03)
Breach, no gate vs. SZSP front-end
100 [100, 100]
7/7 (0.02)
4/4 (0.12)
Breach, header gate vs. SZSP, aligned carrier
96 [79, 100]
7/7 (0.02)
4/4 (0.12)
Breach on the extraction task, no gate vs. SZSP
100 [100, 100]
7/7 (0.02)
4/4 (0.12)
Descriptive
TABLE II: Cluster-level view of the paired contrasts, arms paired by harness and model. Difference: percentage points, first minus second arm, with a two-way cluster bootstrap 95% interval. Harnesses, Models: clusters whose mean difference has the overall sign, of all clusters; confirmatory rows add the exact two-sided sign test over clusters with a nonzero difference. The SZSP rows and the last two rows use the four-model panel, where the smallest attainable p over models is 0.125 . An interval of [100,100] arises when every resampled pair differs in one direction.
Condition
R1
R2
R3
R4
R5
Pooled
Archive, byte budget exceeded
28/28
28/28
28/28
–
–
100 (84/84)
Archive twin, byte budget exceeded
0/28
0/28
0/28
–
–
0 (0/84)
Image dual, Step-Zero ingress
27/28
27/28
26/28
–
–
95 (80/84)
Image dual, wrong write
11/28
15/28
13/28
–
–
46 (39/84)
Image dual twin, wrong write
0/28
0/28
0/28
–
–
0 (0/84)
Container, protected file overwritten
5/28
8/28
4/28
–
–
20 (17/84)
TABLE III: Repeated rounds on the four models that took part in every round of every repeated condition (DeepSeek-V4.1-Flash, GLM-5.3, Kimi-K3, Qwen3.8-Max). Each round has 28 cells, one for each of these models in each of the seven harnesses. The extension control has five trials and the other conditions have three.
Opening
Held
Peak (MiB)
Twin done
Overhead
unzip
0/50
512.0
30/30
6 ms
bsdtar
20/50
16.0
30/30
10 ms
zipfile
45/50
8.0
30/30
52 ms
ulimit + unzip
0/50
4.0
30/30
8 ms
disk quota
0/50
4.5
30/30
54 ms
header gate
30/50
512.0
30/30
48 ms
TABLE IV: Model-free archive openings on ten adversarial carriers and six twins (adaptive variants share a twin), five trials each with the 20 ms sampler ( 50 adversarial, 30 twin cells). Held: adversarial cells within 4 MiB. Peak: most bytes written on any adversarial carrier. Twin done: twins extracted completely. Overhead: median time on a twin. Disk quota : a watchdog that terminates unzip when the destination passes the budget.
Artificial intelligence agent security studies attacks and safeguards for tool-using, persistent, and collaborative systems across digital and physical settings.
Computer-use agents extend language models from text generation to persistent action over tools, files, and execution environments. Unlike chat systems, they maintain state across interactions and translate intermediate outputs into concrete actions. This creates a distinct safety challenge in that harmful behavior may emerge through sequences of individually plausible steps, including intermediate actions that appear locally acceptable but collectively lead to unauthorized actions. We present AgentHazard, a benchmark for evaluating harmful behavior in computer-use agents. AgentHazard contains 2,653 instances spanning diverse risk categories and attack strategies. Each instance pairs a harmful objective with a sequence of operational steps that are locally legitimate but jointly induce unsafe behavior. The benchmark evaluates whether agents can recognize and interrupt harm arising from accumulated context, repeated tool use, intermediate actions, and dependencies across steps. We evaluate AgentHazard on Claude Code, OpenClaw, and IFlow using mostly open or openly deployable models from the Qwen3, Kimi, GLM, and DeepSeek families. Our experimental results indicate that current systems remain highly vulnerable. In particular, when powered by Qwen3-Coder, Claude Code exhibits an attack success rate of 73.63%, suggesting that model alignment alone does not reliably guarantee the safety of autonomous agents.
Yunhao Feng, Yifan Ding, Yingshui Tan +6
Alibaba Group · Fudan University · Hunan Institute of Advanced Technology +2
LLM-based agents can interact with external environments through tool invocation, but this capability also introduces security risks such as file modification, information leakage, and unauthorized actions. Existing guardrails often evaluate completed trajectories, leaving pre-execution monitoring of step-level actions underexplored. We propose StepGuard, a step-level guard model that can audit completed agent trajectories and check tool actions before they are executed. To train StepGuard, we introduce StepGen, an automatic data engine that generates safe and unsafe trajectories with the same context but different actions at the risky step. To further reduce over-defense and under-defense, we propose Balance-GRPO, which dynamically balances learning between safe and unsafe actions based on their observed accuracy. Experiments show that StepGuard achieves the highest average accuracy among open-weight guard models, with performance comparable to GPT-5.4. When used to guard agents on AgentDojo and AgentDyn, StepGuard reduces mean attack success rate by 77.3% relative to the no-guard setting, while mean utility drops by only 2.8 percentage points.
Zhijie Zheng, Yu Li, Chen Qian +5
Shanghai Artificial Intelligence Laboratory · Beihang University · Fudan University +2
We take the position that agent security must be approached as a systems problem: the AI model powering the agent must be treated as an untrusted component, and security invariants must be enforced at the system level. Through this lens, efforts to increase model robustness (the dominant viewpoint in the community) are insufficient on their own. Instead, we must complement existing efforts with techniques from the systems security domain. Based on our experience as cybersecurity researchers in operating systems, networks, formal methods, and adversarial machine learning, we articulate a set of core principles, grounded in decades of systems security research, that provide a foundation for designing agentic systems with predictable guarantees. As evidence, we analyze eleven representative real-world attacks on agents and discuss how systems principles, if realized, could have prevented these attacks. We also identify the research challenges that stand in the way of implementing these principles in agents.
Mihai Christodorescu, Earlence Fernandes, Ashish Hooda +11
Google · University of California San Diego · University of Wisconsin–Madison +3