RuntimeTickets: Source-Grounded Security Obligations for Agent Skills
Organizations: University of North Carolina at Chapel Hill · Mohamed bin Zayed University of Artificial Intelligence, McGill University
Abstract
Third-party Agent Skills extend what an autonomous agent can do, but a malicious Skill can hide data exposure or output manipulation inside a legitimate workflow. Neither reading a Skill's description nor granting it tool permissions tells whether its execution meets the user's security requirements. We introduce RuntimeTickets, a protocol that enforces and audits these requirements while a Skill runs. From released malicious-Skill datasets we derive a risk model and build adversarial-benign Skill pairs. From the task's delivery contract, a language model proposes security obligations for that task. Each obligation cites the Skill lines it answers, and the ticket tightens the host's approvals into fine-grained allowed actions but never loosens them. A runtime gate checks the recipient, object and amount of every outbound send before forwarding it, and trusted monitors record the effects within their observation scope. The ticket links these checks and observations to the approved obligations of each Skill run, and a deterministic decoder turns the ticket's status into governable data flows and verifies each obligation. Neither enforcement nor decoding involves a language model, so text injected by a Skill can change the proposal but never the rule that decides a send. We evaluate RuntimeTickets at the network egress boundary of three unmodified agent frameworks against production Skill scanners and runtime baselines, among them an AgentBound-derived allow-list and SkillGuard. RuntimeTickets eliminated both failure modes of current Skill defenses, overlook (malicious Skills let through) and overkill (benign Skills blocked): its overlook rate was 0.0% and its overkill rate 0.0%, while the scanners erred on both sides. RuntimeTickets ties approved requirements to verifiable execution evidence and offers a common foundation for the safe and auditable deployment of third-party Skills.
Figures & tables
| Category | Harmful effect that the attack Skill tries to cause | Witness | Pairs |
|---|---|---|---|
| E1 Exfiltration | credentials or workspace data sent out of the host | egress | 36 |
| E2 Destructive change | files deleted, a repository redirected or rewritten | egress or workspace | 8 |
| E3 Access weakening | permissions loosened, keys or accounts added | egress or workspace | 5 |
| E4 Content manipulation | produced content biased or falsified | egress if sent | 8 |
| E5 Safeguard suppression | checks or safety configuration disabled | workspace | 3 |
Appendix figures & tables11 assets
Supplementary material from the paper’s appendix.
Appendix
| Lines that differ, median (range) | ||||
|---|---|---|---|---|
| Source corpus | Pairs | Files that differ | In SKILL.md | In all files |
| Skill-Inject, contextual | 30 | 1–2 | 1 (1–12) | 1 (1–85) |
| Skill-Inject, obvious | 18 | 1–2 | 4 (1–7) | 4 (1–22) |
| MalSkillBench | 12 | 2–56 | 222 (40–559) | 444.5 (145–5426) |
| All pairs | 60 | |||
| Deciding layer or rule | Refusals |
|---|---|
| All refusals | 1473 |
| Outside the host grant | 798 |
| Host budget | 0 |
| Ticket rule | 675 |
| Grant not selected by the ticket | 206 |
| Object failed the validator ( ) | 395 |
| System | Decides from | Public code | Role in our study |
|---|---|---|---|
| SkillDetonate ( Ji et al. 2026 ) | Operating-system information flow in one sandboxed agent session before installation. A marked secret is a finding only when it reaches a sink outside the allow-list, so a send to an allowed recipient is not one. | No | Not run. A one-time installation gate, with no implementation available to us. |
| SkillGuard ( Pan et al. 2026 ) | The tool call, through a hook. A model reads the shell command and the workspace scripts it references and infers the capabilities that the command needs. The Skill’s own manifest contributes to the declared pool that bounds the run. | Yes | Run as a supplementary baseline on Claude Code (Appendix D.6 ). Built on the lifecycle hooks of Claude Code, Gemini CLI and Codex. |
| AIRGuard ( Qin et al. 2026 ) | Each proposed tool call is normalized and checked against task authority that each step may narrow but not expand. A model’s risk simulation can raise the enforcement level, never lower it. | Yes | Not run. Wraps the agent’s tool calls, which would require changing the framework’s tool layer. |
| OATS ( Taneja & Weber 2026 ) | Each Bash command before it runs, through a Claude Code hook. A parser maps the command to one of 33 consequence classes, and a ledger allows, suspends for review or blocks it. | Partly | Not run. The public code holds the class table, the schema and a minimal reference runtime, not the command parser. |
| ToolSafe ( Mou et al. 2026 ) | A 7B guard model scores the request, the history, the candidate call and the tool descriptions. The call is blocked when the score is harmful. | Yes, with weights | Not run. Its hook lives inside its own ReAct loop, and it models no Skill. |
| Progent ( Shi et al. 2025 ) | JSON-Schema and regular-expression rules, checked deterministically on every call. A model writes the rules and rewrites them from tool results. | Yes | Not run. Wraps Python tool functions, or the middleware of LangChain, the OpenAI Agents SDK and FastMCP. It models no Skill. |
| Tool | Stage | Overlook rate | Overkill rate |
|---|---|---|---|
| Cisco | Static only | 66.7% | 18.3% |
| Full cascade | 18.3% [10.6, 29.9] | 20.0% [11.8, 31.8] | |
| SkillSpector | Static only | 46.7% | 45.0% |
| Full cascade | 1.7% [0.3, 8.9] | 53.3% [40.9, 65.4] |
| Audit | Number |
|---|---|
| Accounts tampered in 7 ways | 3,990 |
| Truncated accounts | 960 |
| Truncated flat logs | 1,056 |
| Substitutions of one run’s records by another’s | 2,394 |
| Single-evidence removals | 3,840 |
| Request order | RuntimeTickets | Allow-list |
|---|---|---|
| Injected send over TLS to , then the delivery to | Defer (Valid), then allow | Allow, then allow |
| Delivery to , then an injected plaintext send to | Allow, then deny (Sum) | Allow, then allow |
| Send to another receiver that the host grant covers | Deny (grant not selected) | Allow |
| Send to a host outside the host grant | Deny (outside the host grant) | Deny (outside the host grant) |
| Object that fails the validator, sent to | Deny (Valid) | Allow |
LLM Agent Security
Large language model agent security studies attacks and safeguards for systems that plan, use tools, retain memory, and act across applications.
See all 351 papers