Information-flow control (IFC) is a promising approach to enforce policy in agentic AI systems. However, IFC suffers from label creep, a phenomenon where monotonically increasing security labels propagate through a system and progressively limit its utility. Label creep in agentic AI systems can be mitigated using two major strategies: (1) improved system design and (2) label declassification. In this paper, we characterize how far system design alone can mitigate label creep in agentic systems. Specifically, we measure how system topology, scale, and agent composition affect utility loss and security enforcement cost proxies for conservative IFC enforcement in a stateful agentic system simulator. We develop a pipeline to systematically vary these features and measure their impacts on label propagation. Our results demonstrate strong associations between structural graph features and our IFC utility loss metric (up to a Spearman correlation ∣ρ∣ of 0.96) and show that system size and tool allocation substantially affect how labels propagate. These findings identify design choices to reduce IFC utility loss, but they also demonstrate that system design alone cannot fully prevent utility loss due to monotonic label propagation. This motivates the development of label declassification mechanisms that aid these design choices and diminish IFC utility cost.
Figures & tables
Fig. 1: The task DAG, direct-communication baseline topology Gmin , and possible task executions resulting from different tool orderings.
Fig. 2: Gross utility loss and excess utility loss with increasing task complexity. Different topologies experience different levels of gross and excess utility loss, motivating this research. Mesh and Gmin overlap.
Fig. 3: Label budget sweep across topologies.
Fig. 4: Spearman correlation ρ between graph features and IFC utility loss for labeling budgets in [4, 10].
Fig. 5: Gross and excess utility loss by number of tool-calling agents with a sensitive tool budget b=10 . Solid lines show random allocation. Dashed lines show clustered allocation.
Fig. 6: Utility loss proxy vs. security enforcement cost proxy for tasks with 10 sensitive calls. Larger markers indicate more tool-calling agents.
Autonomous LLM agents processing mixed-confidentiality data face severe security risks from prompt injection attacks and reasoning errors. While dynamic Information Flow Control (IFC) provides structural security guarantees, traditional taint tracking permanently taints an agent's context upon reading unvetted data, severely restricting downstream utility. We present APPA (Agentic Permissions Policy Algebra), an IFC framework that resolves this usability bottleneck through engine-managed context branching and prospective acquisition enforcement. Before data acquisition occurs, APPA prospectively evaluates label descents and missing prerequisites, generating actionable remedy plans (Authorize, Accept). To inspect unvetted data without polluting the primary context, a label-seeded child trajectory is spawned, absorbing label descent locally and allowing a trusted sanitizer to return a bounded derivative to the unchanged parent. Governed by a two-monoid model over security labels and shared event logs, we formally prove parent label preservation and merge confinement. Finally, we evaluate APPA on a multi-turn tool-chaining benchmark across four models: it suppresses exfiltration (31%-50% down to 0%-7% attack success), and on three of the four, branching recovers a substantial share of the utility that taint tracking alone forfeits.
We take the position that agent security must be approached as a systems problem: the AI model powering the agent must be treated as an untrusted component, and security invariants must be enforced at the system level. Through this lens, efforts to increase model robustness (the dominant viewpoint in the community) are insufficient on their own. Instead, we must complement existing efforts with techniques from the systems security domain. Based on our experience as cybersecurity researchers in operating systems, networks, formal methods, and adversarial machine learning, we articulate a set of core principles, grounded in decades of systems security research, that provide a foundation for designing agentic systems with predictable guarantees. As evidence, we analyze eleven representative real-world attacks on agents and discuss how systems principles, if realized, could have prevented these attacks. We also identify the research challenges that stand in the way of implementing these principles in agents.
Mihai Christodorescu, Earlence Fernandes, Ashish Hooda +11
Google · University of California San Diego · University of Wisconsin–Madison +3
Multi-agent systems improve capability through task decomposition and role specialization, but these same mechanisms introduce an important safety blind spot: a harmful objective can be fragmented into locally plausible subtasks, allowing malicious intent to evade detection by any single agent. This is a growing social-impact challenge: systems handling sensitive information or consequential tools can turn routine delegation into unauthorized disclosure or unsafe action. We argue that this failure mode is better understood as a semantic information-flow problem than as a single-turn prompt classification task. To address this, we propose SafeFlow, a defense framework for multi-agent systems that formalizes malicious cross-agent propagation as a semantic information-flow problem. SafeFlow attaches structured semantic taints to root requests, propagates them through a dynamic collaboration graph, and performs workflow-level validation to reconstruct the global risk context before irreversible actions are committed. Evaluated on four benchmarks spanning prompt injection, jailbreak-based unsafe tool use, risky code execution, and harmful web-agent behavior, SafeFlow reduces attack success rates compared to undefended baselines and external defenses while retaining high benign task completion and a high paired safe--harm success rate. Our findings show that multi-agent systems still lack mechanisms for preserving risk semantics across delegation boundaries. This gap can turn routine delegation into privacy harms or unsafe actions that affect people and organizations. SafeFlow keeps this risk visible throughout the workflow, before it results in harm.
Haowen Dai, Zonghao Ying, Wenfeng Li +10
University of Nottingham Ningbo China · Beihang University · Shandong University +6