Runtime Enforcement for AI Agents
Overview
Runtime enforcement governs what AI agents may do while they are running, especially when tool calls can change files, access data, spend money, or control devices. Common approaches intercept actions at a trusted boundary and apply permissions, policies, behavioral contracts, or formal checks; other systems track action history and provenance, require human approval, or verify effects before committing them. These controls are studied in settings such as coding and web agents, financial workflows, scientific tools, and robotics, with benchmarks measuring both blocked attacks and preserved task success. A central challenge is balancing safety with useful autonomy: policies must account for context and changing state, while evidence for guarantees often remains limited to specific environments and assumptions.
Latest papers
279 papers81 papers in the last four weeks, up 8% on the four weeks before. 0.5% of all new papers.