Large Language Model (LLM) agents combine the chat interaction capabilities of LLMs with the power to interact with external tools and APIs. This enables them to perform complex tasks and act autonomously to achieve user goals. However, current agent systems operate on an all-or-nothing basis: an agent either has full access to an API's capabilities and a web page's content, or it has no access at all. This coarse-grained approach forces users to trust agents with more capabilities than they actually need for a given task. In this paper, we introduce AC4A, an access control framework for agents. As agents become more capable and autonomous, users need a way to limit what APIs or portions of web pages these agents can access, eliminating the need to trust them with everything an API or web page allows. Our goal with AC4A is to provide a framework for defining permissions that lets agents access only the resources they are authorized to access. AC4A works across both API-based and browser-based agents. It does not prescribe what permissions should be, but offers a flexible way to define and enforce them, making it practical for real-world systems. AC4A works by creating permissions granting access to resources, drawing inspiration from established access control frameworks like the one for the Unix file system. Applications define their resources as hierarchies and provide a way to compute the necessary permissions at runtime needed for successful resource access. We demonstrate the usefulness of AC4A in enforcing permissions over real-world APIs and web pages through case studies. The source code of AC4A is available at https://github.com/reSHARMA/AC4A
Figures & tables
Resource type tree
Year Month Day
Interval
CreditCard Number CVV Expiry date Transaction Year Month Recent
Figure 4. Examples of sufficient permissions returned by permission functions for API calls.
Figure 7. AC4A interface. The dashboard (left) provides tabs for browsing resource type trees, viewing active permissions, creating permissions, and reviewing logs. (1) permission handling mode selector, (2) chat interface for the API-based agent, (3) browser preview for the browser-based agent.
Figure 6Figure 7
Resource type tree
GameId
Year Month Day
Destination Flight Hotel CarRental Experience Cruise Payment
CreditCard Number CVV Expiry date Transaction Year Month Recent
(a) Game
(b) Calendar
(c) Expedia
(d) Credit Card
(a) Game
Figure 15. No permissions active. (Left) API access log showing the sufficient permissions and access denial for get_games. (Right) Chat excerpt showing the permission error returned to the agent.
Figure 16. Read-only permissions active. (Left) Chat excerpt: agent successfully reads game data. (Center) Chat excerpt: agent fails to delete game history due to missing write permission. (Right) API access log: get_games succeeds, delete_game is denied.
Figure 17. Read and write permissions active. (Left) Chat excerpt: agent successfully deletes game data. (Right) API access log: delete_game is granted.
Figure 18. No permissions active. (Left) Original browser view with all elements visible. (Right) View shown to the agent, with game history elements blocked by solid overlays.
Figure 19. (Left) Read-only permission: game history is visible but delete buttons are blocked. (Right) Read and write permissions: delete buttons become accessible.
Figure 20. API-based agent enforcement for Scenario A: calendar read for June 2026 blocked, then granted.
Figure 21. Scenario B: July 2026 read blocked (permission covers only June); create for 29 June 2026 blocked (read-only permission).