cs.AINov 5, 2026

The Power of Meta Access in the Software of the Future --- Demonstrated on LLM Chat UIs and Chat AI

Authors: Julian Kunkel, Ali Doosthosseini

Organizations: Universität Göttingen, GWDG · Universität Göttingen

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Chat remains the default surface for many large language model (LLM) tasks, even as agentic tools such as OpenCode excel at coding and automation. Yet the model sees the current dialogue but not the workspace around it. We explore meta access---giving the assistant a structured, permissioned view of the interface itself and propose a ladder of levels: feature tutoring; metadata, settings, and preference access; retrieval over summaries and messages, confirmed mutation, and full orchestration. Meta access can turn a chat client from many screens into one entry point for finding, navigating, organising, and operating it, while raising risks around grounding, privacy, prompt injection, and trust. General-purpose agentic harnesses are runtimes; because they can change their own settings, meta access becomes reflexive and needs a separate control-plane boundary. We ground this in the Meta Chat prototype of the Chat AI client. It builds bounded context from, e.g., summaries, settings, and relevant messages; links references to conversations, topics, and personas; supports confirmed, reversible changes; offers a browsable tutorial library; and recommends personas as user-started conversations. By exposing application state in structured form, meta access can extend screen readers from widget descriptions to semantic workspace operations. Meta access thus turns chat from a window onto one conversation into a control layer for the whole workspace, extending chat toward agentic capability without surrendering user control. As assistants become embedded in IDEs, dashboards, office suites, and operating systems, AI-era interfaces should expose a structured, permissioned model of their state and capabilities, ideally by opening their existing metamodels through a debugger-like interface.

Figures & tables

WorkDialoguePersonalMetadataContentInterfaceActionsGroundingSecurityUser study
Conversational search
Radlinski & Craswell [ 22 ]+--+--0--
Trippas et al. [ 25 ]+--+--0-+
Zamani et al. [ 32 ]+-0+--00-
Mo et al. [ 35 ]+00+--00-
Zhang et al. [ 36 ]+00+0+00-
Table 1: Comparative matrix of representative related work. + : explicitly addressed and central; 0 : partial, indirect, or limited; - : not addressed or out of scope. Ratings describe scope, not quality. This work implements selective message-level content retrieval and confirmed, reversible actions; the user-study column remains out of scope.
Figure 1: The tutorial library, opened from the Browse tutorials entry point in the Meta Chat banner. Tutorials are grouped by task and can be started directly; each runs a guided tour of the real controls.
Figure 2: Running a tutorial in the Meta Chat , captured from the local application. The tutorial status panel shows step 2 of 3, names the expected action, and highlights the prompt area. Each step waits for the corresponding interface action or message completion before advancing.
Figure 3: Routing a settings question to the interface, shown with synthetic data. The answer names the control, offers to open the settings panel, and starts a tour that highlights the gear icon. Navigation remains read-only; changing the value is a separate confirmed proposal.
Figure 4: Reading current preference values in the Meta Chat , shown with synthetic data. The answer prints the user’s memory settings, offers to open the corresponding settings page, and can propose a value change. A change is applied only after an explicit confirmation and can be reverted while the resulting state still matches.
Figure 5: Navigating conversations and topics in the Meta Chat , shown with synthetic data. The answer links to the cited conversation and to its topic; the sidebar highlights both targets so the user can see where the link will lead.
Figure 6: Persona recommendation and handoff in the Meta Chat , shown with synthetic data. The assistant recommends a persona from catalogue metadata and offers a start link; clicking it opens a new chat with the persona’s title, model, and system prompt. The Meta Chat itself never reads the persona prompt.
LCapabilityExampleRisk
0No meta access: only the current dialogueOrdinary chat interfacesNone new
1Feature tutoring: describe the interface and run tutorials“How do I attach a file?”Wrong instructions
2Metadata access across all conversations: titles, topics, settings, preferences, controls, and personas; navigation“Where is my food-order chat?”Wrong target; preference disclosure; misleading catalogue entry
3Conversation content: summaries and message-level retrieval“What did we decide about the budget?”Sensitive content exposure; stale summaries
4Confirmed mutating actions: edit chats, topics, or settings“Turn off automatic memory saving.”Unintended edits or setting changes
5Multi-step, autonomous orchestration“Tidy my topics and rename weak titles.”Compounding errors, loss of control
Table 2: A ladder of meta access. The current Meta Chat implements levels 1–3 and confirmed level-4 mutations for chats, settings, and memories, with limited composition across several proposals; full level-5 orchestration remains future work. The levels describe autonomy over workspace state; for self-configuration in general-purpose harnesses they are combined with a target class (Section 4.6 ).
Figure 7: Message-level retrieval in the Meta Chat , shown with synthetic data. The user asks where the “Fork-Join-Modell” was discussed; the answer identifies the chat OpenMP Basics for Beginners and offers exact-message citations that open and highlight the source.
Figure 8: Reading and changing per-chat settings in the Meta Chat , shown with synthetic data. The assistant reports the model and temperature of the LLM Evaluation chat, then proposes a model change in a warning card with an explicit Accept changes action.
Figure 9: Auditing model use across the workspace in the Meta Chat , shown with synthetic data. The assistant identifies chats still using an outdated model and presents their titles as links, supporting a later bulk update after the user reviews the affected conversations.
AreaExample meta featuresGuardrails
Message-level searchFind where a decision was made; compare conversations; extract decisions or action itemsCite exact messages and snippets; label summary versus full-text evidence; enforce workspace scope and permissions; mark extraction as inference; refuse comparisons when access is unclear
Navigation and tutoringOpen a setting; deep-link to a chat, message, or topic; highlight UI elements; explain unavailable featuresResolve live UI state and the capability registry; validate every deep link; highlight only real controls; state the reason for unavailability; never invent settings or routes
Persona recommendation and handoffRecommend a persona; explain categories; start a persona chatExpose catalogue metadata only; show the source; treat the definition as untrusted; import only on a user click; validate the model and settings before applying
Preference reading“What are my memory settings?”Read current local values; timestamp the snapshot; distinguish a value from its control; explain the implications of the value
Mutating actionsRename or move a chat; change a setting; delete dataUse a typed capability; show a preview or diff; require explicit confirmation; support undo; keep an audit log; grant least authority
Table 3: Guardrail contracts for meta-access capabilities. The same contract applies at the point where the capability is exposed to the model and at the point where the client executes or renders it.
More in this topic

Tool Access Control for LLM Agents

Tool access control for LLM agents limits which tools, resources, and actions agents may use, and under what conditions.

See all 234 papers

Explore similar work

CardsList
Oct 9, 2026

Web$hell: A Unified Interface for Agentic Web Access

Large language model (LLM) agents are becoming direct users of the open web. As they take on increasingly complex and information-intensive tasks, effective web access is becoming a core capability for agentic problem solving. Existing approaches have advanced agentic web access through different access models, including search-augmented agents, browser-based agents, and, more recently, filesystem-based agents. Nevertheless, as users expect agents to complete long-horizon, end-to-end tasks, they increasingly need to conduct broad information gathering, perform browser actions, and revisit acquired evidence within a single trajectory. Yet our preliminary analyses reveal that existing access models struggle to meet these demands. They primarily support either information gathering or browser interaction, while combining their capabilities can involve substantial costs or performance trade-offs. To this end, we present Web$hell, an interface that unifies web access under a single access model. Using a common schema defined by WebContract, WebCompiler compiles live webpages into addressable information and action objects. These objects are maintained in WebCache and handled through a shared set of operations, providing agents with consistent, operable, and persistent web access. Through extensive experiments across web search, web navigation, and deep research benchmarks, we show that Web$hell achieves strong task performance at lower cost. Our results further indicate the importance of interface design for effective web access, with Web$hell opening new possibilities for more capable and efficient LLM agents on the web.

Sunghwan Kim, Deogyong Kim, Sangam Lee +2

Yonsei University

Web AgentsLLM Agent HarnessesLong-Horizon LLM Agents
AskPDF
Mar 21, 2026

AC4A: Access Control for Agents

Large Language Model (LLM) agents combine the chat interaction capabilities of LLMs with the power to interact with external tools and APIs. This enables them to perform complex tasks and act autonomously to achieve user goals. However, current agent systems operate on an all-or-nothing basis: an agent either has full access to an API's capabilities and a web page's content, or it has no access at all. This coarse-grained approach forces users to trust agents with more capabilities than they actually need for a given task. In this paper, we introduce AC4A, an access control framework for agents. As agents become more capable and autonomous, users need a way to limit what APIs or portions of web pages these agents can access, eliminating the need to trust them with everything an API or web page allows. Our goal with AC4A is to provide a framework for defining permissions that lets agents access only the resources they are authorized to access. AC4A works across both API-based and browser-based agents. It does not prescribe what permissions should be, but offers a flexible way to define and enforce them, making it practical for real-world systems. AC4A works by creating permissions granting access to resources, drawing inspiration from established access control frameworks like the one for the Unix file system. Applications define their resources as hierarchies and provide a way to compute the necessary permissions at runtime needed for successful resource access. We demonstrate the usefulness of AC4A in enforcing permissions over real-world APIs and web pages through case studies. The source code of AC4A is available at https://github.com/reSHARMA/AC4A

Reshabh K Sharma, Dan Grossman

University of Washington, Seattle, Washington, USA

Tool Access Control for LLM AgentsLLM Agent SecurityWeb Agents
AskPDF
Jun 29, 2026

LUMOS: A Semantic Operating-System Layer for Accessibility-Grounded AI Agents

Current operating systems expose interfaces optimized for human users but not for AI agents. Humans benefit from pixels, icons, windows, visual grouping, mouse movement, and keyboard shortcuts; AI agents instead need compact semantic state, grounded actions, and reliable feedback. As a result, many computer-use agents are forced to interpret screenshots, OCR output, and visual crops, introducing high token costs, visual ambiguity, latency, and coordinate uncertainty. This paper introduces LUMOS (Language Model Unified Machine-Readable Operating-System Semantics), a semantic interaction layer between AI agents and operating systems. LUMOS converts native accessibility metadata and browser UI structures into machine readable semantic blueprints with stable identifiers, roles, names, values, bounds, and action affordances. It also supports live semantic pointer grounding by querying the UI element under or near the cursor through operating-system automation APIs. An LLM then acts through an accessibility grounded observe act loop using constrained visible-UI primitives rather than application-specific scripts. LUMOS does not claim to replace visual agents; instead, it reduces dependence on screenshots when operating systems already provide semantic structure. These results suggest a path toward AI-native operating systems and machine-readable interaction layers.

Yogeswar Reddy Thota

Department of Electrical and Computer Engineering University of Texas at Dallas Richardson, TX, USA

Computer-Use AgentsGUI Automation
AskPDF