When Agents Act Unwatched: The Reduced-Supervision Paradox in Agentic AI

Hanjing Shi; Dominic DiFranzo

cs.CYAug 28, 2026

When Agents Act Unwatched: The Reduced-Supervision Paradox in Agentic AI

Authors: Hanjing Shi, Dominic DiFranzo

Organizations: Department of Computer Science and Engineering Lehigh University, Bethlehem, PA, USA

Abstract

Agentic AI is sold on a simple promise: the system keeps acting when the user stops watching. That promise creates an accountability inversion. As stepwise supervision recedes, verification does not disappear; it moves into the runtime infrastructure that defines authority, records action, interrupts execution, checks outcomes, and supports repair. We call this the reduced-supervision paradox. Using a 63-artifact audit, we examine its public visibility across 46 research papers and 17 engineering, documentation, security, and governance sources. We find that agents' action surfaces are far easier to reconstruct than the mechanisms needed to answer for their actions. Tool mediation and monitoring traces were clearly visible in 40 and 37 artifacts, whereas checkpoint placement was clearly visible in 6, validator independence in 4, recovery in 2, and contestability in 1. Three action paths show why this imbalance matters. A repository path can preserve rich diffs after a consequential change. A browser path can cross organizational boundaries faster than permissions travel. A service path can follow policy while leaving affected people without recourse. We argue that observability can become a substitute for accountability when it shifts verification onto users after meaningful intervention is no longer possible. Our action-path diagnostic instead asks whether a delegated action remains connected to authority, evidence, interruption, independent judgment, recovery, and challenge. The claim is deliberately bounded to public visibility; it does not establish the prevalence or effectiveness of undisclosed controls. We contribute an action-level account that recasts the harness from a technical wrapper into accountability infrastructure.

Figures & tables

Source class	Artifacts
Research papers	46
Technical documentation	6
Engineering notes	3
Governance frameworks	3
Security, survey, commentary, policy, and related research	5
Total	63

Table 1: Source composition of the 63-artifact corpus.

Function	Present	Partial	Unclear	Absent
Tool mediation	40	15	8	0
Monitoring or traces	37	23	3	0
Checkpoint placement	6	40	12	4
Validator independence	4	36	20	2
Recovery or rollback	2	38	20	2
Contestability	1	28	31	3

Table 2: Visibility of central harness functions in 63 public artifacts.

Stage	Public question	Failure when missing
Permission	Who authorized what scope?	Technical access substitutes for delegated authority.
Action	Which state change mattered?	Summaries hide the consequential transition.
Trace	What evidence connects request to effect?	Debugging records cannot support an account.
Checkpoint	Where can execution still be stopped?	Approval arrives after the decision that mattered.
Validation	What independent evidence checks the action?	The actor endorses its own reasoning.
Recovery	How can the effect be repaired?	Error documentation replaces remedy.

Table 3: Minimum public questions for an accountable action path.

Risk mode	Recognition rule	Runtime response to inspect
Autonomy creep	Delegated scope expands through defaults, remembered permission, or reduced prompts.	Scoped identities, expiration, revocation, and least-privilege tools.
Misplaced checkpoints	Review occurs before the wrong step or after practical irreversibility.	Action-class gates before consequential state change.
Displaced verification labor	A stakeholder must judge the result after evidence has moved into inaccessible traces or summaries.	Role-specific evidence and review points.
Self-reinforcing validation	Actor and validator share a model, context, assumptions, or evidence source.	Independent evidence, stopping authority, and disclosed shared assumptions.
Delayed escalation	Handoff occurs after uncertainty or cumulative risk should have altered the path.	Escalation triggers tied to uncertainty, stakes, and downstream effects.
Poor recovery pathways	A bad action lacks undo, repair, compensation, containment, or redress.	Recovery designed by action class and consequence.

Table 4: Diagnostic risk modes for action-path analysis. These categories support inspection and design; they are not prevalence estimates.

AI Accountability

AI accountability concerns how responsibility for AI decisions and harms is assigned, evidenced, challenged, and remedied.

See all 157 papers

Feb 18, 2026

Overseeing Agents Without Constant Oversight: Challenges and Opportunities

To enable human oversight, agentic AI systems often provide a trace of reasoning and action steps. Designing traces to have an informative, but not overwhelming, level of detail remains a critical challenge. In three user studies on a Computer User Agent, we investigate the utility of basic action traces for verification, explore three alternatives via design probes, and test a novel interface's impact on error finding in question-answering tasks. As expected, we find that current practices are cumbersome, limiting their efficacy. Conversely, our proposed design reduced the time participants spent finding errors. However, although participants reported higher levels of confidence in their decisions, their final accuracy was not meaningfully improved. To this end, our study surfaces challenges for human verification of agentic systems, including managing built-in assumptions, users' subjective and changing correctness criteria, and the shortcomings, yet importance, of communicating the agent's process.

Madeleine Grunde-McLaughlin, Hussein Mozannar, Maya Murad +3

University of Washington, United States · Microsoft Research, United States

Human Oversight of AI Agentic AI Human-in-the-Loop Evaluation Explainable Artificial Intelligence

Ask PDF

Apr 7, 2026

Auditable Agents

LLM agents call tools, query databases, delegate tasks, and trigger external side effects. Once an agent system can act in the world, the question is no longer only whether harmful actions can be prevented--it is whether those actions remain answerable after deployment. We distinguish accountability (the ability to determine compliance and assign responsibility), auditability (the system property that makes accountability possible), and auditing (the process of reconstructing behavior from trustworthy evidence). Our claim is direct: no agent system can be accountable without auditability. To make this operational, we define five dimensions of agent auditability, i.e., action recoverability, lifecycle coverage, policy checkability, responsibility attribution, and evidence integrity, and identify three mechanism classes (detect, enforce, recover) whose temporal information-and-intervention constraints explain why, in practice, no single approach suffices. We support the position with layered evidence rather than a single benchmark: lower-bound ecosystem measurements suggest that even basic security prerequisites for auditability are widely unmet (617 security findings across six prominent open-source projects); runtime feasibility results show that pre-execution mediation with tamper-evident records adds only 8.3 ms median overhead; and controlled recovery experiments show that responsibility-relevant information can be partially recovered even when conventional logs are missing. We propose an Auditability Card for agent systems and identify six open research problems organized by mechanism class.

Yi Nian, Aojie Yuan, Haiyue Zhang +7

University of Southern California · Arizona State University · University of Tennessee, Knoxville +1

AI Accountability AI Agent Auditing LLM Auditing AI Agent Security

Ask PDF

Jun 8, 2026

Observability for Delegated Execution in Agentic AI Systems

Delegation-scoped execution is not identifiable from standard observables: audit logs and execution traces can be identical under multiple incompatible delegation assignments. This gap is especially acute in LLM-based agentic systems, where agents dynamically select tools, vary execution sequences across runs for the same instruction, and spawn cooperating sub-agents. These dynamics fragment and interleave traces, making delegation-scoped reconstruction from causal structure alone structurally underdetermined. Although individual actions are authorized and logged, existing audit, tracing, and security schemas lack the semantics to reconstruct what actions occurred under a given delegation across heterogeneous systems. We focus on delegation-scoped attribution and access/share footprint reconstruction, not intent inference or reasoning reconstruction. We present an agent-aware observability substrate consisting of a lightweight gateway and a common information model that binds delegation context at execution time. This enables reliable cross-tool delegation-scoped reconstruction and direct forensic queries without heuristic time-window correlation.

Abhinav Mishra, Kumar Sharad

Splunk, Cisco Inc

AI Agent Auditing AI Agent Monitoring Language Model Auditing

Ask PDF

Abstract

AI Accountability

Explore similar work

Overseeing Agents Without Constant Oversight: Challenges and Opportunities

Auditable Agents

Observability for Delegated Execution in Agentic AI Systems