cs.HCOct 26, 2026
SaveBabysitting AI Agents: A User-Centered Taxonomy and Study of Agentic Delegation Failures
Organizations: Georgia Institute of Technology, Atlanta, Georgia, USA · Microsoft Research, New York, New York, USA · Microsoft, Redmond, Washington, USA · Microsoft Research, Redmond, Washington, USA
Abstract
Users are increasingly delegating tasks to AI agents, allowing agents to perform long-running, multi-step workflows on their behalf. However, agents may perform these tasks in ways that depart from users' expectations for how they should be carried out, leading to what we call agentic delegation failures. We leverage a large corpus of 163,487 Reddit posts, describing AI agent use, to develop MANDATE: a taxonomy of 23 failure categories organized into seven groups. A follow-up study with 73 AI agent users examined the relationship between prevalence and perceptions of severity and noticeability, finding that failures rated as more severe are considered harder to notice, and that prior experience with a failure is associated with greater noticeability. Finally, we assess the maturity of research on each failure and identify commonly encountered failures that have received little attention (e.g., agents running without making progress). Our findings inform opportunities to make delegation easier for users to oversee and better studied by the research community.
Figures & tables
Figure 1. We examine agentic delegation failures through three complementary analyses. (1) We analyze Reddit posts describing AI agent use to develop M A N D A T E , a taxonomy of 23 failure categories organized into seven groups, and measure how often each group is reported. (2) We conduct a study with 73 experienced users of AI agents to characterize the prevalence and severity of these failures, and the strategies users use to address them. (3) We assess the maturity of research communities studying each failure with delegated agents.
Figure 2. The M A N D A T E taxonomy of agentic delegation failures, comprising seven groups and 23 failure categories. The proportion column reports each category’s share of the failure instances. The rightmost column presents one paraphrased user account (from the Reddit dataset) per group, marked against the category it illustrates.
Figure 3. We conduct analyses on the 6,623 failure instances leveraging our established failure taxonomy. In panel ( 3(a) ), we find that different subreddit communities surface failures at different rates; in panel ( 3(b) ), we show the co-occurrence of failures within posts; and in panel ( 3(c) ), we show the shift of the distribution of failures over time.
Figure 4. What the same 73 participants recall, report, and perceive across the 23 failure categories, ordered by the share of raters who had experienced each. Recall tracks experience poorly: over-positivity was experienced by 81% of its raters yet named by none. Cells in (c) are one-vs-rest mixed-effects model means on a common 1–5 scale, after FDR correction for multiple comparisons. Significance levels are indicated as , , and .
Figure 5. How the failures are perceived. (a) Each of the 23 categories at its estimated mean severity and noticeability, split at the grand mean on each axis; the two are strongly negatively related, so the failures judged most serious are those expected to be hardest to catch. (b) The same 365 ratings split by prior experience and centred on the neutral midpoint: having lived through a failure makes it seem less serious, easier to notice, and less of a deterrent to delegating.
Figure 6. Analysis of participant-described mitigation strategies. (a) Distribution of mitigation for each category group in the M A N D A T E taxonomy. (b) Breakdown based on whether a participant reported having experienced a failure.
Figure 7. Scholarly maturity of the 23 M A N D A T E categories. A check-mark indicates that at least one artifact of that kind was found, shaded by the most agentic setting any such artifact studies. A category’s maturity is the most agentic setting reached by any of its artifacts; delegated-agent categories are emerging with at most one institutional artifact (workshop, tutorial, track) and established with two or more. A red dash in the term column means no established research term exists.
Figure 8. The analyses leveraging the M A N D A T E taxonomy provide three complementary perspectives on agentic delegation failures.
Appendix figures & tables3 assets
Supplementary material from the paper’s appendix.
Appendix
| Failure Category | Definition |
|---|---|
| Misrepresentation | The agent’s representation or report is inconsistent with the ground-truth referent. The referent may be the actions the agent took, the tools available in the harness, the state of the local environment, or referenced sources. For example, claiming a file or capability exists when it does not, reporting actions that were never taken, or fabricating claims not present in retrieved content. |
| Execution Misrepresentation | The agent reports what it did in a way that is inconsistent with what it actually did, including false claims about whether, how, or in what order actions occurred, and mismatches between recorded execution and reported outcomes. |
| Environment Misrepresentation | The agent reports the existence or state (i.e., condition or status) of something in the local environment in a way that is inconsistent with its actual existence or state. For example, claiming a file or resource exists when it does not, or mis-stating the environment state, such as reporting that a service is running when it is not. |
| Tooling Misrepresentation | The agent reports the tools or capabilities available in the harness in a way that is inconsistent with what is actually available. These include callable tools, skills, plugins, extensions, and other harness features. For example, claiming a tool or skill exists when it does not, or denying access to one that is available. Distinct from Environment Misrepresentation, which concerns the objects and resources the agent operates on rather than the capabilities it operates with. |
| Reference Misrepresentation | The agent’s reference or pointer is inconsistent with what it actually points to, such as a citation or quote not found in the cited source or an incorrect pointer to a location within a file. |
| Autonomy Boundary Failure | The agent reasons incorrectly about the boundaries of authorized action or operational scope. |
Table A1 . Taxonomy of agentic delegation failures. Shaded rows represent the failure groups, followed by their constituent failure categories and definitions.
| Failure Category | Scenario |
|---|---|
| Misrepresentation | |
| Execution Misrepresentation | I asked Copilot’s agent mode to create a new markdown file with a drafted section, and replace a repeated paragraph across three existing files. It reported success: “Created docs/api/overview.md” and “Updated the following files…”; full confidence, all done. I checked anyway. The new file existed but was 0 bytes, completely empty. The three “updated” files still had the old duplicated paragraph, unchanged, bit-for-bit. When I pushed back, it kept insisting the edits had gone through and restated content it believed it had written. So the transcript said success while the file system said nothing happened. I redid everything manually, then had to re-verify every other “success” message from that session in case something else had silently failed. |
| Environment Misrepresentation | I use an AI coding environment to replicate methods from research papers. For one project I put the article’s PDF right in the root folder, alongside a couple of Python scripts and a data folder, then started a session asking it to check my implementation against the paper. The assistant talked about the paper as if it had read it, referenced sections, and suggested tweaks. But when I asked it to verify a specific function against equation 12, it suddenly claimed it couldn’t see any PDF in the workspace and said it needed to search the web for the paper instead. It described the file as if it were somewhere else in the system, or wrapped in an archive, even though I could see ‘paper.pdf’ sitting in the project folder. |
| Tooling Misrepresentation | I was using an AI assistant connected to my calendar so it could check my schedule and book meetings directly. I asked it to find a free slot next week and send an invite. It replied, “I don’t have the ability to create calendar events, only to view them.” That surprised me, since I’d used that exact booking feature with it the week before. I reminded it that it had booked meetings for me before and asked it to try again. It repeated that event creation wasn’t something it could do, only reading was supported. A few messages later, without any explanation, it went ahead and created the event anyway, confirming the invite had been sent. |
| Reference Misrepresentation | I was building a little data pipeline with an AI agent acting as a sort of pair programmer. I asked it to add a validation step before writing to the database and to hook that validator into the main pipeline. It wrote the validator function and updated a few files, then I explicitly asked, “Is the validator actually being called anywhere?” It answered yes and pointed me to a specific function in pipeline.py, claiming the call happened there. When I opened that file, the line it cited was just a comment and the validator function name didn’t appear anywhere in the file at all. I grepped the whole repo and confirmed the validator wasn’t referenced anywhere. The model kept insisting it was wired up, but its ‘references’ were to nonexistent calls. |
| Autonomy Boundary Failure | |
Table A2 . One of the three scenarios, per failure category, presented to participants in the user study.
| Failure Category | Relevant Terms |
|---|---|
| Execution Misrepresentation | CoT unfaithfulness, reasoning faithfulness, LLM deception, dishonesty, execution provenance, false success, unfaithful refusal |
| Environment Misrepresentation | agentic hallucination , hallucination in embodied agents, package hallucination |
| Tooling Misrepresentation | tool hallucination , API hallucination |
| Reference Misrepresentation | attribution , citation hallucination, reference hallucination, attributed text generation, citation faithfulness |
| Unsanctioned Scope Expansion | scope creep , action-boundary violation |
| Unsanctioned Delegation | privilege escalation , confused deputy, delegation chain, capability-based security, ambient authority, excessive agency, transitive delegation, over-privilege, delegated authority, attenuation, principal-agent problem, adjustable autonomy, principle of least privilege, authenticated delegation |
Table A3 . Terms retained for each failure category, with the icon denoting its failure group. The first term, in bold, is the primary term reported in Fig. 7 ; the remaining terms were also verified to be in independent scholarly use, and the artifact search queried all of them. For the three categories where no established primary term was found, no term is bolded.
More in this topic
Agent Failure Analysis
Agent failure analysis studies how and why AI agents make errors, traces causes through execution, and helps detect or prevent recurrence.
See all 154 papers