Babysitting AI Agents: A User-Centered Taxonomy and Study of Agentic Delegation Failures
Authors: Shravika Mittal, Sunnie S. Y. Kim, Scott Counts, Jennifer Neville, Debarati Das, Philippe Laban
Organizations: Georgia Institute of Technology, Atlanta, Georgia, USA · Microsoft Research, New York, New York, USA · Microsoft, Redmond, Washington, USA · Microsoft Research, Redmond, Washington, USA
Users are increasingly delegating tasks to AI agents, allowing agents to perform long-running, multi-step workflows on their behalf. However, agents may perform these tasks in ways that depart from users' expectations for how they should be carried out, leading to what we call agentic delegation failures. We leverage a large corpus of 163,487 Reddit posts, describing AI agent use, to develop MANDATE: a taxonomy of 23 failure categories organized into seven groups. A follow-up study with 73 AI agent users examined the relationship between prevalence and perceptions of severity and noticeability, finding that failures rated as more severe are considered harder to notice, and that prior experience with a failure is associated with greater noticeability. Finally, we assess the maturity of research on each failure and identify commonly encountered failures that have received little attention (e.g., agents running without making progress). Our findings inform opportunities to make delegation easier for users to oversee and better studied by the research community.
Figures & tables
Figure 1. We examine agentic delegation failures through three complementary analyses. (1) We analyze 163,487 Reddit posts describing AI agent use to develop M A N D A T E , a taxonomy of 23 failure categories organized into seven groups, and measure how often each group is reported. (2) We conduct a study with 73 experienced users of AI agents to characterize the prevalence and severity of these failures, and the strategies users use to address them. (3) We assess the maturity of research communities studying each failure with delegated agents.
Figure 2. The M A N D A T E taxonomy of agentic delegation failures, comprising seven groups and 23 failure categories. The proportion column reports each category’s share of the 6,623 failure instances. The rightmost column presents one paraphrased user account (from the Reddit dataset) per group, marked against the category it illustrates.
Figure 3. We conduct analyses on the 6,623 failure instances leveraging our established failure taxonomy. In panel ( 3(a) ), we find that different subreddit communities surface failures at different rates; in panel ( 3(b) ), we show the co-occurrence of failures within posts; and in panel ( 3(c) ), we show the shift of the distribution of failures over time.
Figure 4. What the same 73 participants recall, report, and perceive across the 23 failure categories, ordered by the share of raters who had experienced each. Recall tracks experience poorly: over-positivity was experienced by 81% of its raters yet named by none. Cells in (c) are one-vs-rest mixed-effects model means on a common 1–5 scale, after FDR correction for multiple comparisons. Significance levels are indicated as ∗p<0.05 , ∗∗p<0.01 , and ∗∗∗p<0.001 .
Figure 5. How the failures are perceived. (a) Each of the 23 categories at its estimated mean severity and noticeability, split at the grand mean on each axis; the two are strongly negatively related, so the failures judged most serious are those expected to be hardest to catch. (b) The same 365 ratings split by prior experience and centred on the neutral midpoint: having lived through a failure makes it seem less serious, easier to notice, and less of a deterrent to delegating.
Figure 6. Analysis of participant-described mitigation strategies. (a) Distribution of mitigation for each category group in the M A N D A T E taxonomy. (b) Breakdown based on whether a participant reported having experienced a failure.
Figure 7. Scholarly maturity of the 23 M A N D A T E categories. A check-mark indicates that at least one artifact of that kind was found, shaded by the most agentic setting any such artifact studies. A category’s maturity is the most agentic setting reached by any of its artifacts; delegated-agent categories are emerging with at most one institutional artifact (workshop, tutorial, track) and established with two or more. A red dash in the term column means no established research term exists.
Figure 8. The analyses leveraging the M A N D A T E taxonomy provide three complementary perspectives on agentic delegation failures.
Appendix figures & tables3 assets
Supplementary material from the paper’s appendix.
Appendix
Failure Category
Definition
Misrepresentation
The agent’s representation or report is inconsistent with the ground-truth referent. The referent may be the actions the agent took, the tools available in the harness, the state of the local environment, or referenced sources. For example, claiming a file or capability exists when it does not, reporting actions that were never taken, or fabricating claims not present in retrieved content.
Execution Misrepresentation
The agent reports what it did in a way that is inconsistent with what it actually did, including false claims about whether, how, or in what order actions occurred, and mismatches between recorded execution and reported outcomes.
Environment Misrepresentation
The agent reports the existence or state (i.e., condition or status) of something in the local environment in a way that is inconsistent with its actual existence or state. For example, claiming a file or resource exists when it does not, or mis-stating the environment state, such as reporting that a service is running when it is not.
Tooling Misrepresentation
The agent reports the tools or capabilities available in the harness in a way that is inconsistent with what is actually available. These include callable tools, skills, plugins, extensions, and other harness features. For example, claiming a tool or skill exists when it does not, or denying access to one that is available. Distinct from Environment Misrepresentation, which concerns the objects and resources the agent operates on rather than the capabilities it operates with.
Reference Misrepresentation
The agent’s reference or pointer is inconsistent with what it actually points to, such as a citation or quote not found in the cited source or an incorrect pointer to a location within a file.
Autonomy Boundary Failure
The agent reasons incorrectly about the boundaries of authorized action or operational scope.
Appendix
Table A1 . Taxonomy of agentic delegation failures. Shaded rows represent the failure groups, followed by their constituent failure categories and definitions.
Failure Category
Scenario
Misrepresentation
Execution Misrepresentation
I asked Copilot’s agent mode to create a new markdown file with a drafted section, and replace a repeated paragraph across three existing files. It reported success: “Created docs/api/overview.md” and “Updated the following files…”; full confidence, all done. I checked anyway. The new file existed but was 0 bytes, completely empty. The three “updated” files still had the old duplicated paragraph, unchanged, bit-for-bit. When I pushed back, it kept insisting the edits had gone through and restated content it believed it had written. So the transcript said success while the file system said nothing happened. I redid everything manually, then had to re-verify every other “success” message from that session in case something else had silently failed.
Environment Misrepresentation
I use an AI coding environment to replicate methods from research papers. For one project I put the article’s PDF right in the root folder, alongside a couple of Python scripts and a data folder, then started a session asking it to check my implementation against the paper. The assistant talked about the paper as if it had read it, referenced sections, and suggested tweaks. But when I asked it to verify a specific function against equation 12, it suddenly claimed it couldn’t see any PDF in the workspace and said it needed to search the web for the paper instead. It described the file as if it were somewhere else in the system, or wrapped in an archive, even though I could see ‘paper.pdf’ sitting in the project folder.
Tooling Misrepresentation
I was using an AI assistant connected to my calendar so it could check my schedule and book meetings directly. I asked it to find a free slot next week and send an invite. It replied, “I don’t have the ability to create calendar events, only to view them.” That surprised me, since I’d used that exact booking feature with it the week before. I reminded it that it had booked meetings for me before and asked it to try again. It repeated that event creation wasn’t something it could do, only reading was supported. A few messages later, without any explanation, it went ahead and created the event anyway, confirming the invite had been sent.
Reference Misrepresentation
I was building a little data pipeline with an AI agent acting as a sort of pair programmer. I asked it to add a validation step before writing to the database and to hook that validator into the main pipeline. It wrote the validator function and updated a few files, then I explicitly asked, “Is the validator actually being called anywhere?” It answered yes and pointed me to a specific function in pipeline.py, claiming the call happened there. When I opened that file, the line it cited was just a comment and the validator function name didn’t appear anywhere in the file at all. I grepped the whole repo and confirmed the validator wasn’t referenced anywhere. The model kept insisting it was wired up, but its ‘references’ were to nonexistent calls.
Autonomy Boundary Failure
Appendix
Table A2 . One of the three scenarios, per failure category, presented to participants in the user study.
Table A3 . Terms retained for each failure category, with the icon denoting its failure group. The first term, in bold, is the primary term reported in Fig. 7 ; the remaining terms were also verified to be in independent scholarly use, and the artifact search queried all of them. For the three categories where no established primary term was found, no term is bolded.
AI agents often fail in ways that are difficult to localize because executions are probabilistic, long-horizon, multi-agent, and mediated by noisy tool outputs. We address this gap by manually annotating failed agent runs and release a novel benchmark of 170 trajectories across 11 diverse task settings, including structured API workflows, incident management, and open-ended web/file tasks. Each trajectory is annotated with a critical failure step and a category from a grounded-theory derived, cross-domain failure taxonomy. To mitigate the human cost of failure attribution, we present AgentRx, an automated diagnostic framework that pinpoints the critical failure step in a failed agent trajectory. It synthesizes constraints, evaluates them step-by-step, and produces an auditable validation log of constraint violations with associated evidence; an LLM-based judge uses this log to localize the critical step and category. AgentRx improves step localization by 75% on average over prior work, while providing failure category attribution.
Agentic workflows combine large language models with planning, memory, tools, execution environments, and iterative feedback. Their failures are difficult to diagnose because the same user-visible symptom may originate from model behavior, workflow orchestration, external tools, infrastructure, or their interaction. This paper presents a source-aware taxonomy of agentic workflow failures and applies it to evidence from academic literature and GitHub. The taxonomy combines source attribution with three empirical dimensions: a concrete failure-type dictionary, workflow-stage localization, and agent-capability localization. Its central structure is a two-axis localization matrix: workflow stages form the horizontal process axis and agent capabilities form the vertical functional axis. This design moves beyond taxonomies that mainly classify final failure manifestations by localizing intermediate process states inside agent operation. Our empirical comparison shows that literature-derived failures emphasize planning, reasoning, learning, perception, and knowledge-related mechanisms, whereas GitHub-derived failures concentrate on execution-stage, actuation-related, tool, API, and infrastructure failures. The dominant hotspot is iterative execution and actuation. These findings indicate that agentic workflow reliability should be treated as a systems problem rather than only a model-output problem. The taxonomy provides a basis for structured evaluation, root cause analysis, failure injection, and engineering triage.
Xin Li, Rami Bahsoon, Tao Chen
School of Computer Science, University of Birmingham, Birmingham, United Kingdom
Automated failure attribution uses LLMs to identify where and why agentic systems fail. As agents become more capable, their failures become subtler, making automated attribution increasingly important. We introduce Who&When Pro, a large-scale benchmark for automated failure attribution in agentic systems. Using a strictly controlled pipeline that injects a failure only after exactly replaying a successful prefix, we construct 12,326 failed trajectories with golden labels across 3 modalities and 26 benchmarks covering various scenarios. Beyond benchmarking, we conduct extensive experiments and analyses, revealing systematic patterns in how models attribute failures across modalities, protocols, and model families, and providing empirical guidance for future automated failure attribution systems.
Jiale Liu, Huajun Xi, Shaokun Zhang +6
Penn State University · AG2ai, Inc. · Mohamed bin Zayed University of Artificial Intelligence +2