Why Agentic Workflows Fail: A Taxonomy Derived from Literature and Empirical Data

Xin Li; Rami Bahsoon; Tao Chen

cs.SEOct 12, 2026

Why Agentic Workflows Fail: A Taxonomy Derived from Literature and Empirical Data

Authors: Xin Li, Rami Bahsoon, Tao Chen

Organizations: School of Computer Science, University of Birmingham, Birmingham, United Kingdom

Abstract

Agentic workflows combine large language models with planning, memory, tools, execution environments, and iterative feedback. Their failures are difficult to diagnose because the same user-visible symptom may originate from model behavior, workflow orchestration, external tools, infrastructure, or their interaction. This paper presents a source-aware taxonomy of agentic workflow failures and applies it to evidence from academic literature and GitHub. The taxonomy combines source attribution with three empirical dimensions: a concrete failure-type dictionary, workflow-stage localization, and agent-capability localization. Its central structure is a two-axis localization matrix: workflow stages form the horizontal process axis and agent capabilities form the vertical functional axis. This design moves beyond taxonomies that mainly classify final failure manifestations by localizing intermediate process states inside agent operation. Our empirical comparison shows that literature-derived failures emphasize planning, reasoning, learning, perception, and knowledge-related mechanisms, whereas GitHub-derived failures concentrate on execution-stage, actuation-related, tool, API, and infrastructure failures. The dominant hotspot is iterative execution and actuation. These findings indicate that agentic workflow reliability should be treated as a systems problem rather than only a model-output problem. The taxonomy provides a basis for structured evaluation, root cause analysis, failure injection, and engineering triage.

Figures & tables

Fig. 1: Overview of the methodology. Literature evidence is used to derive the initial taxonomy, GitHub evidence is used to extend and stress-test it, and reliability controls are used to adjudicate the final coded dataset.

Protocol item	Specification
Databases and expansion	IEEE Xplore, ACM Digital Library, and arXiv; backward and forward snowballing from relevant studies.
Search string	( agent OR agentic OR LLM agent OR autonomous agent OR tool-use agent OR multi-agent OR workflow orchestration OR reflection ) AND ( failure OR fault OR error OR bug OR breakdown OR invalid output ).
Inclusion criteria	I1: studies or evaluates an agentic, tool-using, or multi-agent workflow; I2: reports failure phenomena, causes, symptoms, error patterns, or evaluation breakdowns; I3: provides evidence concrete enough to extract at least one failure episode.
Exclusion criteria	E1: traditional software defects without agentic workflow behavior; E2: pure model risks without workflow or tool-use context; E3: method-only papers with no identifiable failure evidence; E4: duplicate or superseded versions; E5: inaccessible full text.
Screening procedure	Title/abstract screening followed by full-text screening. Ambiguous records were retained until full-text inspection to avoid premature exclusion.

TABLE I: Literature retrieval and screening protocol.

Dataset	$n$	Source	Stage	Cap.	Type
GitHub	150	0.71	0.69	0.67	0.65
Literature	80	0.79	0.61	0.63	0.72

TABLE II: Independent human–human annotation agreement (Cohen’s

\kappa

). The 50 additional non-failure steps are not included in these failure-label rows.

Control	Implementation
Source attribution	Classify each episode as Agentic, Non-Agentic, or Mixed according to whether the primary responsibility lies in agent behavior, external conditions, or their interaction.
Failure-type coding	Select the most specific dictionary type supported by evidence; use general labels only when finer attribution is unsupported.
Matrix localization	Map each episode to one workflow stage and one agent capability to capture its intermediate process state.
Tool support	Use fuzzy keyword matching, dictionary matching, regular expressions, and structured LLM suggestions only as candidate labels; final statistics use adjudicated labels.
Reliability check	Report full-dataset human–LLM agreement as auxiliary evidence and independent human–human checks on 150 GitHub failures and 80 literature records (with 50 additional non-failure steps reviewed).

TABLE III: Annotation and reliability controls.

Fig. 2: Full taxonomy of agentic-workflow failures with three source-attribution categories, 16 failure families, 65 operational failure types, and a stage–capability matrix. Each paired count is (literature + GitHub), totaling 397 + 1,196 failure records. The top-level counts are Agentic (271+618), Non-Agentic (100+568), and Mixed (26+10); literature attribution in these displays includes auxiliary labels and is not a fully human-adjudicated source distribution. Family stage–capability tags describe dominant localization, not an additional type label. Matrix cells show primary stage (S0–S4) by capability (A–F) for 1,586 valid records (397 literature; 1,189 GitHub); the seven GitHub records without complete matrix labels remain in type-level counts. A dash denotes zero; the largest cell is S3

\times

D (827).

Decision point	Rule	Example consequence
Earliest responsible stage	Prefer the evidenced stage where the defect was introduced; otherwise use its first observable manifestation, not merely the final user-visible symptom.	Goal misunderstanding leading to wrong output is S1; wrong final synthesis after correct work is S4.
Stage boundary	Separate task understanding (S1), planning-time choices (S2), runtime action/feedback/recovery (S3), and final response construction (S4).	Bad tool choice in a plan is S2; correct tool with malformed runtime arguments is S3.
Capability boundary	Separate decisions and inferences from concrete actions, and separate knowledge/memory from monitoring and recovery.	Wrong algorithm choice is C; malformed shell command is D; context loss is B; repeated failed retry is E.
Source attribution	Use Agentic for avoidable agent behavior, Non-Agentic for external conditions outside direct control, and Mixed only when both sides materially contribute or evidence is underdetermined.	API outage is Non-Agentic; wrong arguments are Agentic; malformed tool response plus no validation is Mixed.
Specificity	Use the most specific supported failure type; use AF00, NF00, or MF00 only when the source is clear but evidence does not support a finer subtype.	Revisit general labels when logs, comments, or fixes reveal a concrete mechanism.

TABLE IV: Condensed boundary rules for applying the taxonomy.

GitHub-dominant types Failure type Lit.% GH% Gap External API/Tool Failure (NF03) 2.8 26.3 +23.6 Tool Misuse Failure (AF05) 2.8 14.9 +12.1 Non-Agentic Failure (General, NF00) 0.5 6.4 +5.9 Input Formatting Failure (NF14) 0.0 5.4 +5.4 Unsafe Code Execution (AF25) 0.5 5.2 +4.7 Constraint Modeling Failure (AF06) 3.5 5.9 +2.3

Literature-dominant types Failure type Lit.% GH% Gap Perception Interpretation Failure (AF31) 8.6 1.0 -7.6 Planning Failure (AF03) 8.1 0.7 -7.4 Generalization Failure (NF17) 4.3 0.1 -4.2 Plan–Execution Mismatch (AF04) 4.3 1.0 -3.3 Knowledge Acquisition Failure (AF13) 5.3 2.4 -2.9 Knowledge Representation (AF14) 3.8 1.0 -2.8

TABLE V: Largest source differences by failure type. Percentages are normalized within each source; gaps are GitHub percentage minus literature percentage.

Fig. 3: Comparison of literature-derived and GitHub-derived failures across workflow stages and agent capabilities. Labels indicate the GitHub-minus-literature percentage-point gap. The main marginal gaps are statistically reliable: the stage distribution differs by source,

{\chi^{2}}(4)=348.92

p<0.001

V=0.469

, with S3 showing a +46.6 pp gap, 95% CI [+39.4,+53.2], adjusted Fisher

p<0.001

; the capability distribution also differs,

{\chi^{2}}(5)=334.19

p<0.001

V=0.458

, with D: Actuation showing a +48.1 pp gap, 95% CI [+41.3,+54.1], adjusted Fisher

p<0.001

Panel of Figure 9 — Fig. 4: Stage–capability heatmaps for literature (left), GitHub (center), and the combined dataset (right). Each cell shows the number of failures localized to a workflow stage and capability pair.

Agent Failure Analysis

Agent failure analysis studies how and why AI agents make errors, traces causes through execution, and helps detect or prevent recurrence.

See all 153 papers

AgentEval: DAG-Structured Step-Level Evaluation for Agentic Workflows with Error Propagation Tracking
Apr 26, 2026Dongxin Guo, Jikun Wu, Siu Ming YiuAgent Failure Analysis AI Agent Evaluation
The University of Hong Kong · Stellaris AI Limited
Exploring the Semantic Gap in Agentic Data Systems: A Formative Study of Operationalization Failures in Analytical Workflows
Jul 1, 2026Jalal Mahmud, Eser KandoganAgent Failure Analysis Agentic Workflows
Megagon Labs California, USA
AgentRx: Diagnosing AI Agent Failures from Execution Trajectories
Feb 2, 2026Shraddha Barke, Arnav Goyal, Alind Khare +3Fault Diagnosis AI Agent Reliability
1Microsoft Research · 2Microsoft · 3UIUC

Apr 26, 2026J/K move · Enter open · S save

AgentEval: DAG-Structured Step-Level Evaluation for Agentic Workflows with Error Propagation Tracking

Dongxin Guo, Jikun Wu, Siu Ming Yiu

The University of Hong Kong · Stellaris AI Limited

Agent Failure Analysis AI Agent Evaluation Agentic Workflows Agent Evaluation

Open paper AskPDF arXiv

Agentic systems that chain reasoning, tool use, and synthesis into multi-step workflows are entering production, yet prevailing evaluation practices like end-to-end outcome checks and ad-hoc trace inspection systematically mask the intermediate failures that dominate real-world error budgets. We present AgentEval, a framework that formalizes agent executions as evaluation directed acyclic graphs (DAGs), where each node carries typed quality metrics assessed by a calibrated LLM judge (GPT-4o), classified through a hierarchical failure taxonomy (3 levels, 21 subcategories), and linked to upstream dependencies for automated root cause attribution. An ablation study isolates the impact of DAG-based dependency modeling: it alone contributes +22 percentage points to failure detection recall and +34 pp to root cause accuracy over flat step-level evaluation with identical judges and rubrics. Across three production workflows (450 test cases, two agent model families, predominantly sequential architectures with a 12% non-DAG trace rate), AgentEval achieves 2.17x higher failure detection recall than end-to-end evaluation (0.89 vs. 0.41), Cohen's kappa = 0.84 agreement with human experts, and 72% root cause accuracy against an 81% human ceiling. Cross-system evaluation on tau-bench and SWE-bench traces confirms transferability (failure detection recall >= 0.78) without taxonomy or rubric modification. A 4-month pilot with 18 engineers detected 23 pre-release regressions through CI/CD-integrated regression testing, reducing median root-cause identification time from 4.2 hours to 22 minutes and driving measurable failure rate reductions in two workflows.

Abstract

Agent Failure Analysis

Explore similar work

AgentEval: DAG-Structured Step-Level Evaluation for Agentic Workflows with Error Propagation Tracking

Exploring the Semantic Gap in Agentic Data Systems: A Formative Study of Operationalization Failures in Analytical Workflows

AgentRx: Diagnosing AI Agent Failures from Execution Trajectories