Willful Disobedience: Automatically Detecting Failures in Agentic Traces

Reshabh K Sharma; Shraddha Barke; Benjamin Zorn

cs.SEMar 25, 2026

Willful Disobedience: Automatically Detecting Failures in Agentic Traces

Authors: Reshabh K Sharma, Shraddha Barke, Benjamin Zorn

Organizations: University of Washington Seattle, Washington, USA · Microsoft Research Redmond, Washington, USA

Abstract

AI agents are increasingly embedded in real software systems, where they execute multi-step workflows through multi-turn dialogue, tool invocations, and intermediate decisions. These long execution histories, called agentic traces, make validation difficult. Outcome-only benchmarks can miss critical procedural failures, such as incorrect workflow routing, unsafe tool usage, or violations of prompt-specified rules. This paper presents AgentPex, an AI-powered tool designed to systematically evaluate agentic traces. AgentPex extracts behavioral rules from agent prompts and system instructions, then uses these specifications to automatically evaluate traces for compliance. We evaluate AgentPex on 424 traces from $τ^2$ -bench across models in telecom, retail, and airline customer service. Our results show that AgentPex distinguishes agent behavior across models and surfaces specification violations that are not captured by outcome-only scoring. It also provides fine-grained analysis by domain and metric, enabling developers to understand agent strengths and weaknesses at scale. The source code of AgentPex is available at https://github.com/microsoft/agentpex.

Figures & tables

Figure 1. The agent reaches the correct final outcome but violates the transition, output, and predicted-plan specifications. AgentPex flags these violations.

Figure 3. The AgentPex pipeline from raw trace import through specification extraction to evaluation and aggregate scoring.

Figure 5. Traces sorted by AgentPex aggregate (low to high) alongside the $\tau^{2}$ composite score. Low- $\tau^{2}$ traces concentrate among low AgentPex scores, showing agreement without ground truth.

Figure 6. Per-metric scores across models. Similar metric profiles confirm that findings from Claude generalize to GPT-4.1 and o4-mini.

Figure 7. Per-metric scores by domain. Predicted plan reflects telecom difficulty.

Figure 8. ROC and Precision–Recall curves for output_spec when classifying $\tau^{2}$ failures. Red dots indicate candidate score thresholds (50–80). ROC-AUC = 0.680. Dashed lines denote random (ROC) and base-rate (PR) baselines.

Metric	Claude 3.5 Sonnet	GPT-4.1	o4-mini
Overall Aggregate	57.6	62.6	62.9
Output Spec	63.6	65.0	66.9
Transition Spec	59.2	63.7	80.6
Predicted Plan Spec	81.0	80.1	77.0
Argument Groundedness	98.7	99.1	98.1
$\tau^{2}$ Reward	41.4	37.5	38.6

Table 1. Aggregate and per-metric scores by model.

Figure 9. $\tau^{2}$ composite and AgentPex aggregate scores are similar across models.

Computational Cost Metric	Cost
API calls	$\approx$ 9 LLM API calls
Total tokens	$\approx$ 77,621 ( $\sim$ 69k in / $\sim$ 8.5k out)
Wall-clock time	139 sec
Estimated cost	$0.019

Table 2. Average per-trace evaluation cost.

Explore similar work

AgentRx: Diagnosing AI Agent Failures from Execution Trajectories
Feb 2, 2026Shraddha Barke, Arnav Goyal, Alind Khare +3Agent Failure Analysis AI Agent Reliability
1Microsoft Research · 2Microsoft · 3UIUC
AgentEval: DAG-Structured Step-Level Evaluation for Agentic Workflows with Error Propagation Tracking
Apr 26, 2026Dongxin Guo, Jikun Wu, Siu Ming YiuAgent Failure Analysis AI Agent Evaluation
The University of Hong Kong · Stellaris AI Limited
AgentAssay: Token-Efficient Regression Testing for Non-Deterministic AI Agent Workflows
Mar 3, 2026Varun Pratap BhardwajAI Agent Evaluation Mutation Testing
Independent Researcher

Feb 2, 2026J/K move · Enter open · S save

AgentRx: Diagnosing AI Agent Failures from Execution Trajectories

Shraddha Barke, Arnav Goyal, Alind Khare, Avaljot Singh+2

1Microsoft Research · 2Microsoft · 3UIUC

Agent Failure Analysis AI Agent Reliability LLM Agent Evaluation AI Agent Benchmarks Fault Localization Agent Trajectory Analysis+1

Open paper AskPDF arXiv

AI agents often fail in ways that are difficult to localize because executions are probabilistic, long-horizon, multi-agent, and mediated by noisy tool outputs. We address this gap by manually annotating failed agent runs and release a novel benchmark of 170 trajectories across 11 diverse task settings, including structured API workflows, incident management, and open-ended web/file tasks. Each trajectory is annotated with a critical failure step and a category from a grounded-theory derived, cross-domain failure taxonomy. To mitigate the human cost of failure attribution, we present AgentRx, an

\textit{automated diagnostic framework}

that pinpoints the critical failure step in a failed agent trajectory. It synthesizes constraints, evaluates them step-by-step, and produces an auditable validation log of constraint violations with associated evidence; an LLM-based judge uses this log to localize the critical step and category. AgentRx improves step localization by 75% on average over prior work, while providing failure category attribution.

Abstract

Explore similar work

AgentRx: Diagnosing AI Agent Failures from Execution Trajectories

AgentEval: DAG-Structured Step-Level Evaluation for Agentic Workflows with Error Propagation Tracking

AgentAssay: Token-Efficient Regression Testing for Non-Deterministic AI Agent Workflows