Annotrace: A Workbench for Step-Level Annotation of Agent Traces
Authors: Chenyi Huang, Arun Josephraj Arokiaraj, Bin Wu, Sahan Bulathwela
Organizations: Department of Computer Science, University College London, London, United Kingdom · Centre for Artificial Intelligence, University College London, United Kingdom
Every study of why LLM agents fail begins by labelling traces by hand, and every such study builds its own throwaway tool. Observability platforms record agent traces but offer no structured labelling; annotation tools label text but have no notion of a step, a delegation or a tool call, leaving it unclear what the smallest unit of execution to label should be. Annotrace reads six trace formats, including OpenTelemetry GenAI, LangSmith and Langfuse, and turns each run into an ordered, nested list of steps. The annotator writes their own labelling codebook; a frontier model proposes a starting set from a sample of their own traces, answers first, and is asked exactly the questions the human is asked, making annotation a joint human-AI process. Annotrace measures agreement between any two annotators, human or AI. On a multi-agent question-answering corpus it reproduces a previous manual effort's 235 annotation units exactly, and two models labelling the same traces agree 91% of the time at κ=0.82.
Figures & tables
Figure 2: Annotrace workflow. Top: adapters convert six trace formats into nested steps. Middle: a model proposes an annotation schema and answers the same questions as the human annotator. Bottom: adapters can recover and separately label Thinking–Action–Response cycles embedded in a single text field.
Format
Step kinds
Nested
OpenTelemetry
invoke_agent, chat, execute_tool
yes
LangSmith
chain, llm, tool, retriever
yes
Langfuse
agent, generation, tool, span
yes
Chat messages
message, tool call, tool result
no
Who&When
message
no
Coordinator
decision, reply, cycle, output
yes
Table 1: The six adapters. Kinds are each format’s own vocabulary; a question can be restricted to any subset of them.
Figure 3: The annotation screen. The sidebar holds the run’s question and the agent’s answer, so both stay in view while the steps scroll. Colour marks the kind of step until a label is chosen, then marks the label. Steps still wanting an answer say so.
AI agents are increasingly embedded in real software systems, where they execute multi-step workflows through multi-turn dialogue, tool invocations, and intermediate decisions. These long execution histories, called agentic traces, make validation difficult. Outcome-only benchmarks can miss critical procedural failures, such as incorrect workflow routing, unsafe tool usage, or violations of prompt-specified rules. This paper presents AgentPex, an AI-powered tool designed to systematically evaluate agentic traces. AgentPex extracts behavioral rules from agent prompts and system instructions, then uses these specifications to automatically evaluate traces for compliance. We evaluate AgentPex on 424 traces from τ2-bench across models in telecom, retail, and airline customer service. Our results show that AgentPex distinguishes agent behavior across models and surfaces specification violations that are not captured by outcome-only scoring. It also provides fine-grained analysis by domain and metric, enabling developers to understand agent strengths and weaknesses at scale. The source code of AgentPex is available at https://github.com/microsoft/agentpex.
Reshabh K Sharma, Shraddha Barke, Benjamin Zorn
University of Washington Seattle, Washington, USA · Microsoft Research Redmond, Washington, USA
Large language model (LLM) agents now solve complex tasks through long plan-and-execution traces, yet the ability to locate errors in a completed traces still lags far behind, especially in the silent failure regime. Existing approaches predict suspect steps via classifiers or LLM judges, or recover correct answers via retry, but none feed the intervention outcome back to refine the attribution itself. We propose \methodname, a method that closes this gap by diagnosing a candidate error step, testing it through controlled replay with a diagnosis-specific patch, and using the verified outcome flip as contrastive evidence to refine the final attribution. Across four localization benchmarks spanning multi-hop reasoning across domains, \methodname achieves the highest localization accuracy among same-auditor methods across all four benchmarks, with the largest gains on structured tool-use traces, while providing actionable localization even when ground-truth answers are unavailable.
Xiaofeng Lin, Yingxu Wang, Tung Sum Thomas Kwok +4
University of California, Los Angeles, USA · Cisco, USA · University of Mississippi, USA
LLM-powered agents increasingly tackle complex tasks by invoking tools, querying databases, executing code, and manipulating intermediate artifacts. These agents follow trajectories that are typically stored as chronological logs, obscuring the underlying dataflow -- the dependencies between their actions and the artifacts they create and manipulate. This limits developers' ability to understand the agents' trails, compare executions, debug failures, and re-use the computations. We present AgentTrails, a prototype system for agent provenance and sensemaking. AgentTrails converts raw trajectories into structured provenance graphs, where tool calls are modeled as computational actions and inputs and outputs as data artifacts. The system supports the comparison of executions by placing multiple provenance graphs on a shared canvas and constructing a joined quotient graph that aligns recurring tools, artifacts, and dependency structures across trajectories. On top of this representation, AgentTrails supports pattern extraction, downstream analysis, and skill abstraction. We demonstrate AgentTrails on real-world agent trajectories, showing that it reveals hidden dependencies, aligns divergent executions, and surfaces recurring tool-use patterns beyond chronological logs.