Behaviour analysis for large language model-driven agents

Tianqiang Yan; Lizhen Qu

cs.AIOct 23, 2026

Behaviour analysis for large language model-driven agents

Authors: Tianqiang Yan, Lizhen Qu

Organizations: Faculty of Information Technology, Monash University, Melbourne, Australia

Abstract

Understanding language-agent behaviour requires learning how actions are organised across a trajectory and how this organisation relates to context and outcomes. We propose a general framework that connects data-driven behavioural discovery with analysis and experimental investigation. Categories developed from recorded activity define what is measured. Analyses preserve the behaviours' positions in the process and express their relationships as interpretable hypotheses. Statistical assessment and tests in held-out contexts examine the strength and scope of the evidence. Experiments, including prompt steering, then test how behaviour and outcomes change under deliberately varied conditions. A study of iterative self-reflection illustrates how discovery, dependency learning and prompt comparisons can inform one another. We develop this approach for trajectories of varying length, event order and agent roles, and identify advances in coding, analysis and experimentation needed to support its wider use.

Figures & tables

Figure 1: General framework for interpretable behaviour analysis. a , Generation conditions define the agent, its recorded trajectories and an independently specified outcome. b , Categories induced from trajectory records are defined by explicit evidence, opportunities, anchors and actors;

\bot

denotes no opportunity. c , Analyses assess statistical support and generalisation, with safeguards against spurious associations. The graph illustrates one possible statistical representation. Highlighted variables are candidate targets; arrows denote dependencies. R, retry; C, check; V, revise;

w

, event-aligned window. d , Continuations from the same recorded history compare baseline, single-target and joint instructions. Recoding measures targeted and other behaviours; outcomes and cost are recorded for each condition. Findings inform revisions to behavioural definitions, hypotheses and instructions. All matrices and trajectories are schematic.

Figure 2: Measuring behaviour in context. a , An illustrative coding design includes three agent roles and an environment. Roles and event-anchored windows locate records of statements, executed actions and tool errors within the trajectory. b , Statements, executed actions, subsequent changes and output properties provide distinct types of evidence. c , A binary absence code merges an error without recovery with an episode without an error. Opportunity-aware coding uses

\bot

for no opportunity; shared opportunities can themselves induce dependencies. d , The same triggering event can occur at different ordinal steps. Event-aligned windows compare the corresponding occasions rather than a fixed step number. All examples are schematic.

Figure 3: Measurement, prediction and instruction tests in ReBeCA. a , A schematic encoding matrix, generated for illustration, shows six variables over five rounds. Cp, completeness; Sp, specificity; CO, clarity and organization; DA, depth of analysis.

\mathrm{MI}

and

\mathrm{PPL}

indicate increases in estimated mutual information and perplexity; outlined cells mark selected round-specific variables. b , Response-text changes in mathematics and translation (MT), and correctness flips in mathematics. Mathematics uses the same 399 fully graded episodes at every round; translation uses 400 trajectories. Initial and final mathematics accuracy are shown below. c , Preprint-reported cross-validated log-likelihood (CVLL) for all variables (Full), no behavioural variables (Null) and selected variables; folds are defined by subject or source language. Connecting lines link estimates for each variable set. d , Correct answers out of 50 AIME problems for each model and prompt condition; parentheses show count differences from that model’s baseline. CO 2 and Sp 4 label instructions targeting clarity and organization at round two and specificity at round four; Joint combines them. e , Preprint-reported conditional associations for

\mathrm{CO}_{2}

and

\mathrm{Sp}_{4}

, averaged over 20 disjoint subsets, with 95% confidence intervals unadjusted for selection. f , Pooled steering contrasts relative to baseline and their interaction (open circle), in percentage points. Intervals use 20,000 paired bootstrap resamples of the 50 problems, retaining all four fixed models (seed 0). Panels b,d,f were recomputed from source outcomes; panels c,e reproduce preprint estimates.

Behaviour	Observations to distinguish	Opportunities	Relations to assess
Planning 87	Stated plan; precondition check; revised plan; executed step	New information; failed step	Replanning after new information; correspondence between stated plans and subsequent actions
Tool use 86	Tool choice; arguments; returned output; error response; state verification	Timeout; invalid argument; refused permission	Feedback preceding retries, tool changes and replanning; associations with recovery and cost
Self-reflection 84	Identified problem; proposed change; implemented change; renewed check	Critique; new evidence	Correspondence between critiques and revisions; variation across reflection rounds
Collaboration 92 , 12	Work assignment; evidence request; objection; hand-over; shared-state update	Role-specific messages; transferred work	Interaction patterns associated with coordination, omission and duplicated work

Table 1: Behavioural questions beyond the prototype. Proposed measurements distinguish actions, their opportunities and the questions they support. ReBeCA used a different codebook for self-reflection; the entries below specify proposed research designs.

Agent Trajectory Analysis

Agent trajectory analysis studies how agents move or act over time to predict behavior, evaluate execution, and diagnose failures.

See all 48 papers

An Experimental Design Approach to Evaluating Agentic AI's Autonomous Model Discovery
Jul 7, 2026Hao He, Xueying Liu, Chris J. Kuhlman +1AI Coding Agents AI Agent Evaluation
Department of Statistics, Virginia Tech · Department of Statistical Science, Baylor University · Advanced Research Computing, Virginia Tech +1
Using Grounded Theory for Agent Behavior Analysis at Scale
Aug 31, 2026Zhuoran Lu, Yangyang Yu, Zhuoyan Li +5Agent Failure Analysis Agent Trajectory Analysis
1Purdue University · 2Stevens Institute of Technology · 3Cornell University +2
Agent Behavior Maps: Locating Agents by What They Would Do, Without Running Them
Oct 21, 2026Shota Fujikawa, Takuya Tamura, Kosuke Akimoto +2LLM Agent Evaluation LLM Agent Harnesses
The University of Tokyo · NEC Corporation

Jul 7, 2026J/K move · Enter open · S save

An Experimental Design Approach to Evaluating Agentic AI's Autonomous Model Discovery

Hao He, Xueying Liu, Chris J. Kuhlman, Xinwei Deng

Department of Statistics, Virginia Tech · Department of Statistical Science, Baylor University · Advanced Research Computing, Virginia Tech+1

AI Coding Agents AI Agent Evaluation

Open paper AskPDF arXiv

Large language model coding agents increasingly perform open-ended data modeling and analysis. These agents are stochastic and adaptive, and therefore their autonomous model discovery behavior cannot be adequately characterized by a single benchmark run. In this work, we propose an experimental design and analysis framework for systematically evaluating this discovery process, quantifying its variability, and identifying important factors. The proposed framework treats these agents as stochastic model-discovery operators, which map task-specific discovery data and an optimization target to a fitted model. Specifically, we investigate two such operators, Codex and Claude Code, under controlled experimental factors including agent's reasoning effort, task, optimization metric, and composition of training data. For each agent-task-metric combination, regression models and inference are conducted for multiple responses such as output quality, dollar cost, wall-clock time, and process complexity. Furthermore, we develop a utility-aligned canonical decomposition to characterize the dominant direction of the reasoning-effort effect and to assess whether that direction aligns with a performance-cost utility direction. The proposed framework is demonstrated on a testbed of networked word-forming games with insightful findings on reasoning effort with respect to cost and process complexity.

Abstract

Agent Trajectory Analysis

Explore similar work

An Experimental Design Approach to Evaluating Agentic AI's Autonomous Model Discovery

Using Grounded Theory for Agent Behavior Analysis at Scale

Agent Behavior Maps: Locating Agents by What They Would Do, Without Running Them