Behavioral Consistency of LLM Agents

LLM: Large Language Model

Follow this topic Ask about this topic

Latest papers

23 papers6 papers in the last four weeks, up 20% on the four weeks before. 0.0% of all new papers.

Oct 19, 2026

Beyond Behavioral Accuracy: Evaluating Individual-Level Fidelity in LLM Investor Simulation

Large language models (LLMs) are increasingly used as user simulators, but can they faithfully reproduce an individual's behavior and decision logic over time? We investigate this question in a longitudinal paper-trading study with 80 participants. Using rolling next-day predictions, we evaluate trading activity, buy-sell structure, asset selection, and portfolio outcomes, complemented by controlled input ablations and participant-rated explanation alignment. No evaluated LLM reliably outperforms a simple recent-activity persistence baseline for predicting trade occurrence, while buy-sell structure and asset selection remain difficult to reproduce. Similar activity-level performance can also yield substantially different portfolio trajectories. Ablations show that activity predictions depend primarily on recent trades, whereas asset choices are sensitive to multiple evidence sources. Participants rate explanations as more aligned with their recurring strategies than with their decision-specific rationales. These findings reveal a gap between coarse behavioral agreement and individual-level fidelity, motivating separate evaluation of behavioral accuracy, explanation alignment, and evidence dependence in personalized user simulation.

Jiajie He, Yuanhong Jiang, Xintong Chen +2

Hithink Research · McMaster University · Nanyang Technological University

Human Behavior Simulation User Simulation Behavioral Consistency of LLM Agents Faithfulness of Language Model Explanations

Oct 19, 2026

Self-Consistency of Entity Sets and Citations in a Deep Research System under Repeated Prompting

Deep research agents (DRA) are commonly used for open-ended synthesis tasks where closed-world QA benchmarks are poorly suited to characterize relevant aspects of performance. We study the self-consistency of one commercial deep research system under repeated identical prompting, representing each report with the set of named entities it mentions, categorised by a custom schema, and the set of citations it retrieves. Across repeated runs of a single open-ended cybersecurity related prompt, we quantify pairwise overlap to characterize how stable the retrieved and synthesized output structure is under repeated identical prompts. In ten runs, the pairwise entity Jaccard results in a median of 0.12 and the pairwise citation Jaccard is below 0.02, with discovery curves that do not visibly saturate. The magnitude and structure of (dis)agreement between entity sets varies by entity class, which suggests that self-consistency in a DRA is better understood as a function of the underlying entity space than as a stable property of the system.

Lauri Pykälä, Kimmo K. Kaski

Department of Computer Science, Aalto University School of Science, P.O. Box 11000, 00076 Aalto, Finland

Deep Research Agents Behavioral Consistency of LLM Agents LLM Evaluation Automated Agent Evaluation

Oct 19, 2026

The Scheduler Is a Hidden Retriever: How Asynchrony Shapes Evidence Acquisition in Search Agents

Language-model agents increasingly solve complex tasks through parallel reasoning and asynchronous information gathering. Controlled replay across two model families and three benchmarks shows that delivery timing redirects search decisions made before all results arrive, despite identical initial evidence. Acquisition disagreement reaches up to 24 percentage points above same-order repeat variation. This dependence persists after all initial evidence arrives and through a second search, turning temporary differences in what agents have seen into lasting differences in what they acquire. An acquisition-regret bound with a matching construction shows that invariance at a fixed early arrival count can limit the use of evidence already received. We introduce Confluent Evidence Acquisition (CEA), which instead releases local reasoning when each subquestion's evidence is complete and allocates shared search independently of completion order. For a fixed plan and stable conditional response law, we prove joint invariance of decisions, acquisitions, and answers. With question-only plans across three benchmarks, CEA brings acquisition disagreement to within 1.0 percentage point of repeat variation and achieves accuracy comparable to complete-input controls, improving over early commitment by up to 12.6 percentage points. Replacing CEA's fixed allocation with first-ready allocation under contention adds 21.6 percentage points of acquisition disagreement, while local query-target disagreement remains near repeat variation. Timed comparisons show that CEA reaches decisions earlier and responds faster than the same program waiting for all initial evidence.

Ruiyang Ren, Yuhao Wang, Yinan Zhang +1

Nanyang Technological University Singapore

LLM Inference Scheduling Evidence Retrieval Inference-Time Search Behavioral Consistency of LLM Agents

Oct 16, 2026

The Window Is in the Reasoning: When LLM Agents Decide to Follow Prompt Injections

LLM agents that read untrusted content can be hijacked by prompt injections. Runtime monitors that try to catch a hijack before the agent acts, and benchmarks that score each task with a single run, share an assumption: whether an agent follows an injection is settled when the injection is read. We test this assumption with branching rollouts that replay an episode exactly up to the step at which the agent reads an injection and then resample the rest of the episode many times, splitting the outcome variance into a part fixed at read time and a part decided afterwards. Across four AgentDojo suites, a Qwen3 agent without thinking is largely decided at read time (0.91 of the outcome variance lies between states); with thinking on, this share drops to 0.62 (95% CI of the drop [-0.38, -0.22]), in the same direction for three attack templates and two success labels. How much is fixed at read time is a property of the model that the same measurement reveals, and thinking lowers the attack success rate of Qwen3 but raises it for two other models. Resampling from reasoning prefixes shows that the decision forms gradually as the agent reasons. Zero-shot chain-of-thought monitors built on cheaper models miss much of what is already decided, whereas stronger monitors, and a lookahead monitor that judges simulated next actions, come close to what the prefix determines. Finally, single-run labels of thinking agents disagree with a rerun up to 35% of the time, so benchmarks need repeated runs per task.

Haoxuan Jia, Yang Liu, Bin Chong +13

Fullive-AI · Nanyang Technological University · Supply Chain Tech Team Y, JD.com +6

Prompt Injection Attacks on AI Agents Prompt Injection Detection LLM Agent Evaluation Behavioral Consistency of LLM Agents

Oct 16, 2026

On the possibilities and limitations of LLM-based multi-agent systems for multi-criteria decision-making in healthcare operations management

Multi-criteria models for healthcare operations need weights that express how stakeholders trade off patient waiting time, provider workload, and operating cost, but these weights are difficult to quantify. Teams of large language model (LLM) agents, each prompted to represent a stakeholder, have been used to supply them. We propose a framework for assessing the reliability of LLM-generated preferences as inputs to multi-objective optimization and quantifying how their uncertainty affects decisions. We study explicit preferences by asking agents to rate the importance of the criteria in an outpatient appointment scheduling problem, and implicit preferences by inferring trade-offs from the agents' pairwise choices between emergency department configurations. The computational experiments show that the explicit preferences respond to the clinic's operating conditions and differ across roles, but deliberation among the agents can erase these differences, whereas simple averaging preserves them. The pairwise choices used to reveal the agents' implicit preferences depend largely on which option is presented first, and we find limited evidence that a statistical model using contextual information alongside presentation order predicts the agents' choices on new comparison pairs better than a model using presentation order alone. We then formulate robust weighted-sum optimization models to account for preference uncertainty. In most cases, these models select different schedules or configurations from those obtained using average preference weights. These results indicate that LLM-generated preferences need an audit of this kind before they are used as optimization weights in healthcare decision support.

Marco Boresta, Tommaso Giovannelli, Sepehr Namirad +1

Istituto di Analisi dei Sistemi ed Informatica (IASI), CNR, Rome, Italy · Department of Mechanical, Materials, and Industrial Engineering, University of Cincinnati, Cincinnati, OH · Department of Computer, Control, and Management Engineering (DIAG), Sapienza University of Rome, Rome, Italy

Preference Modeling Multi-Criteria Decision Analysis Healthcare Behavioral Consistency of LLM Agents Multi-Agent Deliberation

Oct 16, 2026

Beyond Marginal Influence: Interaction-Aware Memory Selection for LLM Agents

LLM-agent memory selection often uses retrieval rankings based on record relevance or individual influence. Such rankings cannot identify sufficient subsets under redundant or jointly necessary evidence. We study per-decision memory sufficiency: selecting a small subset that preserves full-memory behavior. To separate search error from model error, we introduce JointCoreBench, a benchmark with nine interaction families, six workflow domains, natural-language decisions, and exact sufficient-set oracles, and JointCore, a relevance cost-ordered group-deletion baseline that tests blocks before individual records. On 6,912 exact conditions, cost-ordered JointCore recovers a minimum-cost sufficient set in 100.0% of cases, reduces subset evaluations from 30.00 to 11.13 (62.9%), and achieves 87.8% active-memory savings. Across four 12B-70B model families and 432 tasks per model, its unweighted mean is 10.28 versus 16.00 model queries and 50.6% versus 43.1% minimum-cost recovery relative to cost aware singleton deletion. Both methods preserve full-memory behavior under the deterministic primary interface. Tests with physical deletion, UNKNOWN masks, PENDING sentinels, counterfactual values, and alternative labels identify interface dependent behavior. JointCore is an efficient benchmark baseline; the study does not establish deployment-ready memory compression.

BHVSP Subrahmanyam, Neeraj Choudhary

Department of Computer Science and Engineering, Mahindra University, India

Memory Management for LLM Agents Memory Agent Benchmarks Behavioral Consistency of LLM Agents

Oct 15, 2026

GraphACM: Graph-based Action Consistency Measurement for Uncertainty Quantification in LLM Agents

Behavioral consistency across repeated executions provides an informative signal for assessing the reliability of large language model (LLM) agents. However, existing approaches typically summarize consistency with a single task-level score, without identifying where behavioral differences occur or whether actions are compared under equivalent decision contexts. We propose a graph-based framework for fine-grained behavioral consistency measurement and uncertainty quantification. Given multiple rollouts of the same task, we merge equivalent states into a directed, action-labeled graph, enabling action comparisons at shared decision states. We introduce node-level and global measures of action consistency, using the global consistency score as a proxy for uncertainty to predict task success or failure. Experiments across three LLMs on ALFWorld, AppWorld, and WebShop demonstrate that our measure is positively associated with task success and achieves an average AUROC of 0.811 for task-success prediction, outperforming the strongest behavioral-consistency baseline (0.729) and verbalized confidence (0.769). These results highlight the effectiveness of state-conditioned behavioral consistency and pinpoint where agent actions agree.

Yaxin Ma, Wenbo Zhang, Satya Krishna Pothapragada +1

University of Florida, Gainesville, FL, USA · University of California, Irvine, CA, USA

Behavioral Consistency of LLM Agents LLM Agent Evaluation LLM Agents LLM Agent Reliability LLM Uncertainty Estimation

Sep 26, 2026

AgentHabit: Characterizing Distinct Behaviors of Agents on Everyday Tasks

Large language model (LLM) agents assist users with everyday tasks that can be completed in many reasonable ways. Even when their answers are useful, how agents carry out these tasks may not match users' preferences and needs. For example, agents differ in whether they ask clarifying questions or search the web. We introduce HABIT, a taxonomy of 23 behavioral axes in five categories, which three authors and three LLMs derive bottom-up from 408 agent trajectories across 17 domains. On held-out tasks, HABIT distinguishes models more clearly than existing taxonomies of human values and agent actions while supporting comparably consistent annotation. Building on HABIT, we construct AgentHABIT, a benchmark that profiles each agent's behavioral tendencies from its trajectories on 86 everyday tasks. Profiling 18 models with AgentHABIT reveals a range of distinctive tendencies. For example, most GPT and Claude models state their assumptions and offer alternatives when requirements conflict, whereas Qwen and Google's models more often leave assumptions or changes to requirements unstated. These profiles remain recognizable even when built from entirely different sets of tasks, indicating that they reflect general tendencies rather than task-specific behavior. Prompting agents to adopt specific behaviors shifts some axes readily but barely changes others, while fine-tuning on another model's trajectories changes only part of a model's profile and leaves much of it intact. Overall, HABIT and AgentHABIT provide a systematic framework for characterizing how agents carry out everyday tasks beyond task success, offering insights to guide the development of agents whose behavior better fits users' needs.

Woojung Song, Hoyeol Yang, Jeonghoon Shim +4

Seoul National University, Seoul, Republic of Korea

LLM Agents LLM Agent Evaluation AI Agent Benchmarks Behavioral Consistency of LLM Agents

Sep 21, 2026

Making Agents More Consistent: Skills Should Form Habits for Repeat Tasks

On repeated work, agents are inconsistent. We ran 42 tasks three times each and found that, depending on the model, 38% to 74% returned answers that did not agree. Consistency is what a buyer, an auditor, or a regulator requires, and agents do not have it. They are wasteful too: 95.3% to 97.2% of what an agent generates goes to re-deriving a plan the system already knows. We propose skill habit formation. An agent mines its own execution history for candidate skills, deterministic variants that compete against the incumbent rather than replacing it. A candidate declares the region of input space it claims, so the common case runs as a script and the rest falls through to reasoning. Four gates of ascending cost admit candidates; the central one tests a candidate's execution trace against a retained reference, within a tolerance measured from that reference's own run-to-run variability. On text-to-SQL, three of four reasoning arms reproduced their own output on 11 to 13 of 42 repeated questions and the fourth on 26 of 42, while a habit-formed variant reproduced on all 456 dispatches we repeated and was non-inferior to every arm it replaced (p<0.0001). It also used 14% to 56% fewer tokens, turning net positive after 7 to 53 reuses. We measured what this costs in accuracy. The guard admitted work it should have deferred on 2.6% of natural paraphrases and 26% of inputs near its boundary, and 11 of 13 such failures were invisible to the trace-conformance gate at any threshold. Deterministic errors repeat exactly: a bad habit is as reliable as a good one, and that is the price of the property that makes the system auditable. Separating routing from parameter extraction raised end-to-end accuracy from 0.888 to 0.952 at 43% of the cost.

Travis Weber, Rohit Taneja

Pheo Inc

LLM Agent Skill Learning Behavioral Consistency of LLM Agents LLM Agent Reliability

Sep 11, 2026

Same Patient, Different Order: Action-Level Reliability of Clinical LLM Agents Under Repeated Runs

A clinical agent benchmark can report the same verdict on identical inputs while the agent files a materially different order on each run. Such agents order tests, request medications and place referrals, yet benchmarks typically score one run per task and rarely ask whether identical inputs produce identical actions; MedAgentBench, the benchmark we use, scores a single attempt and says so. To measure this gap we introduce "same-input rerun", which replays a task with every input held fixed and compares the orders rather than the score, with six reliability metrics, and apply it to 1000 MedAgentBench runs across 50 tasks from its five write-capable families, two open-weight models below ten billion parameters quantised to four bits, and two temperatures. The study establishes that action-level divergence exists and can pass unrecorded by the score, not that any rate generalises. Under the 8B model at temperature 0.7, all 43 ordering groups emit a different set of orders across five identical runs, 26 emit the order on some runs and not others, and 28 record a different coded value, dose or analyte. In 22 of those 43 the benchmark reports the same failing verdict for materially different behaviour, as it does for all 10 divergent groups of the 4B model at 0.7. Orders also reach different endpoints across runs, one of which the record server rejects while the agent is told it succeeded. These findings motivate repeated-run evaluation, action-level stability reporting and execution-faithful environment feedback in clinical-agent benchmarks.

Rohith Reddy Bellibatlu, Manpreet Singh, Zhoutian Han +1

Florida International University, Miami, FL, USA · Boston University, Boston, MA, USA · Stevens Institute of Technology, Hoboken, NJ, USA

Behavioral Consistency of LLM Agents LLM Agent Reliability LLM Agent Evaluation Clinical Language Model Evaluation AI Agent Benchmarks

Sep 8, 2026

Closing the Consistency Gap: Self-Evolving Agents That Learn to Stay on Course

Large language model (LLM)-powered agents can be accurate on average yet unreliable in production, a discrepancy that has been observed but remains largely unaddressed. When given the same task five times, a ReAct agent on the AppWorld benchmark using GPT-4.1 succeeds in all five runs only 53% of the time, even though its per-run pass rate averages 77%. We call this 24-point shortfall the consistency gap, and we argue that addressing it is a precondition for trustworthy AI agent deployment. We present a self-evolving agent framework that reduces this gap by identifying unstable, low-consistency steps in agent trajectories and converting them into episodic memory the agent can draw on in future runs. At its core is a Consistency Analyzer that pinpoints where and why a trajectory is likely to flip across executions, and a Guideline Generator that converts the diagnosis into targeted guidelines, committed to memory and injected into future agent executions on similar tasks. On AppWorld with ReAct/GPT-4.1, our framework raises the fraction of tasks that succeed in all five runs by +16 points on same-task evaluation and +13 points on similar-task generalization.

Evelyn Duesterwald, Benjamin Elder, Lilian Ngweta +2

IBM Software Innovation Lab · IBM Research

LLM Agent Reliability Self-Evolving Agents Memory-Augmented Language Model Agents Behavioral Consistency of LLM Agents

Sep 4, 2026

Moral Competence Before Moral Content: Why LLM Agents Lack the Prerequisites for Coherent Alignment

AI alignment requires AI systems to adhere to human norms, values, or intentions. Under value pluralism there is no correct target, but a shared prerequisite is that the system's behavior expresses a coherent policy: a mapping from situations to verdicts that is invariant while a situation's morally relevant features are preserved, and sensitive when they change. We introduce four structural conditions for such coherent policies: verdict stability, monotonicity, decisiveness, and Pareto viability. Together they measure a form of moral competence that is evaluable from behavior alone, without reference to a moral standard or expert baseline, forming a structural floor for alignment rather than a normative target. We demonstrate the methodology on three simulated deployments featuring LLM-based agents facing moral dilemmas. Evaluating nine frontier models under a factorial design of five paraphrases, five escalation levels, and three dominance conditions, we show no model expresses a coherent policy across the three deployments: surface-form perturbation alone produces verdict-rate shifts of up to

99

percentage points at a single escalation level, and a model's success on one scenario does not predict its competence on another. This suggests LLM-based agents are not currently the kind of object to which alignment can meaningfully apply.

Arno Libert, Derck W. E. Prinzhorn, Daan R. Henselmans

Aithos Research Foundation, Keizersgracht 62, Amsterdam, 1015 CS, the Netherlands.

LLM Safety Alignment Behavioral Consistency of LLM Agents Moral Reasoning in Language Models AI Safety Evaluation

Aug 26, 2026

Evaluation of Multi-Turn Consistency in LLM Agents: Survival Analysis and Failure-Rationale Taxonomy

Large language model (LLM) agents may perform well on isolated tasks yet drift into inconsistency over extended interaction. We evaluate temporal consistency in a controlled 20-step multi-agent setting inspired by delayed-gratification studies. At each step, an agent chooses between continuing to delay a reward or claiming it immediately (terminating the episode). Across a full-factorial manipulation of social visibility (private vs public), persona stressors, and deliberation policy, we run 84,540 trajectories spanning 8 model families. Treating the first reward-claim as a time-to-event outcome, we estimate Kaplan-Meier survival curves and fit discrete-time hazard regression to quantify how experimental factors shift failure risk over time. Then, to analyze rationales and language patterns associated with failure, we build a seven-category taxonomy from 13,780 deliberation traces from agents who choose to terminate the episode, using an LLM-assisted labeling paired with human audit (

κ=0.83

). Rationale profiles change systematically with time and context: early failures are more impulse-driven, later failures more fatigue- and cost-benefit-framed, while public settings increase norm-oriented justifications. We also find a deliberation-inconsistency association: among failures, longer deliberation correlates with higher rates of intra-rationale contradiction (simultaneous pro-delay and pro-claim statements), challenging the assumption that more reasoning text implies greater consistency. Together, the survival and rationale analyses reveal distinct temporal reliability regimes and model-specific "failure fingerprints", offering an evaluation lens for diagnosing inconsistency in multi-turn agent behavior.

Igor Bogdanov, Olga Manakina, Chung-Horng Lung

Systems and Computer Engineering Carleton University Ottawa, ON, Canada · Department of Cognitive Science Carleton University Ottawa, ON, Canada

Behavioral Consistency of LLM Agents LLM Agent Reliability Reasoning Consistency in Language Models Long-Horizon Agent Evaluation

Aug 24, 2026

TRACE: A Self-Evolving Skill Bank for Consistent, Limit-Aware LLM Agents

Reliable deployment of LLM agents in user-facing products depends not on raw task-solving ability but on consistency and limit-awareness: behaving the same way across repeated trials, and recognizing when a request cannot, or cannot yet, be safely fulfilled. CAR-bench exposes this reliability gap in the domain of in-car assistants: an LLM-simulated user issues incomplete or ambiguous requests, requiring the agent to resolve uncertainty through multi-turn dialogue and tool use while strictly adhering to domain policies. Even frontier models show a substantial gap between what they can solve at least once (Pass@3) and what they solve consistently across trials (Pass^k). We bridge this gap with TRACE (TRAjectory-Contrastive Evolution), which iteratively improves a skill-based agent's behavioral knowledge without modifying model weights. This knowledge is organized as a Skill Bank of modular, retrievable skills, each encoding a self-contained set of tool-use rules and behavioral guidelines. TRACE evolves this bank through an agentic self-evolution loop: after each evaluation round, it groups trajectories by the skills invoked and refines each skill by contrasting successful and failed behaviors. The updated bank then guides subsequent rounds, while during deployment the Actor performs state-conditioned skill orchestration at every turn. On GPT-5.5, TRACE improves consistency (Pass^3) by 34.6 points, from 59.9% to 94.5%, while shrinking the gap between potential and reliable performance to just 4.0 points. On the official hidden set, TRACE achieved first place using GPT-5.6-Sol, attaining a Pass^3 score of 70%-a 40% relative improvement over the baseline. These results show that TRACE converts high model potential into stable, consistent performance gain. Project homepage: https://darwin-agent.github.io/Car-bench-TRACE.

Wenhao Wu, Menghao Zhang, Xin Wang +3

Xiaomi Inc. · Nanjing University · Tsinghua University

LLM Agent Reliability LLM Agent Orchestration LLM Agent Self-Improvement LLM Agent Skill Learning AI Agent Benchmarks Conversational Agents+2

Jul 31, 2026

Measuring Cross-Task Behavioral Consistency in Language Model Agents

Agent evaluation relies almost entirely on outcome metrics such as success rate, which capture whether an agent succeeds but not how consistently it behaves. We argue that behavioral consistency across tasks is a distinct and measurable property, and we introduce the Behavioral Consistency Metric (BCM) to quantify it. BCM trains a model to predict task success from behavioral features of agent execution traces, derives a per-trajectory feature-attribution vector, and measures the mean pairwise similarity of these vectors within an agent system. Across roughly 9,000 trajectories from six language model agents on software engineering tasks, our central finding is that cross-task and within-task consistency are distinct axes that can diverge: some systems are locally reproducible, behaving similarly on repeated attempts at one task, yet globally fragmented, with no stable strategy across different tasks, while others are consistent at both scales. Prior work measures only same-task reproducibility and so cannot observe this separation. We further find that consistency is not reducible to success rate, since systems with comparable success can differ sharply in consistency, and that the frontier-versus-open-source consistency gap persists under a within-task control that holds task difficulty constant. We position BCM as a process-level reliability signal that complements outcome metrics, and we are explicit about the conditions under which it is meaningful.

Amritesh Banerjee, Pranil Raichura

University of Massachusetts Amherst · Mantis AI Research (MIT CSAIL)

AI Agent Reliability Software Engineering Agents LLM Agent Evaluation LLM Agent Reliability Behavioral Consistency of LLM Agents

Jun 22, 2026

When Agents Commit Too Soon: Diagnosing Premature Commitment in LLM Agents

Tool-using LLM agents make early choices about what to search for, which evidence to trust, and what to do next. A wrong early choice can steer every later tool call, so an agent may produce a coherent but incorrect result. Existing evaluation does not reveal this process: final-answer scoring comes only after completion, and agreement across runs can be high even when every run is wrong. We ask whether hidden states reveal that repeated runs have already settled into stable behavior. Our contributions are threefold. First, we introduce representational commitment, which measures hidden-state similarity across repeated runs at the same step. On Llama-3.1-70B running ReAct on HotpotQA, we show that higher step-4 similarity is associated with lower whole-trajectory variability (r = -0.35); we find the same pattern in action sequences and replicate it on two other models and StrategyQA. This provides an internal measure for studying when agent behavior stabilizes. Second, we establish the measure's scope. Shared evidence explains much of the observed association, the HotpotQA result disappears on hard questions, and correct and confidently wrong questions overlap. We therefore show that the measure detects stability, not correctness, and cannot by itself establish that commitment was premature. Third, we evaluate monitoring and intervention. A monitor built from three partial runs predicts whole-trajectory variability (mean AUROC 0.81). A commitment prompt reduces variability by 28% relative to a token-matched filler, but not significantly relative to the standard agent after correction, and it does not improve accuracy. Together, these contributions provide a measurement and evaluation framework for studying how agent trajectories stabilize and for testing when additional evidence, sampling, or review may be useful.

Aman Mehta, Anupam Datta

Snowflake AI Research

AI Agent Reliability LLM Agent Evaluation LLM Agents Long-Horizon LLM Agents LLM Agent Reliability Behavioral Consistency of LLM Agents

May 30, 2026

Doing What They Say, Not What They Reason: Locating the Faithfulness Gap in LLM Agents

Do LLM agents act on the reasoning they state? This question of process fidelity is central to LLM-based social simulation, yet hard to measure where no reference for correct behavior exists. We study it in a controlled setting: a Texas Poker simulator with a verifiable reference action for every decision by splitting the faithfulness gap into two steps: reasoning-to-conclusion (does the stated decision follow from the agent's own reasoning?) and conclusion-to-action (does the agent execute what it states?). The two steps behave very differently. Conclusion-to-action is reliable: inconsistency is 0.7% for Claude Haiku 4.5 and 1.4% for DeepSeek-Reasoner once the conclusion is read from an explicit tag, whereas free-text conclusion extraction reports 22-26%. Reasoning-to-conclusion is where fidelity frays, but not through a single dominant failure. In a step-level diagnostic the agent's errors split roughly evenly between bad inputs, borderline cases, and rule misapplication deriving a conclusion that contradicts the agent's own restated rule from inputs it estimated correctly. This composition is model-dependent: rule misapplication accounts for a third of Haiku's interpretable errors but only 8% of DeepSeek's. The one robust signal is directional: when an agent does misapply its own stated rule, it almost always (99.5% for Haiku) errs in the risk-averse direction. The override is partly hedging behavior, not a capability limit: instructing the agent to apply the rule mechanically halves the misapplication rate (13.9% to 6.8% of decisions) and raises adherence by eight points. Process-fidelity evaluation should therefore elicit machine-checkable conclusions and probe for directional biases rather than assume a single upstream failure mode, lest it conflate measurement noise with model behavior.

Yufeng Wang

AI Agent Reliability LLM Agent Evaluation LLM Decision-Making Agentic Reasoning LLM Reasoning Behavioral Consistency of LLM Agents

Apr 23, 2026

How Consistent Are LLM Agents? Measuring Behavioral Reproducibility in Multi-Step Tool-Calling Pipelines

Large language model (LLM) agents with tool-calling capabilities are increasingly deployed in production systems, yet a fundamental reliability question remains under-explored: does the same agent behave the same way twice? We present a systematic empirical study of behavioral consistency in multi-step tool-calling agents, measuring whether agents select the same tools, in the same order, with the same arguments, across repeated identical invocations. Unlike prior work on consistency in ReAct-style agents(search-only, free-text actions), we study the richer setting of structured tool-calling interfaces with typed parameters and consequential side effects.

Abel Yagubyan

Independent Researcher

AI Agent Reliability LLM Agent Evaluation LLM Tool Use Behavioral Consistency of LLM Agents LLM Agent Reliability

Apr 23, 2026

When Agents Look the Same: Quantifying Distillation-Induced Similarity in Tool-Use Behaviors

Model distillation is a primary driver behind the rapid progress of LLM agents, yet it often leads to behavioral homogenization. Many emerging agents share nearly identical reasoning steps and failure modes, suggesting they may be distilled echoes of a few dominant teachers. Existing metrics, however, fail to distinguish mandatory behaviors required for task success from non-mandatory patterns that reflect a model's autonomous preferences. We propose two complementary metrics to isolate non-mandatory behavioral patterns: Response Pattern Similarity (RPS) for verbal alignment and Action Graph Similarity (AGS) for tool-use habits modeled as directed graphs. Evaluating 18 models from 8 providers on

τ

-Bench and

τ^2

-Bench against Claude Sonnet 4.5 (thinking), we find that within-family model pairs score 5.9 pp higher in AGS than cross-family pairs, and that Kimi-K2 (thinking) reaches 82.6%

S_{\text{node}}

and 94.7%

S_{\text{dep}}

, exceeding Anthropic's own Opus 4.1. A controlled distillation experiment further confirms that AGS distinguishes teacher-specific convergence from general improvement. RPS and AGS capture distinct behavioral dimensions (Pearson

r

= 0.491), providing complementary diagnostic signals for behavioral convergence in the agent ecosystem. Our code is available at https://github.com/Syuchin/AgentEcho.

Chenghao Yang, Yuning Zhang, Zhoufutu Wen +4

School of Cyber Science and Technology, USTC · Anhui Province Key Laboratory of Digital Security · 3M-A-P +1

Tool-Augmented Language Model Agents LLM Agent Evaluation AI Agent Benchmarks Tool-Using Agents Knowledge Distillation Behavioral Consistency of LLM Agents

Apr 3, 2026

Improving Role Consistency in Multi-Agent Collaboration via Quantitative Role Clarity

In large language model (LLM)-driven multi-agent systems, disobey role specification (failure to adhere to the defined responsibilities and constraints of an assigned role, potentially leading to an agent behaving like another) is a major failure mode \cite{DBLP:journals/corr/abs-2503-13657}. To address this issue, in the present paper, we propose a quantitative role clarity to improve role consistency. Firstly, we construct a role assignment matrix

S(φ)=[s_{ij}(φ)]

, where

s_{ij}(φ)

is the semantic similarity between the

i

-th agent's behavior trajectory and the

j

-th agent's role description. Then we define role clarity matrix

M(φ)

as

\text{softmax}(S(φ))-I

, where

\text{softmax}(S(φ))

is a row-wise softmax of

S(φ)

and

I

is the identity matrix. The Frobenius norm of

M(φ)

quantifies the alignment between agents' role descriptions and their behaviors trajectory. Moreover, we employ the role clarity matrix as a regularizer during lightweight fine-tuning to improve role consistency, thereby improving end-to-end task performance. Experiments on the ChatDev multi-agent system show that our method substantially improves role consistency and task performance: with Qwen and Llama, the role overstepping rate decreases from

46.4\%

to

8.4\%

and from

43.4\%

to

0.2\%

, respectively, and the role clarity score increases from

0.5328

to

0.9097

and from

0.5007

to

0.8530

, respectively, the task success rate increases from

0.6769

to

0.6909

and from

0.6174

to

0.6763

, respectively.

Guoling Zhou, Wenpei Han, Fengqin Yang +3

School of Information Science and Technology, Northeast Normal University, Changchun, Jilin · School of Computer Science and Engineering, Guangxi Normal University, Guilin, Guangxi

Behavioral Consistency of LLM Agents LLM Agents Multi-Agent Collaboration LLM Agent Training LLM Agent Reliability

Mar 20, 2026

Same Voice, Different Lab: On the Homogenization of Frontier LLM Personalities

LLM assistant personalities play a critical role in user experience and perceived response quality. We present a large-scale experiment of frontier LLM personalities using external ELO-based traits scoring across 144 traits. We find that all models tested converge on a form of trait expression that is systematic, methodical, and analytical and suppress traits such as remorseful and sycophantic. Moreover, models tend to diverge more in their expression of middle-of-distribution traits such as poetic or playful, but even these so-called creative models tend to have more neutral identities. These similarities suggest an implicit emergence of a standard of optimal assistant behavior. In a landscape of varied training methods, character training, therefore, stands out for its uniformity, offering insight into a tacit consensus between model developers.

Avinash Krishna, Kalyana Chadalavada, Unso Eun Seo Jo

Independent · Anthropic · Cornell University

Personality Modeling in Language Models Behavioral Consistency of LLM Agents Normative Conformity in Language Models Language Model Post-Training

Feb 12, 2026

When Agents Disagree With Themselves: Behavioral Consistency as an Uncertainty Signal for LLM Agents

Running the same LLM agent on identical inputs yields 2.3-4.2 distinct action sequences per 10 runs; this behavioral variance constitutes a training-free, black-box uncertainty signal that instantiates selective classification and distribution-free calibration for agentic systems. Across 8,000 runs of four models on 200 HotpotQA questions, consistent tasks (at most 2 unique paths) achieve 82-87% accuracy while inconsistent tasks (4 or more paths) achieve 41-65%, a gap that survives controls for task difficulty. Divergence concentrates at step 2 (50.5% of Llama tasks), and consistency metrics detect failures with AUROC 0.62-0.78. Exploiting this signal, selective prediction (answering only when k=3 runs agree) achieves 87-88% accuracy at 54-62% coverage, a 6-14pp gain over single-run baselines, and matches a split-conformal baseline without a held-out calibration set. A cross-benchmark validation on SWE-bench (50 tasks, 1,000 runs) preserves the consistency hierarchy while revealing an ~8x spread in mean trajectory length across models, and bootstrap analysis shows single-run evaluations misrank models 29.3% of the time.

Aman Mehta

Snowflake AI Research

AI Agent Reliability Selective Prediction LLM Agent Evaluation Reasoning Consistency in Language Models Language Model Calibration LLM Reliability+3

Sep 3, 2025

Are LLM Agents Behaviorally Coherent? A Latent Profiling Approach for Social Simulation

LLM agents are increasingly proposed as substitutes for human participants in social research, usually on the strength of survey responses that match those of the humans they represent. We argue that such output-level alignment is not enough. Drawing on the behavioral sciences, we propose Latent Profiling, a framework for testing behavioral coherence: whether an agent's conversational behavior follows, as human behavioral models predict, from its own latent profile elicited by short surveys. As a case study, we test six hypotheses about preference, openness and agreement across nine open models. Agents reproduce surface-level trends but systematically fail in-depth tests: they are not behaviorally coherent, even when their individual responses look human. Latent Profiling thus offers a theory-grounded instrument for a behavioral science of LLMs.

James Mooney, Josef Woldense, Zheng Robert Jia +4

Department of Computer Science and Engineering, University of Minnesota · Department of African American & African Studies, University of Minnesota · Department of Sociology, University of Chicago +1

Behavioral Consistency of LLM Agents LLM Agent Evaluation LLM Agents Large Language Model-Based Social Simulation Human Behavior Simulation