Behavioral Consistency of LLM Agents

LLM: Large Language Model

Follow this topic Ask about this topic

Latest papers

57 papers29 papers in the last four weeks, up 625% on the four weeks before. 0.2% of all new papers.

Consistent but Biased: Chair Personas Trade Noise for Bias in Multi-Agent Clinical Decisions
Oct 22, 2026Yitong Bai, Chengyi Zhang, Ziyang WangLLM Agents Clinical Decision-Making
Columbia University · Swansea University · Aston University
When LLMs Forget What They Believe: Characterizing State Integrity Collapse in Long-Horizon Dialogues
Oct 22, 2026Yancheng Zhu, Yifan YuanLLM Reliability Behavioral Consistency of LLM Agents
School of Artificial Intelligence, Shenzhen University, Shenzhen, China
Beyond Behavioral Accuracy: Evaluating Individual-Level Fidelity in LLM Investor Simulation
Oct 19, 2026Jiajie He, Yuanhong Jiang, Xintong Chen +2Human Behavior Simulation User Simulation
Hithink Research · McMaster University · Nanyang Technological University
Self-Consistency of Entity Sets and Citations in a Deep Research System under Repeated Prompting
Oct 19, 2026Lauri Pykälä, Kimmo K. KaskiDeep Research Agents Behavioral Consistency of LLM Agents
Department of Computer Science, Aalto University School of Science, P.O. Box 11000, 00076 Aalto, Finland
The Scheduler Is a Hidden Retriever: How Asynchrony Shapes Evidence Acquisition in Search Agents
Oct 19, 2026Ruiyang Ren, Yuhao Wang, Yinan Zhang +1LLM Inference Scheduling Evidence Retrieval
Nanyang Technological University Singapore
The Window Is in the Reasoning: When LLM Agents Decide to Follow Prompt Injections
Oct 16, 2026Haoxuan Jia, Yang Liu, Bin Chong +13Prompt Injection Attacks on AI Agents Prompt Injection Detection
Fullive-AI · Nanyang Technological University · Supply Chain Tech Team Y, JD.com +6
On the possibilities and limitations of LLM-based multi-agent systems for multi-criteria decision-making in healthcare operations management
Oct 16, 2026Marco Boresta, Tommaso Giovannelli, Sepehr Namirad +1Preference Modeling Multi-Criteria Decision Analysis
Istituto di Analisi dei Sistemi ed Informatica (IASI), CNR, Rome, Italy · Department of Mechanical, Materials, and Industrial Engineering, University of Cincinnati, Cincinnati, OH · Department of Computer, Control, and Management Engineering (DIAG), Sapienza University of Rome, Rome, Italy
Beyond Marginal Influence: Interaction-Aware Memory Selection for LLM Agents
Oct 16, 2026BHVSP Subrahmanyam, Neeraj ChoudharyMemory Management for LLM Agents Memory Agent Benchmarks
Department of Computer Science and Engineering, Mahindra University, India
GraphACM: Graph-based Action Consistency Measurement for Uncertainty Quantification in LLM Agents
Oct 15, 2026Yaxin Ma, Wenbo Zhang, Satya Krishna Pothapragada +1Behavioral Consistency of LLM Agents LLM Agent Evaluation
University of Florida, Gainesville, FL, USA · University of California, Irvine, CA, USA
AgentHabit: Characterizing Distinct Behaviors of Agents on Everyday Tasks
Sep 26, 2026Woojung Song, Hoyeol Yang, Jeonghoon Shim +4LLM Agents LLM Agent Evaluation
Seoul National University, Seoul, Republic of Korea
Making Agents More Consistent: Skills Should Form Habits for Repeat Tasks
Sep 21, 2026Travis Weber, Rohit TanejaLLM Agent Skill Learning Behavioral Consistency of LLM Agents
Pheo Inc
Same Patient, Different Order: Action-Level Reliability of Clinical LLM Agents Under Repeated Runs
Sep 11, 2026Rohith Reddy Bellibatlu, Manpreet Singh, Zhoutian Han +1Behavioral Consistency of LLM Agents LLM Agent Reliability
Florida International University, Miami, FL, USA · Boston University, Boston, MA, USA · Stevens Institute of Technology, Hoboken, NJ, USA
Closing the Consistency Gap: Self-Evolving Agents That Learn to Stay on Course
Sep 8, 2026Evelyn Duesterwald, Benjamin Elder, Lilian Ngweta +2LLM Agent Reliability Self-Evolving Agents
IBM Software Innovation Lab · IBM Research
Moral Competence Before Moral Content: Why LLM Agents Lack the Prerequisites for Coherent Alignment
Sep 4, 2026Arno Libert, Derck W. E. Prinzhorn, Daan R. HenselmansLLM Safety Alignment Behavioral Consistency of LLM Agents
Aithos Research Foundation, Keizersgracht 62, Amsterdam, 1015 CS, the Netherlands.
Evaluation of Multi-Turn Consistency in LLM Agents: Survival Analysis and Failure-Rationale Taxonomy
Aug 26, 2026Igor Bogdanov, Olga Manakina, Chung-Horng LungBehavioral Consistency of LLM Agents LLM Agent Reliability
Systems and Computer Engineering Carleton University Ottawa, ON, Canada · Department of Cognitive Science Carleton University Ottawa, ON, Canada
TRACE: A Self-Evolving Skill Bank for Consistent, Limit-Aware LLM Agents
Aug 24, 2026Wenhao Wu, Menghao Zhang, Xin Wang +3LLM Agent Reliability LLM Agent Orchestration
Xiaomi Inc. · Nanjing University · Tsinghua University
Measuring Cross-Task Behavioral Consistency in Language Model Agents
Jul 31, 2026Amritesh Banerjee, Pranil RaichuraAI Agent Reliability Software Engineering Agents
University of Massachusetts Amherst · Mantis AI Research (MIT CSAIL)
When Agents Commit Too Soon: Diagnosing Premature Commitment in LLM Agents
Jun 22, 2026Aman Mehta, Anupam DattaAI Agent Reliability LLM Agent Evaluation
Snowflake AI Research
Doing What They Say, Not What They Reason: Locating the Faithfulness Gap in LLM Agents
May 30, 2026Yufeng WangAI Agent Reliability LLM Agent Evaluation
How Consistent Are LLM Agents? Measuring Behavioral Reproducibility in Multi-Step Tool-Calling Pipelines
Apr 23, 2026Abel YagubyanAI Agent Reliability LLM Agent Evaluation
Independent Researcher
When Agents Look the Same: Quantifying Distillation-Induced Similarity in Tool-Use Behaviors
Apr 23, 2026Chenghao Yang, Yuning Zhang, Zhoufutu Wen +4Tool-Augmented Language Model Agents LLM Agent Evaluation
School of Cyber Science and Technology, USTC · Anhui Province Key Laboratory of Digital Security · 3M-A-P +1
Improving Role Consistency in Multi-Agent Collaboration via Quantitative Role Clarity
Apr 3, 2026Guoling Zhou, Wenpei Han, Fengqin Yang +3Behavioral Consistency of LLM Agents LLM Agents
School of Information Science and Technology, Northeast Normal University, Changchun, Jilin · School of Computer Science and Engineering, Guangxi Normal University, Guilin, Guangxi
Same Voice, Different Lab: On the Homogenization of Frontier LLM Personalities
Mar 20, 2026Avinash Krishna, Kalyana Chadalavada, Unso Eun Seo JoPersonality Modeling in Language Models Behavioral Consistency of LLM Agents
Independent · Anthropic · Cornell University
When Agents Disagree With Themselves: Behavioral Consistency as an Uncertainty Signal for LLM Agents
Feb 12, 2026Aman MehtaAI Agent Reliability Selective Prediction
Snowflake AI Research
Payoff scaling shapes cooperation in LLM agents across languages
Jan 27, 2026Trung-Kiet Huynh, Dao-Sy Duy-Minh, Thanh-Bang Cao +13Behavioral Consistency of LLM Agents Multi-Agent LLMs
Faculty of Information Technology, University of Science (HCMUS), Ho Chi Minh City, Vietnam · Vietnam National University – Ho Chi Minh City (VNU-HCM), Ho Chi Minh City, Vietnam · Faculty of Computer Science and Engineering, Ho Chi Minh City University of Technology (HCMUT), Ho Chi Minh City, Vietnam +2
Are LLM Agents Behaviorally Coherent? A Latent Profiling Approach for Social Simulation
Sep 3, 2025James Mooney, Josef Woldense, Zheng Robert Jia +4Behavioral Consistency of LLM Agents LLM Agent Evaluation
Department of Computer Science and Engineering, University of Minnesota · Department of African American & African Studies, University of Minnesota · Department of Sociology, University of Chicago +1
Effects of Theory of Mind and Prosocial Beliefs on Steering Human-Aligned Behaviors of LLMs in Ultimatum Games
May 30, 2025Neemesh Yadav, Yihuai Lan, Shan Dong +4Theory of Mind Large Language Model-Based Social Simulation
Singapore Management University · Australian National University

Oct 22, 2026J/K move · Enter open · S save

Consistent but Biased: Chair Personas Trade Noise for Bias in Multi-Agent Clinical Decisions

Yitong Bai, Chengyi Zhang, Ziyang Wang

Columbia University · Swansea University · Aston University

LLM Agents Clinical Decision-Making Multi-Agent Deliberation Behavioral Consistency of LLM Agents LLM Agent Reliability

Open paper AskPDF arXiv

In clinical systems built from several large language model (LLM) agents, one agent often chairs the discussion and makes the final decision. Whether the chair's persona prompt changes that decision, or the agreement between repeated runs that some systems treat as reliability, has not been tested with the case held constant. We ask whether the persona changes the decision or only its timing, how it affects agreement between runs, and which behaviors appear when the prompt describes only personality traits. A simulated department of seven agents discussed 20 published cases of degenerative cervical myelopathy, with the case reports replayed unchanged in every run. Only the prompts of the chair and its deputy changed (no persona, or a consultative, autocratic or avoidant one), written either as rules of conduct or as personality traits. With personas written as rules, the chair chose to operate now in 93% of runs when autocratic and in 2% when avoidant, against 57% without a persona. Surgery was still planned in nearly all decisions, so the personas mainly changed whether to operate now or after more tests. These effects held with a second model and reworded prompts. The autocratic and avoidant personas made the three runs of a case agree more often, but only about as often as chance predicts, and Fleiss

κ

did not rise. Prompts that described only personality traits kept more than half of the autocratic and avoidant effects on operating now. They reproduced quick decisions, delays and extra tests, but not ignoring red flags, dropping backup plans or passing the decision to the family. Studies should report and vary the chair's persona as they do the model, and correct agreement between runs for chance before calling it reliability.