LLM Agent Reliability

LLM: Large Language Model

LLM agent reliability studies whether agents can complete tasks correctly, follow instructions and safety constraints, use evidence faithfully, and behave consistently across repeated or extended interactions. Common approaches combine structured harnesses and deterministic tool interfaces with state tracking, evidence checks, runtime monitoring, calibrated abstention, and recovery from failures; training methods also teach agents to verify actions or learn from execution feedback. Evaluations increasingly use interactive benchmarks, noisy or changing tools, repeated trials, and trajectory-level measures alongside final task success. A central tension is that added reasoning, delegation, memory, or orchestration can improve some tasks but also introduce new failure paths, so reliability depends on the model, harness, tools, and evaluation setting together.

Latest papers

843 papers181 papers in the last four weeks, up 93% on the four weeks before. 1.0% of all new papers.

Taming Actor-Observer Asymmetry in Agents via Dialectical Alignment
Apr 21, 2026Bobo Li, Rui Wu, Zibo Ji +5AI Agent Benchmarks Multi-Agent Reasoning
National University of Singapore · Sichuan University · University of Minnesota Twin Cities +2
Refute-or-Promote: An Adversarial Stage-Gated Multi-Agent Review Methodology for High-Precision LLM-Assisted Defect Discovery
Apr 21, 2026Abhinav AgarwalSoftware Vulnerability Detection LLMs for Cybersecurity
AI scientists produce results without reasoning scientifically
Apr 20, 2026Martiño Ríos-García, Nawaf Alampara, Chandan Gupta +5AI Agent Reliability Scientific Reasoning
Laboratory of Organic and Macromolecular Chemistry (IOMC), Friedrich Schiller University Jena, Humboldstr. 10, 07743 Jena, Germany · Department of Civil Engineering, Indian Institute of Technology Delhi, Hauz Khas, New Delhi 110016, India · School of Interdisciplinary Research, Indian Institute of Technology Delhi, Hauz Khas, New Delhi 110016, India +3
JTPRO: A Joint Tool-Prompt Reflective Optimization Framework for Language Agents
Apr 20, 2026Sandip Ghoshal, Anshul Mittal, Jyotika Singh +9AI Agent Reliability LLM Tool Use
Oracle AI
Do LLMs Need to See Everything? A Benchmark and Study of Failures in LLM-driven Smartphone Automation using Screentext vs. Screenshots
Apr 20, 2026Shiquan Zhang, Tianyi Zhang, Le Fang +3Mobile GUI Automation LLM Agent Evaluation
School of Computing and Information Systems, University of Melbourne, Melbourne, Australia · School of Computer Science, University of Auckland, Auckland, New Zealand
Towards Self-Improving Error Diagnosis in Multi-Agent Systems
Apr 19, 2026Jiazheng Li, Emine Yilmaz, Bei Chen +1Multi-Agent LLM Systems Multi-Agent Systems
King’s College London · Amazon Alexa AI · University College London
Agents Explore but Agents Ignore: LLMs Lack Environmental Curiosity
Apr 19, 2026Leon Engländer, Sophia Althammer, Ahmet Üstün +2LLM Agent Evaluation Tool-Using Agents
Cohere · Poolside
Towards Shutdownable Agents: Generalizing Stochastic Choice in RL Agents and LLMs
Apr 19, 2026Carissa Cullen, Harry Garland, Alexander Roman +3Agent Reliability AI Agent Safety
University of Oxford · University College London · New College of Florida +3
Answer Only as Precisely as Justified: Calibrated Claim-Level Specificity Control for Agentic Systems
Apr 19, 2026Tianyi Huang, Samuel Xu, Jason Tansong Dang +2Uncertainty Calibration LLM Agent Reliability
Ryquo · App-In Club.
Seeing Isn't Believing: Mitigating Belief Inertia via Active Intervention in Embodied Agents
Apr 19, 2026Hanlin Wang, Chak Tou Leong, Jian Wang +1Embodied AI LLM Agent Reliability
Department of Computing, The Hong Kong Polytechnic University · College of Computer Science, Sichuan University
HiveMind: OS-Inspired Scheduling for Concurrent LLM Agent Workloads
Apr 18, 2026Justice Owusu Agyemang, Jerry John Kponyo, Obed Kwasi Somuah +3LLM Inference Efficiency AI Coding Agents
Sperix Labs · VIA Cybersecurity Lab, KNUST · Quantum and Assistive Technologies Lab, KNUST
Harness as an Asset: Enforcing Determinism via the Convergent AI Agent Framework (CAAF)
Apr 18, 2026Tianbao ZhangSafety-Critical Control LLM Guardrails
Independent Researcher
Know When to Trust the Skill: Delayed Appraisal and Epistemic Vigilance for Single-Agent LLMs
Apr 17, 2026Eren UnluLLM Agents LLM Uncertainty Estimation
Globeholder Paris, France
Don't Start What You Can't Finish: A Counterfactual Audit of Support-State Triage in LLM Agents
Apr 17, 2026Eren UnluLLM Agent Evaluation LLM Refusal Behavior
Globeholder Paris, France
When Agents Go Quiet: Output Generation Capacity and Format-Cost Separation for LLM Document Synthesis
Apr 17, 2026Justice Owusu Agyemang, Michael Agyare, Miriam Kobbinah +2LLM Inference Efficiency Long-Form Document Generation
Sperix Labs · GCTU, Ghana · NCA, Ghana +1
Evaluating Tool-Using Language Agents: Judge Reliability, Propagation Cascades, and Runtime Mitigation in AgentProp-Bench
Apr 17, 2026Bhaskar GurramAI Agent Reliability Tool-Augmented Language Model Agents
Zasti Inc., Ashburn, VA, USA
Weak-Link Optimization for Multi-Agent Reasoning and Collaboration
Apr 17, 2026Haoyu Bian, Chaoning Zhang, Jiaquan Zhang +4Multi-Agent Collaboration Inference-Time Compute Allocation
University of Electronic Science and Technology of China, Chengdu 611731, China · Beihang University, Beijing 100191, China · Xi’an University of Architecture and Technology, Xi’an 710064, China
LLMs Corrupt Your Documents When You Delegate
Apr 17, 2026Philippe Laban, Tobias Schnabel, Jennifer NevilleLLM Reliability Long-Horizon LLM Agents
Microsoft Research
The cognitive companion: a lightweight parallel monitoring architecture for detecting and recovering from reasoning degradation in LLM agents
Apr 15, 2026Rafflesia Khan, Nafiul Islam KhanAI Agent Monitoring LLM Agent Reliability
Software Developer, IBM Dublin, Ireland · Diploma of Computer Science and Engineering Citi Polytechnic Institute Khulna, Bangladesh
Towards Robust Real-World Spreadsheet Understanding with Multi-Agent Multi-Format Reasoning
Apr 14, 2026Houxing Ren, Mingjie Zhan, Zimu Lu +4Multi-Agent LLMs LLM Agent Reliability
CUHK MMLab · SenseTime Research · Shenzhen Loop Area Institute +1
The Long-Horizon Task Mirage? Diagnosing Where and Why Agentic Systems Break
Apr 13, 2026Xinyu Jessica Wang, Haoyue Bai, Yiyou Sun +7Long-Horizon Agent Tasks Agent Failure Analysis
University of Wisconsin–Madison · University of California, Berkeley · Georgia Institute of Technology
From Agent Loops to Structured Graphs:A Scheduler-Theoretic Framework for LLM Agent Execution
Apr 13, 2026Hu WeiLLM Agent Orchestration LLM Agents
Guardrails Beat Guidance: A Large-Scale Study of Rules, Skills, and Persistent Configuration for Coding Agents
Apr 13, 2026Xing Zhang, Guanghui Wang, Yanwei Cui +4Software Engineering Agents LLM Agent Reliability
AWS Generative AI Innovation Center · HSBC Holdings Plc., HSBC Technology Center, China.
Resilient Write: A Six-Layer Durable Write Surface for LLM Coding Agents
Apr 12, 2026Justice Owusu Agyemang, Jerry John Kponyo, Elliot Amponsah +2LLM Agent Reliability AI Coding Agents
Sperixlabs, Ghana · Kwame Nkrumah University of Science and Technology, Kumasi, Ghana · VIA Cybersecurity Lab, Kwame Nkrumah University of Science and Technology, Kumasi, Ghana
From Helpful to Trustworthy: LLM Agents for Pair Programming
Apr 11, 2026Ragib Shahariar AyonAI-Assisted Software Development Software Engineering Agents
Texas State University, San Marcos, TX, USA
Verify Before You Commit: Towards Faithful Reasoning in LLM Agents via Self-Auditing
Apr 9, 2026Wenhao Yuan, Chenchen Lin, Jian Chen +3Faithfulness of Language Model Reasoning LLM Agent Reliability
The University of Hong Kong · Sun Yat-sen University
LogAct: Enabling Agentic Reliability via Shared Logs
Apr 9, 2026Mahesh Balakrishnan, Ashwin Bharambe, Davide Testuggine +7LLM Agent Reliability AI Agent Monitoring
Meta
ReDAct: Uncertainty-Aware Deferral for LLM Agents
Apr 8, 2026Dzianis Piatrashyn, Nikita Kotelevskii, Kirill Grishchenkov +7LLM Agents LLM Agent Reliability
Mohamed bin Zayed University of Artificial Intelligence · Ocapital · National University of Science and Technology (NUST) MISIS +3
The Illusion of Stochasticity in LLMs
Apr 8, 2026Xiangming Gu, Soham De, Michalis Titsias +3LLM Reliability LLM Agent Reliability
Google DeepMind · National University of Singapore
Social Dynamics as Critical Vulnerabilities that Undermine Objective Decision-Making in LLM Collectives
Apr 7, 2026Changgeon Ko, Jisu Shin, Hoyun Song +3Multi-Agent LLM Systems LLM Agent Reliability
Korea Advanced Institute of Science and Technology (KAIST)

Apr 21, 2026J/K move · Enter open · S save

Taming Actor-Observer Asymmetry in Agents via Dialectical Alignment

Bobo Li, Rui Wu, Zibo Ji, Meishan Zhang+4

National University of Singapore · Sichuan University · University of Minnesota Twin Cities+2

AI Agent Benchmarks Multi-Agent Reasoning LLM Agent Reliability

Open paper AskPDF arXiv

Large Language Model agents have rapidly evolved from static text generators into dynamic systems capable of executing complex autonomous workflows. To enhance reliability, multi-agent frameworks assigning specialized roles are increasingly adopted to enable self-reflection and mutual auditing. While such role-playing effectively leverages domain expert knowledge, we find it simultaneously induces a human-like cognitive bias known as Actor-Observer Asymmetry (AOA). Specifically, an agent acting as an actor (during self-reflection) tends to attribute failures to external factors, whereas an observer (during mutual auditing) attributes the same errors to internal faults. We quantify this using our new Ambiguous Failure Benchmark, which reveals that simply swapping perspectives triggers the AOA effect in over 20% of cases for most models. To tame this bias, we introduce ReTAS (Reasoning via Thesis-Antithesis-Synthesis), a model trained through dialectical alignment to enforce perspective-invariant reasoning. By integrating dialectical chain-of-thought with Group Relative Policy Optimization, ReTAS guides agents to synthesize conflicting viewpoints into an objective consensus. Experiments demonstrate that ReTAS effectively mitigates attribution inconsistency and significantly improves fault resolution rates in ambiguous scenarios.