AI Agent Reliability

Ask about this topic

Momentum

34 papers in the last four weeks, up 240% on the four weeks before. 0.3% of all new papers.

Jul 13Week of Sep 28
View on map

Latest papers 215

All topics
CardsList

  1. Verified Tool Calls Improve LLM Agent Reliability Under Non-Atomic Failures

    Jul 31, 2026Isham Kalappurackal Mansoor, Abhishek Phadke, Pratip RanaAI Agent ReliabilityLLM Tool Use

    Department of Computer Science Old Dominion University Norfolk, VA 23529 · School of Engineering and Computing Christopher Newport University Newport News, VA 23606

  2. InfraBench: Evaluating Infrastructure Agents Across Layers, Lifecycle, and Risk

    Jul 31, 2026Yuan Gao, Zeren Yang, Junnan Li +6AI Agent ReliabilityAI Agent Evaluation

    Wanxiang · University of Wisconsin–Madison · 2Iowa State University

  3. One Human, NN Agents: Audit-Budget Allocation for LLM Agent Fleets under Miscalibrated, Correlated Confidence

    Jul 30, 2026Cesare Zavattari, Alessandro Tommasi, Giuseppe PrencipeScalable OversightAI Agent Reliability

    Dipartimento di Informatica, Università di Pisa, Pisa, Italy

  4. Fidelity Is Not Safety: Gently-Compressed LLMs Pass Every Data-Free Quality Guard Yet Invent Procedure Steps in Agentic Execution

    Jul 30, 2026I. Kennedy, T. KennedyModel CompressionAI Agent Reliability

  5. ClawTrack: Towards Trace-Level Evaluation and Improvement of Real-World Autonomous Agents

    Jul 30, 2026Xingjian Wu, Xuhang Zhu, Xingchen Liu +6AI Agent ReliabilityLLM Agent Evaluation

    Meituan

  6. ΣΣ-Mem: An Online Reliability Memory for LLM-based Multi-Agent Systems

    Jul 30, 2026Peilin Feng, Suorong Yang, Soujanya PoriaAI Agent ReliabilityMulti-Agent LLM Systems

    1DeCLaRe Lab, Nanyang Technological University

  7. Stop Shipping AI Agents on Faith: Capability Is Not Production Readiness

    Jul 30, 2026Fouad BousetouaneAI Agent ReliabilityAI Agent Evaluation

    The University of Chicago, USA

  8. LayerRAG-Bench: A Cross-Layer Reliability Benchmark for Agentic Retrieval-Augmented Generation

    Jul 29, 2026Musa ShamsAI Agent ReliabilityRetrieval-Augmented Generation

    Independent Researcher

  9. Think Short, Defer Smart, Act, and Repeat: Calibrated Reasoning and Uncertainty-Aware Deferral for Edge LLM Agents

    Jul 29, 2026Amirmohammad Farzaneh, Osvaldo SimeoneLLM Inference EfficiencyAI Agent Reliability

    Institute for Intelligent Networked Systems (INSI) Northeastern University London London, UK

  10. Flat Score, Amplified Failures: How the Error Budget Masks Damage in Quantized LLM Agents

    Jul 29, 2026Jiwon Jang, Kisu Yang, Heuiseok Lim +1AI Agent ReliabilityBenchmark Design

    VAIV Company · Seoul National University · Korea University

  11. Living-Harness Is an Interactive-Agent Evolver

    Jul 29, 2026Yuetian Du, Yucheng Wang, He Xu +9AI Agent ReliabilityAgent Harness Optimization

    Zhejiang University · The Hong Kong University of Science and Technology · Alibaba Group +2

  12. SARC-DQ: Runtime Data-Quality Gating for Agentic AI: Silent Evidence Defects, the Incompetence Shield, and Downstream-Only Remediation

    Jul 28, 2026Gaston BesansonAI Agent ReliabilityData Provenance

    Universidad Torcuato Di Tella

  13. HANDBOOK.md: A Benchmark for Long-Context Agentic Instruction Following

    Jul 28, 2026Liudas Panavas, Sebastian Minus, Bradley Monton +4Computer-Use Agent BenchmarksAI Agent Reliability

    Surge AI*

  14. When Do Agent Loops Mistake Stagnation for Progress? Self-Evaluation Bias and Externally Grounded Verification in Long-Running Autonomous LLM Agent Loops

    Jul 27, 2026Hyundoo Park, Byungho ChoiAI Agent ReliabilityLLM Self-Refinement

    AsyncSite

  15. Stress-testing large language model agents in a robotic chemistry laboratory

    Jul 25, 2026Lulu Guo, Yingkai Sun, Xiaobo Li +13AI Agent ReliabilityAI Agents for Scientific Discovery

    State Key Laboratory of Precision and Intelligent Chemistry, Hefei National Research Center for Physical Sciences at the Microscale, School of Chemistry and Materials Science, University of Science and Technology of China, Hefei, China · Center for Scientific Intelligence Innovation, Hefei, China · School of Chemistry, School of Computer Science, University of Birmingham, Birmingham, UK

  16. Building AI That Works: ESnet's Pragmatic Approach to AI-Driven Operational Excellence

    Jul 24, 2026Bin Dong, Sukhada Gholba, Brooklin Gore +15AI Agent ReliabilityAgentic Workflows

    Energy Science Network

  17. Toward Continuous Assurance for the Democratization of AI Agent Creation in Industry

    Jul 23, 2026Natan Levy, Harel BergerAI Agent ReliabilityAI Agent Monitoring

    School of Computer Science and Engineering · The Department of Computer and Software Engineering

  18. GuardianAgentBench: Where Agents Fail and How to Guard Them

    Jul 23, 2026Vishal Ishwar Naik, Chenyu Xu, Donna Dong +5AI Agent ReliabilityLLM Guardrails

    Vectara, Inc., Palo Alto, CA, USA · Iowa State University, Ames, IA, USA

  19. Agents in the Wild: Where Research Meets Deployment

    Jul 21, 2026Grace Hui Yang, Pranav N. Venkit, Hooman Sedghamiz +3AI Agent ReliabilityMulti-Agent LLM Systems

    Georgetown University · Washington, D.C., USA · Salesforce +7

  20. Guardrails as Scapegoats: Auditing Unfaithful Safety Refusals in Tool-Augmented LLM Agents

    Jul 21, 2026Aarushi SinghAI Agent ReliabilityLLM Agent Evaluation

    Independent Researcher India

  21. AgentDebugX: An Open-Source Toolkit for Failure Observability, Attribution, and Recovery in LLM Agents

    Jul 21, 2026Kunlun Zhu, Xuyan Ye, Zhiguang Han +9Agent Failure AnalysisAI Agent Reliability

    University of Illinois Urbana-Champaign · University of Toronto · Google +1

  22. Autoresearch with Coding Agents: Generalizers and Metric-Maximizers on Quran Recitation Data

    Jul 20, 2026Nursultan Askarbekuly, Mohamad Al Mdfaa, Ahmed Helaly +2AI Agent ReliabilityReward Hacking

    Innopolis University, Innopolis, Republic of Tatarstan, Russia · Skolkovo Institute of Science and Technology, Moscow, Russia

  23. Zero Hallucination, by Construction: Hallucination-Aware Layered Oversight for Trustworthy Enterprise AI

    Jul 20, 2026Bogdan Raduta, Horia Velicu, Alexandru Preda +1AI Agent ReliabilityLLM Hallucination Mitigation

  24. Verify, Repair, Repeat, or Stop? Robust Stopping for Noisy Verify-Repair Loops in LLM Agents

    Jul 20, 2026Yitao Wu, Si Shen, Rui Yang +2AI Agent ReliabilityLLM Self-Correction

    saofund.ai, Shenzhen, China · Shenzhen Xingqing Zhiti Technology Co., Ltd., Shenzhen, China · School of Information Technology and Engineering, Lanzhou University, Lanzhou, China

  25. DRNOISE: Benchmarking Deep Research Agents in Misleading Evidence Environments

    Jul 19, 2026Jun Nie, Zhiqin Yang, Zhenheng Tang +4AI Agent ReliabilityDeep Research Agents

    Hong Kong Baptist University · University of Science and Technology of China · The Hong Kong University of Science and Technology

  26. Teach it to stop, not just to click

    Jul 19, 2026Barada Sahu, Shivesh PandeyAI Agent ReliabilityAgent Reliability

    Cabal AI · Para AI

  27. Where Does Agent Reliability Come From? A Cross-Benchmark Decomposition of Verification Loops, Specialist Models, and Scaffolding in a Production Enterprise Agent

    Jul 19, 2026Arunabh DastidarAI Agent ReliabilityLLM Answer Verification

    for the Leni Team · Leni Inc.

  28. AI Agents Do Not Fail Alone:The Context Fails First

    Jul 15, 2026Fouad BousetouaneAI Agent ReliabilityAgent Reliability

    The University of Chicago, USA

  29. DeepStress: Stress-Testing Deep Search Agents

    Jul 15, 2026Ismael Rousseau, Geraldine Damnati, Frederic BechetAI Agent ReliabilityAI Agent Evaluation

    Orange Research, Lannion, France · Aix-Marseille Univ., CNRS, LIS UMR 7020, 13000 Marseille, France

  30. Critic Experience Bank: Self-Evolving Step-Level Confidence Estimation for LLM Agents

    Jul 14, 2026Yaopei Zeng, Congchao Wang, JianHang Chen +3AI Agent ReliabilityLLM Agent Evaluation

    Pennsylvania State University · Virginia Tech · Purdue University +1

  31. AgentCheck: A Reproduce-Intervene-Mitigate Workbench for LLM Agents over MCP

    Jul 13, 2026Aritra Mazumder, Nusrat jahan LiaAI Agent ReliabilityLLM Agent Evaluation

    University of Utah · University of Dhaka

  32. To Answer or to Abstain: Mitigating Search-Agent Hallucinations via Abstention-Aware Reinforcement Learning

    Jul 12, 2026Fengji Zhang, Tianyu Fan, Yuxiang Zheng +4AI Agent ReliabilityLLM Hallucination Mitigation

    City University of Hong Kong · Alibaba Group

  33. Confining Nondeterminism: AI-Driven Research Systems as DBMSs for Reliable, Non-Wasteful, Transparent, and Collaborative Research [Vision]

    Jul 11, 2026Kyoungmin Kim, Anastasia AilamakiAI Agent ReliabilityAutonomous Research Agents

    EPFL Lausanne, Switzerland

  34. Compaction as Epistemic Failure: How Agentic LLM Tools Fabricate Confirmed Results from Killed Processes

    Jul 11, 2026Hiroki TambaAI Agent ReliabilityLLM Agent Memory

    Independent Researcher

  35. AgentAbstain: Do LLM Agents Know When Not to Act?

    Jul 11, 2026Xun Liu, Yi Evie Zhang, Vira Kasprova +5AI Agent ReliabilityLLM Agent Evaluation

    University of Illinois Urbana-Champaign

  36. Scoped Verification for Reliable Long-Horizon Agentic Context Evolution under Distribution Shift

    Jul 10, 2026Dan C. Hsu, Luke LuAI Agent ReliabilityLong-Horizon Agent Evaluation

    RedMind Research, San Francisco, CA, USA · National Taiwan University, Taipei, Taiwan

  37. Agent Delivery Engineering Predictive Reliability Framework

    Jul 8, 2026Dexing LiuAI Agent ReliabilityMulti-Agent LLM Systems

    Shanghai Qijing Digital Technology Co., Ltd · Shanghai Qijing Digital Technology Co., Ltd.

  38. Reason Less, Verify More: Deterministic Gates Recover a Silent Policy-Violation Failure Mode in Tool-Using LLM Agents

    Jul 8, 2026Vikas Reddy, Sumanth Reddy Challaram, Abhishek BasuAgent Failure AnalysisAI Agent Reliability

    Independent Researcher · Indian Institute of Technology Kharagpur Kharagpur, India · Massachusetts Institute of Technology Cambridge, USA

  39. Agentic Data Environments

    Jul 8, 2026Elaine Ang, Chenxi Huang, Georgios Liargkovas +13AI Agent ReliabilityAI Agent Safety

    Columbia University

  40. Beyond the Leaderboard: A Synthesis of Tool-Use, Planning, and Reasoning Failures in Large Language Model Agents

    Jul 7, 2026Wael Albayaydh, Rui Zhao, Ivan FlechaisAI Agent ReliabilityLLM Agent Evaluation

    University of Oxford

  41. ToolFailBench: Diagnosing Tool-Use Failures in LLM Agents

    Jul 6, 2026Harsh SoniAI Agent ReliabilityTool-Use Evaluation

    UC Berkeley

  42. The "I Don't Know" Filter: Enhancing Agentic Reliability in Function Calling

    Jul 4, 2026Stefan Broecker, Mason del Rosario, Boris Selitser +1AI Agent ReliabilityLLM Agent Evaluation

    University of California, Davis, Department of Computer Science · Okareo, Inc. · University of California, Davis, Department of Mathematics

  43. The Remarkable Effectiveness of Providing AI Agents with Natural Language Tools: A Replication Study Validating NLT Performance Across 14 Models

    Jul 4, 2026Alexander Somma, Isabelle Plante, Fred PremjiAI Agent ReliabilityLLM Agent Evaluation

    Sage.is AI-UI

  44. Reasoning effort, not tool access, buys first-try reliability in agentic code generation: an observational study

    Jul 2, 2026Achint MehtaAI Agent ReliabilityTest-Time Scaling

    TrendAI

  45. When the Database Fails: Prompting LLM Dialogue Agents for Safe Recovery in Task-Oriented Dialogue

    Jun 30, 2026Mohammad Alijanpour Shalmani, Alale Rezvani Boroujeni, Jiann Shiun YuanAI Agent ReliabilityLLM Hallucination Mitigation

    College of Engineering and Computer Science, University of Central Florida, Orlando, Florida · Department of Marketing, University of Central Florida, Orlando, Florida

  46. Certified Speculative Execution for Untrusted AI Agents

    Jun 30, 2026Chenyu Zhou, Qiliang Jiang, Shuning Wu +1AI Agent ReliabilityConstrained RL

    School of Engineering, Institute of Science Tokyo, Japan · College of Control Science and Engineering, Zhejiang University, China · Department of Electrical and Computer Engineering, National University of Singapore, Singapore

  47. SEVA: Self-Evolving Verification Agent with Process Reward for Fact Attribution

    Jun 29, 2026Aojie Yuan, Yi Nian, Haiyue Zhang +2AI Agent ReliabilityProcess Reward Models

    University of Southern California, Los Angeles, USA · University of Michigan, Ann Arbor, USA

  48. Budgeted Act-or-Defer Multi-Agent LLM Deliberation with Local Reliability Bounds

    Jun 28, 2026Mengdie Flora Wang, Haochen Xie, Guanghui Wang +2AI Agent ReliabilityMulti-Agent LLM Systems

    1AWS Generative AI Innovation Center, USA · General Motors (GM), USA

  49. Manufactured Confidence: How Memory Consolidation Turns Hearsay into Confident Facts

    Jun 28, 2026Alex KwonAI Agent ReliabilityLLM Agent Memory

    Independent Researcher

  50. Preventing Error Propagation in Multi-Agent AI through Runtime Monitoring

    Jun 27, 2026Shahnewaz Karim Sakib, Anindya Bijoy DasAI Agent ReliabilityMulti-Agent Systems

    University of Tennessee at Chattanooga, TN 37403, USA · The University of Akron, OH 44325, USA

  51. Delayed Verification Destabilizes Multi-Agent LLM Belief: Instability Thresholds and Optimal Corrector Placement

    Jun 25, 2026Igor ItkinAI Agent ReliabilityMulti-Agent Coordination

    Independent Researcher, Tel Aviv, Israel

  52. Beyond Function Calling: Benchmarking Tool-Using Agents under Tool-Environment Unreliability

    Jun 24, 2026Yang Tian, Zhengpeng Shi, Yu Zhou +1AI Agent ReliabilityAI Agent Benchmarks

    School of AI, Shanghai Jiao Tong University1 · School of Cyber Science and Engineering, Southeast University2

  53. Position: Correct Answer, Wrong Mechanism -- When AI Scientists Defend General Claims Their Own Data Contradicts

    Jun 22, 2026Steven Young EuligAI Agent ReliabilityReasoning Evaluation

    Department of Physics and Laboratory for Particle Physics and Cosmology (LPPC), Harvard University, Cambridge, MA 02138, USA.

  54. When Agents Commit Too Soon: Diagnosing Premature Commitment in LLM Agents

    Jun 22, 2026Aman MehtaAI Agent ReliabilityLLM Agent Evaluation

    Snowflake AI Research

  55. GroundEval: A Deterministic Replacement for LLM-as-Judge in Stateful Agent Evaluation

    Jun 22, 2026Jeffrey FlyntAI Agent ReliabilityTool-Use Evaluation

    Independent Researcher

  56. RigorBench: Benchmarking Engineering Process Discipline in Autonomous AI Coding Agents

    Jun 21, 2026Meher Bhaskar Madiraju, Meher Sai Preetam MadirajuAI Agent ReliabilityCoding Agents

  57. Grounded Scaling: Why Agentic AI Needs Deterministic Environments

    Jun 21, 2026Liang Ding, Xintong WangAI Agent ReliabilityLong-Horizon Agent Tasks

    1Alibaba Group

  58. OpenBioRQ: Unsolved Biomedical Research Questions for Agents

    Jun 20, 2026Minbyul JeongAI Agent ReliabilityCitation Verification

    Upstage AI

  59. Don't Blindly Trust It: How Unreliable Feedback Breaks Tool-Using LLM Agents

    Jun 19, 2026Chubin Zhang, Zhenglin Wan, Xingrui Yu +5AI Agent ReliabilityTool-Use Evaluation

    Nanyang Technological University, Singapore · National University of Singapore, Singapore · CFAR, Agency for Science, Technology and Research, Singapore +3

  60. EARS: Explanatory Abstention for Reliable Sub-Agent Modeling in Large-scale Multi-Agent Systems

    Jun 17, 2026Shuang Xie, Yunan Lu, Han Li +1AI Agent ReliabilityMulti-Agent LLM Systems

    Shopify · Columbia University