AI Agent Reliability

Follow this topic Ask about this topic

AI agent reliability asks whether an agent can complete goals correctly and consistently—not merely produce a plausible answer—while respecting permissions, grounding actions in evidence, and recovering from errors. Researchers measure repeated-run consistency, pass@k, calibration and abstention, trajectory quality, policy compliance, and verified changes to environment state on benchmarks such as τ-bench, SWE-bench, and OSWorld. Common approaches include fault-injection tests, trace-based failure diagnosis, formal contracts, deterministic runtime gates, tool-output validation, and uncertainty-aware escalation or rollback. Results show that model capability and average task scores can hide silent failures, drift, and poor recovery; the challenge is to provide deployment-representative assurance without excessive cost or restricting useful autonomy.

Latest papers

340 papers51 papers in the last four weeks, up 21% on the four weeks before. 0.3% of all new papers.

Evaluating Deep Research Agents on Expert Consulting Work: A Benchmark with Verifiers, Rubrics, and Cognitive Traps
May 17, 2026Tanmay Asthana, Aman Saksena, Divyansh SahuAI Agent Reliability Deep Research Agents
Deccan AI Research
Trust No Tool: Evaluating and Defending LLM Agents under Untrusted Tool Feedback
May 17, 2026Lecheng Yan, Ruizhe Li, Xicheng Han +5AI Agent Reliability LLM Agent Evaluation
Southern University of Science and Technology · University of Science and Technology of China · Zhejiang University +2
Towards trustworthy agentic AI: a comprehensive survey of safety, robustness, privacy, and system security
May 17, 2026Jinhu Qi, Muzhi Li, Jiahong Liu +9AI Agent Reliability AI Agent Evaluation
Faculty of Engineering, Department of Computer Science and Engineering, The Chinese University of Hong Kong, Hong Kong, China. · Artificial Intelligence Innovation and Incubation Institute, Fudan University, Shanghai, China. · Shanghai Academy of AI for Science, Shanghai, China.
ContractBench: Can LLM Agents Preserve Observation Contracts?
May 17, 2026Jicheng Wang, Yifeng He, Zili Wang +3AI Agent Reliability LLM Agent Evaluation
University of California, Davis · University of Southern California · University of Hong Kong
Reliability and Effectiveness of Autonomous AI Agents in Supply Chain Management
May 16, 2026Carol Xuan Long, David Simchi-Levi, Feng Zhu +3AI Agent Reliability Multi-Agent Systems
Harvard University · MIT/Purdue · MIT +2
R2V Agent: Teaching SLMs When to Ask for Help
May 15, 2026Raghu Vamshi Hemadri, Humaira Firdowse Mohammed, Rishabh Maheshwary +5AI Agent Reliability Tool-Augmented Language Model Agents
New York University Tandon School of Engineering · University of California, San Diego · ServiceNow Research +3
PQR: A Framework to Generate Diverse and Realistic User Queries that Elicit QA Agent Failures
May 15, 2026Yunan Lu, Luigi Liu, Omar Yahia +2AI Agent Reliability Adversarial Prompt Generation
Columbia University · University of California San Diego · Walmart
GraphFlow: An Architecture for Formally Verifiable Visual Workflows Enabling Reliable Agentic AI Automation
May 14, 2026Drewry H. Morris, Luis Valles, Reza Hosseini GhomiAI Agent Reliability Agentic Workflows
MedFlow, Inc.
Trustworthy Multi-Agent Systems: Mitigating Semantic Drift with the Argent Signaling Protocol
May 14, 2026Anantha SharmaAI Agent Reliability Multi-Agent LLM Systems
Synechron Inc · Charlotte, NC, USA
Good to Go: The LOOP Skill Engine That Hits 99% Success and Slashes Token Usage by 99% via One-Shot Recording and Deterministic Replay
May 14, 2026Xiaohua Wang, Kai Yu, XuXiao Liang +2LLM Inference Efficiency AI Agent Reliability
1Artificial Intelligence Research Center, Bengbu Medical University, Bengbu 233000, China. · 2CHARMMIRAEL Biotech Co., Ltd, Nanjing 210000, China.
ChromaFlow: A Negative Ablation Study of Orchestration Overhead in Tool-Augmented Agent Evaluation
May 13, 2026Tarun MittalAI Agent Reliability LLM Agent Orchestration
Octave-X
Do Biological Structural Guarantees Earn Their Complexity?
May 13, 2026Bogdan BanuAI Agent Reliability AI Agent Benchmarks
Not Just RLHF: Why Alignment Alone Won't Fix Multi-Agent Sycophancy
May 13, 2026Adarsh Kumarappan, Ananya MujooAI Agent Reliability Multi-Agent LLM Systems
1California Institute of Technology · 2Evergreen Valley College
Useful Memories Become Faulty When Continuously Updated by LLMs
May 13, 2026Dylan Zhang, Yanshan Lin, Zhengkun Wu +4AI Agent Reliability Persistent Memory for Language Models
University of Illinois Urbana-Champaign · IIIS, Tsinghua University
No Action Without a NOD: A Heterogeneous Multi-Agent Architecture for Reliable Service Agents
May 12, 2026Zixu Yang, Hang Zheng, Nan Jiang +5AI Agent Reliability Multi-Agent LLM Systems
1X-LANCE Lab, School of Computer Science, Shanghai Jiao Tong University, Shanghai, China · 2Shanghai Innovation Institution, Shanghai, China · 3Jiangsu Key Lab of Language Computing, Suzhou, China
Consistency as a Testable Property: Statistical Methods to Evaluate AI Agent Reliability
May 11, 2026Harsh Raj, Niranjan Orkat, Suvrorup Mukherjee +3AI Agent Reliability Agent Reliability
Northeastern University · University of P´ecs · University of Michigan +2
Nautilus Compass: Black-box Persona Drift Detection for Production LLM Agents
May 11, 2026Chunxiao WangAgent Memory AI Agent Reliability
Yiluo Technology Co., Ltd.
A Communication-Theoretic Framework for LLM Agents: Cost-Aware Adaptive Reliability
May 9, 2026Hamed Omidvar, Vahideh AkhlaghiLLM Inference Efficiency AI Agent Reliability
INTELLERCE LLC San Jose, CA, USA
When Agents Overtrust Environmental Evidence: An Extensible Agentic Framework for Benchmarking Evidence-Grounding Defects in LLM Agents
May 9, 2026Strick Sheng, Ziyue Wang, Liyi ZhouAI Agent Reliability LLM Agent Evaluation
The University of Sydney · Nanjing University
AgentCollabBench: Diagnosing When Good Agents Make Bad Collaborators
May 9, 2026Aritra Mazumder, Shubhashis Roy Dipta, Nusrat Jahan Lia +10AI Agent Reliability Multi-Agent Collaboration
University of Utah · University of Maryland, Baltimore County · University of Dhaka +7
Tool Calling is Linearly Readable and Steerable in Language Models
May 8, 2026Zekun Wu, Ze Wang, Seonglae Cho +4AI Agent Reliability Tool-Augmented Language Model Agents
University College London · Holistic AI · Imperial College London
Beyond the Black Box: Interpretability of Agentic AI Tool Use
May 7, 2026Hariom Tatsat, Ariye ShaterAI Agent Reliability Mechanistic Interpretability
Quantitative Analytics, Barclays
Conformal Agent Error Attribution
May 7, 2026Naihe Feng, Yi Sui, Shiyi Hou +2Agent Failure Analysis AI Agent Reliability
Dalhousie University · Halifax, NS, Canada · Layer 6 AI +1
MANTRA: Synthesizing SMT-Validated Compliance Benchmarks for Tool-Using LLM Agents
May 7, 2026Ashwani Anand, Ivi Chatzi, Ritam Raha +1Computer-Use Agent Benchmarks AI Agent Reliability
Max Planck Institute for Software Systems, Kaiserslautern, Germany
Robust Agent Compensation (RAC): Teaching AI Agents to Compensate
May 5, 2026Srinath Perera, Kaviru Hapuarachchi, Frank Leymann +1AI Agent Reliability LLM Agent Reliability
WSO2 Santa Clara, USA · University of Stuttgart Stuttgart, Germany
ARIS: Autonomous Research via Adversarial Multi-Agent Collaboration
May 4, 2026Ruofeng Yang, Yongcan Li, Shuai LiAI Agent Reliability Multi-Agent LLM Systems
Shanghai Jiao Tong University · Shanghai Innovation Institute
When Alignment Isn't Enough: Response-Path Attacks on LLM Agents
May 4, 2026Mingyu Luo, Zihan Zhang, Zesen Liu +7AI Agent Reliability LLM Agent Security
Fudan University, Shanghai, China · The Hong Kong University of Science and Technology, Hong Kong, China · Huazhong University of Science and Technology, Wuhan, China
Reliable AI Needs to Externalize Implicit Knowledge: A Human-AI Collaboration Perspective
May 3, 2026Hengyu Liu, Tianyi Li, Zhihong Cui +5AI Agent Reliability Human-in-the-Loop AI
Department of Computer Science, Aalborg University, Aalborg, Denmark · Department of Informatics, University of Oslo, Oslo, Norway · School of Computing, Macquarie University, Sydney, Australia
NeuroState-Bench: A Human-Calibrated Benchmark for Commitment Integrity in LLM Agent Profiles
May 3, 2026Xiao JiaAI Agent Reliability LLM Agent Evaluation
School of Artificial Intelligence The Chinese University of Hong Kong, Shenzhen · The Chinese University of Hong Kong, Shenzhen
Reward Hacking Benchmark: Measuring Exploits in LLM Agents with Tool Use
May 3, 2026Kunvar ThamanAI Agent Reliability Reward Hacking
Independent Researcher

May 17, 2026J/K move · Enter open · S save

Evaluating Deep Research Agents on Expert Consulting Work: A Benchmark with Verifiers, Rubrics, and Cognitive Traps

Tanmay Asthana, Aman Saksena, Divyansh Sahu

Deccan AI Research

AI Agent Reliability Deep Research Agents LLM Agent Evaluation LLM Agent Verification AI Agent Benchmarks Agentic Reasoning+1

Open paper AskPDF arXiv

Frontier deep research agents (DRAs) are being deployed in enterprise workflows faster than they are being evaluated. Existing benchmarks measure factual recall, single-hop QA, or generic agentic skill, and miss the multi-document, decision-grade deliverables DRAs are asked to produce. We introduce a benchmark of 70 SME-authored management consulting prompts, each embedding cognitive traps that penalize surface-pattern reasoning. Three frontier agents, namely Claude Opus~~4.6, OpenAI o3-deep-research and Gemini~~3.1Pro deep-research, are scored on two complementary layers: deterministic binary verifiers (mean 14.9 per task) and a five-criterion 0--3 SME rubric (Data Integrity, Analytical Rigor, Relevance & Focus, Execution Precision, Format & Deliverability), combined into a Verifier-Rubric Score (VRS, 0--100). Acceptance under a joint threshold (rubric mean $\geq 2.5$ and verifier pass rate $\geq 80\%$ ) is uniformly low: o3 15.7%, Claude 12.9%, Gemini 12.9%. Pairwise differences are statistically indistinguishable. On the continuous VRS, o3 leads (61.4[CI: 55.2,,67.5]), followed by Gemini (52.6) and Claude (38.5); the o3--Claude gap (

Δ{=}22.9

p{<}0.001

) survives Bonferroni correction. No agent averages above the rubric's ``adequate'' threshold of 2.0; no agent's mean verifier pass rate reaches the 80% acceptance floor. Each agent fails distinctively: Claude leads on data fabrication and file-access failures; o3 propagates cascading computation errors; Gemini oscillates between the highest perfect-verifier rate and the most catastrophic collapses. The benchmark, evaluation code, and full prompt corpus are publicly released.