LLM Agent Reliability

LLM: Large Language Model

Overview

LLM agent reliability studies whether agents can complete tasks correctly, follow instructions and safety constraints, use evidence faithfully, and behave consistently across repeated or extended interactions. Common approaches combine structured harnesses and deterministic tool interfaces with state tracking, evidence checks, runtime monitoring, calibrated abstention, and recovery from failures; training methods also teach agents to verify actions or learn from execution feedback. Evaluations increasingly use interactive benchmarks, noisy or changing tools, repeated trials, and trajectory-level measures alongside final task success. A central tension is that added reasoning, delegation, memory, or orchestration can improve some tasks but also introduce new failure paths, so reliability depends on the model, harness, tools, and evaluation setting together.

Latest papers

CardsList

909 papers94 papers in the last four weeks, down 51% on the four weeks before. 0.7% of all new papers.

  1. COCORELI: Enforcing Execution Preconditions for Reliable Collaborative Instruction Following

    Aug 29, 2025Swarnadeep Bhar, Omar Naim, Eleni Metheniti +4Instruction FollowingLLM Agent Planning

  2. Beyond Semantic Similarity: Reducing Unnecessary API Calls via Behavior-Aligned Retriever

    Aug 20, 2025Yixin Chen, Ying Xiong, Shangyu Wu +4In-Context Example SelectionIn-Context Retrieval

  3. EndoCogniAgent: Closed-Loop Agentic Reasoning with Self-Consistency Validation for Endoscopic Diagnosis

    Aug 10, 2025Yi Tang, Kai-Ni Wang, Liang-Peng Pu +3EndoscopyMedical Imaging

  4. From the Fluency Fallacy to the Micro-to-Macro Validity Gap: Opportunities and Pitfalls of LLMs in Social Simulation

    Jul 25, 2025Patrick Taillandier, Jean Daniel Zucker, Arnaud Grignard +4Human Behavior SimulationComputational Social Science

  5. Towards Mitigation of Hallucination for LLM-empowered Agents: Progressive Generalization Bound Exploration and Watchdog Monitor

    Jul 21, 2025Siyuan Liu, Wenjing Liu, Zhiwei Xu +3LLM Hallucination MitigationHallucination in Language Models

  6. SheetMind: Actions Set Accuracy, Agents Set the Failure Mode

    Jun 14, 2025Lyuhao Chen, Xi Cheng, Yanming Kang +11Multi-Agent LLM SystemsLLM Agent Evaluation

  7. DrugAgent: Reliable Multi-Agent Integration of Conflicting Biomedical Evidence for Drug-Target Interaction Assessment

    Aug 23, 2024Yoshitaka Inoue, Tianci Song, Xinling Wang +3Multi-Agent LLM SystemsLLM Agent Reliability