LLM Agent Reliability

LLM: Large Language Model

Follow this topic Ask about this topic

Momentum

90 papers in the last four weeks, up 109% on the four weeks before. 0.9% of all new papers.

Jul 13Week of Sep 28

Latest papers 558

All topics

Critic-Guided Heterogeneous Multi-Agent Reasoning for Reliable Mathematical Problem Solving
Jun 4, 2026Muhammad Talha Sharif, Abdul RehmanLLM Self-Correction Multi-Agent Reasoning
dept. of Computer Science · National University of Computer and Emerging Sciences · Islamabad, Pakistan
ZERO-APT: A Closed-Loop Adversarial Framework for LLM-Driven Automated Penetration Testing under Intelligent Defense
Jun 4, 2026Anlan Zheng, Tiantian ZhuAI Agent Security Benchmarks Automated Penetration Testing
Zhejiang University of Technology Hangzhou, China
When Evidence is Sparse: Weakly Supervised Early Failure Alerting in Dialogs and LLM-Agent Trajectories
Jun 3, 2026Avinash Baidya, Xinran Liang, Ruocheng Guo +2Weakly Supervised Learning LLM Agent Reliability
1Intuit AI Research · 2Princeton University
A Taxonomy of Runtime Faults in Model Context Protocol Servers
Jun 3, 2026Joshua Owotogbe, Indika Kumara, Willem-Jan van den Heuvel +3Agent Failure Analysis Model Context Protocol
Jheronimus Academy of Data Science and Tilburg University, The Netherlands · University of Sannio, Benevento, Italy · University of Luxembourg, Luxembourg +1
Provably Auditable and Safe LLM Agents from Human-Authored Ontologies
Jun 3, 2026Aaron SterlingAI Agent Auditing LLM Auditing
Thistleseeds
Channel Fracture: Three Instances of Cross-Boundary Silent Delivery Reliability Failures in Multi-Agent Systems
Jun 3, 2026Dexing LiuAgent Failure Analysis Multi-Agent Systems
Shanghai Qijing Digital Technology Co., Ltd.
Agent Planning Benchmark: A Diagnostic Framework for Planning Capabilities in LLM Agents
Jun 3, 2026Haoyu Sun, Wenxuan Wang, Mingyang Song +5Tool-Augmented Language Model Agents LLM Agent Evaluation
Tongji University · Shanghai AI Laboratory · Harbin Institute of Technology +5
AIP: A Graph Representation for Learning and Governing Agent Skills
Jun 3, 2026Zachary Blumenfeld, Jim WebberAI Agent Reliability Agent Skill Learning
Neo4j · MD, USA · London, UK
Cascading Hallucination in Agentic RAG: The CHARM Framework for Detection and Mitigation
Jun 3, 2026Saroj MishraLLM Hallucination Mitigation Hallucination in Language Models
University of North Dakota
SMAC-Talk: A Natural Language Extension of the StarCraft Multi-Agent Challenge for Large Language Models
Jun 2, 2026Joel Sol, Homayoun NajjaranMulti-Agent Coordination LLM Agent Evaluation
Faculty of Engineering and Computer Science University of Victoria Victoria, BC, Canada
Lean4Agent: Formal Modeling and Verification for Agent Workflow and Trajectory
Jun 2, 2026Ruida Wang, Jerry Huang, Pengcheng Wang +3Software Engineering Agents Formal Verification
University of Illinois Urbana-Champaign · Independent researcher
Token Budgets: An Empirical Catalog of 63 LLM-Agent Budget-Overrun Incidents, with an Affine-Typed Rust Mitigation as a Case Study
Jun 2, 2026Sajjad KhanAgent Failure Analysis Multi-Agent LLM Systems
independent researcher; MSc in Data Science, University of the West of England, Bristol, UK
Perceive Before Reasoning: A Pre-Reasoning Perception Framework for Efficient and Reliable Proactive Mobile Agents
Jun 2, 2026Zhijie Ding, Weinan Hong, Zicheng Zhu +6Efficient Multimodal Inference Proactive Assistance
HyperAI Team, Xiaomi Corporation · Zhongnan University of Economics and Law · 1HyperAI Team, Xiaomi Corporation +5
Capability Advertisement as a Market for Lemons: A Trust Layer for Heterogeneous Agent Networks
Jun 2, 2026Gaurav Naresh MittalAI Agent Reliability Agent Communication Protocols
SPADE-Bench: Evaluating Spontaneous Strategic Deception in Agents via Plan-Action Divergence
Jun 1, 2026Yuyan Bu, Haowei Li, Qirui Zheng +7LLM Agent Evaluation AI Agent Safety
Beijing Academy of Artificial Intelligence · University of Science and Technology of China · Peking University +2
Where Do Deep-Research Agents Go Wrong? Span-Level Error Localization in Agent Trajectories
Jun 1, 2026Jiaming Wang, Ziteng Feng, Jiangtao Wu +8Agent Failure Analysis Deep Research Agents
NJU-LINK Team, Nanjing University · JIUTIAN Research
QoEReasoner: An Agentic Reasoning Framework for Automated and Explainable QoE Diagnosis in RANs
Jun 1, 2026Qizhe Li, Haolong Chen, Shan Dai +5LLM Agent Orchestration Wireless Communications
The Chinese University of Hong Kong, Shenzhen · Shenzhen Research Institute of Big Data · Huawei Technologies Co., Ltd. +1
Agent System Operations: Categorization, Challenges, and Future Directions
Jun 1, 2026Zexin Wang, Changhua Pei, Yuanhao Liu +10Agent Failure Analysis AI Agent Monitoring
CNIC, CAS · Hangzhou Institute for Advanced Study, UCAS · Singapore Management University +1
Self-Healing Agentic Orchestrators for Reliable Tool-Augmented Large Language Model Systems
May 31, 2026Rahul Suresh Babu, Adarsh AgrawalLLM Agent Orchestration LLM Agent Reliability
Independent Researcher United States of America · Senior Member, IEEE United States of America
Sophrosyne: Agentic Exploration of Relational Data Systems Needs Moderation
May 29, 2026Madhav Jivrajani, Ramnatthan Alagappan, Aishwarya GanesanText-to-SQL Tool-Using Language Agents
University of Illinois Urbana-Champaign
Counterfactual Graph for Multi-Agent LLM Calibration
May 28, 2026Jiatan Huang, Mingchen Li, Ziming Li +3AI Agent Reliability Language Model Calibration
University of Connecticut · University of Massachusetts, Amherst
Physics Is All You Need? A Case Study in Physicist-Supervised AI Development of Scientific Software
May 28, 2026Nhat-Minh NguyenAI for Science AI Coding Agents
Kavli IPMU (WPI), UTIAS, The University of Tokyo, 5-1-5 Kashiwanoha, Kashiwa, Chiba 277-8583, Japan · Center for Data-Driven Discovery, Kavli IPMU (WPI), UTIAS, The University of Tokyo, Kashiwa, Chiba 277-8583, Japan · Institute For Interdisciplinary Research in Science and Education, ICISE, Quy Nhon, 55121, Vietnam
Locally Coherent, Globally Incoherent: Bounding Compositional Incoherence in Multi-Component LLM Agents
May 28, 2026Anany KotawalaMulti-Agent LLM Systems LLM Agent Reliability
Princeton University, Princeton, NJ, USA
How Reliable Are AI Attackers Against a Fixed Vulnerable Target? A 400-Run Empirical Study of LLM Penetration Testing Consistency
May 28, 2026Galip Tolga ErdemAutomated Penetration Testing LLMs for Cybersecurity
Independent Researcher
Scaling Laws for Agent Harnesses via Effective Feedback Compute
May 28, 2026Xuanliang Zhang, Dingzirui Wang, Keyan Xu +2Inference-Time Scaling Agent Harness Optimization
Harbin Institute of Technology
GRASP: Gated Regression-Aware Skill Proposer for Self-Improving LLM Agents
May 28, 2026Johannes Moll, Jean-Philippe Corbeil, Jiazhen Pan +4LLM Agent Self-Improvement LLM Agent Skill Learning
1Technical University of Munich and TUM University Hospital · 2Microsoft Healthcare & Life Sciences
Honest Lying: Understanding Memory Confabulation in Reflexive Agents
May 28, 2026Prakhar Dixit, Sadia Kamal, Tim OatesAI Agent Reliability LLM Self-Refinement
Department of Computer Science, University of Maryland Baltimore County, Baltimore, MD, USA.
How Coding Agents Fail Their Users: A Large-Scale Analysis of Developer-Agent Misalignment in 20,574 Real-World Sessions
May 28, 2026Ningzhi Tang, Chaoran Chen, Gelei Xu +5Agent Failure Analysis Coding Agents
University of Notre Dame · 2Vanderbilt University · 3Google
PatchBoard: Schema-Grounded State Mutation for Reliable and Auditable LLM Multi-Agent Collaboration
May 28, 2026Shuyu Zhang, Yaqi Shi, Jiarui Zhang +2AI Agent Reliability Multi-Agent LLM Systems
School of Computer Science and Technology Xidian University Xi’an, China
First head-to-head comparison of agentic AI applied to the analysis of simulated data of the Einstein Telescope
May 27, 2026Gianluca IngugliaAI Agents for Scientific Discovery AI Agent Evaluation
Marietta Blau Institute for Particle Physics, Austrian Academy of Sciences, 1010 Vienna, Austria

Jun 4, 2026, cs.AIJ/K move · Enter open · S save

Critic-Guided Heterogeneous Multi-Agent Reasoning for Reliable Mathematical Problem Solving

Muhammad Talha Sharif, Abdul Rehman

dept. of Computer Science · National University of Computer and Emerging Sciences · Islamabad, Pakistan

LLM Self-Correction Multi-Agent Reasoning LLM Mathematical Reasoning LLM Agent Reliability

Open paper AskPDF arXiv

Recent Large Language Models (LLMs) have shown impressive reasoning abilities; but they are still susceptible to hallucinations, intermediate reasoning mistakes, and unreliable reasoning results in complex mathematical reasoning problems. In this study, we introduce a critic-based heterogeneous multi-agent approach to improve the dependability of mathematical reasoning. This framework incorporates several LLM agents of different specialties and employs a critic-driven adaptive learning system to assess and guide the reasoning process based on intermediate feedback. The system adopts a generator-validator framework, with the validator not only determining correctness but also offering critiques to guide regeneration of solutions. This allows for adaptive error correction and prevents error cascading. Our experiments on the GSM8K benchmark show that the proposed method achieves up to 13% accuracy improvement over single-shot and non-critic models. Additionally, findings suggest that heterogeneity and critique reduce the need for large models, allowing smaller models to perform on par. Ablation studies reveal the main performance gains are due to the critic-based feedback loop and not model size. In summary, the proposed approach showcases the benefits of combining heterogeneous multi-agent collaboration and critique to obtain reliable and interpretable reasoning systems.

Momentum

Related topics

Latest papers 558