LLM Agent Reliability

LLM: Large Language Model

Follow this topic Ask about this topic

Momentum

90 papers in the last four weeks, up 109% on the four weeks before. 0.9% of all new papers.

Jul 13Week of Sep 28

Latest papers 558

All topics

Beyond State Machines: Executing Network Procedures with Agentic Tool-Calling Sequences
May 4, 2026Purna Sai Garigipati, Onur Ayan, Kishor Chandra Joshi +1LLM Agent Orchestration Tool-Using Agents
Heisenberg Research Center, Huawei Technologies Duesseldorf GmbH, 80992 Munich, Germany · Eindhoven University of Technology, Eindhoven, The Netherlands
DataClawBench: An Agent Benchmark for Exploratory Real-World Financial Data Analysis
May 4, 2026Qiaohong Zhang, Weihao Ye, Jialong Chen +7LLM Agent Reliability Agent Benchmarks
School of Computer Science and Engineering, Sun Yat-sen University · School of Software Engineering, Sun Yat-sen University · Lingnan College, Sun Yat-sen University
AutoResearch: An Execution-Grounded Multi-Agent Framework for Reliable Research Workflow Automation
May 4, 2026Rajesh Kumar, Waqar Ali, Junaid Ahmed +2Execution-Guided Code Generation Scientific Workflow Automation
International Research Center for Complexity Sciences, Hangzhou International Innovation Institute, Beihang University, Hangzhou 311115, China · Department of Computer Science, College of Science, Mathematics and Technology, Wenzhou-Kean University, Wenzhou 325060, China · Computer Systems Engineering Department, Sukkur IBA University, Sindh, Pakistan +1
EngiAgent: Fully Connected Coordination of LLM Agents for Solving Open-ended Engineering Problems with Feasible Solutions
May 4, 2026Xiyuan Zhou, Ruixi Zou, Xinlei Wang +4Multi-Agent LLM Systems Software Engineering Agents
Nanyang Technological University · The Chinese University of Hong Kong, Shenzhen · INSAIT, Sofia University “St. Kliment Ohridski” +2
NeuroState-Bench: A Human-Calibrated Benchmark for Commitment Integrity in LLM Agent Profiles
May 3, 2026Xiao JiaAI Agent Reliability LLM Agent Evaluation
School of Artificial Intelligence The Chinese University of Hong Kong, Shenzhen · The Chinese University of Hong Kong, Shenzhen
Evaluating Agentic AI in the Wild: Failure Modes, Drift Patterns, and a Production Evaluation Framework
May 2, 2026Mukund PandeyAI Agent Reliability AI Agent Evaluation
Independent Researcher
Practical Limits of Autonomous Test Repair: A Multi-Agent Case Study with LLM-Driven Discovery and Self-Correction
May 2, 2026Hyukjoo LeeAutomated Software Testing Software Engineering Agents
Independent Researcher Republic of Korea
BaRA: Budget-constrained and Reliable Web Data Collection Agent
May 2, 2026Soojeong Lee, Joseph Lee, Yongseong Cho +3Multimodal Agents Web Agents
Department of Statistics and Data Science, Yonsei University, Seoul, Republic of Korea · Electronics and Telecommunications Research Institute (ETRI), Daejeon, Republic of Korea · HUSTLERS Corp., Seoul, Republic of Korea
Towards Multi-Agent Autonomous Reasoning in Hydrodynamics
May 1, 2026Jinpai Zhao, Albert Cerrone, Joannes Westerink +1Multi-Agent Orchestration AI Agents for Scientific Discovery
CEEES University of Notre Dame Notre Dame, Indiana, USA
Are Tools All We Need? Unveiling the Tool-Use Tax in LLM Agents
Apr 30, 2026Kaituo Zhang, Zhen Xiong, Mingyu Zhong +4Tool-Augmented Language Model Agents Agentic Reasoning
University of Houston · University of Southern California · 3New York University +2
Crab: A Semantics-Aware Checkpoint/Restore Runtime for Agent Sandboxes
Apr 30, 2026Tianyuan Wu, Chaokun Chang, Lunxi Cao +2Agent Reliability LLM Agent Reliability
HKUST Hong Kong SAR, China
Collaborative Agent Reasoning Engineering (CARE): A Three-Party Design Methodology for Systematically Engineering AI Agents with Subject Matter Experts, Developers, and Helper Agents
Apr 30, 2026Rahul Ramachandran, Nidhi Jha, Muthukumaran RamasubramanianLLM Agents Human-AI Collaboration
Marshall Space Flight Center, Huntsville, Alabama · The University of Alabama in Huntsville, Huntsville, Alabama
Modeling Clinical Concern Trajectories in Language Model Agents
Apr 30, 2026Sukesh Subaharan, Venkatesan VS, Murugadasan P +3LLM Interpretability AI Agent Monitoring
Coimbatore Medical College · Coimbatore, TN · India
Trace-Level Analysis of Information Contamination in Multi-Agent Systems
Apr 30, 2026Anna Mazhar, Huzaifa Suri, Sainyam GalhotraAgentic Workflows AI Agent Monitoring
Cornell University Ithaca, NY, USA · University of Illinois Urbana-Champaign, IL, USA
The Inverse-Wisdom Law: Architectural Tribalism and the Consensus Paradox in Agentic Swarms
Apr 30, 2026Dahlia Shehata, Ming LiMulti-Agent LLM Systems Multi-Agent Coordination
University of Waterloo · Canada
Reinforced Agent: Inference-Time Feedback for Tool-Calling Agents
Apr 29, 2026Anh Ta, Junjie Zhu, Shahin ShayandehMulti-Agent LLM Systems Tool-Use Evaluation
Apple
TDD Governance for Multi-Agent Code Generation via Prompt Engineering
Apr 29, 2026Tarlan Hasanli, Shahbaz Siddeeq, Bishwash Khanal +3LLM Prompting AI Agent Governance
Tampere University Tampere, Finland
Autonomous LLM Agents & CTFs: A Second Look
Apr 29, 2026Youness Bouchari, Matteo Boffa, Marco Mellia +3LLM Agent Orchestration AI Agent Security Benchmarks
Politecnico di Torino · Universit`a di Torino · Huawei Technologies France
Operating-Layer Controls for Onchain Language-Model Agents Under Real Capital
Apr 28, 2026T. J. Barton, Chris Constantakis, Patti Hauseman +4AI Agent Reliability LLM Agent Evaluation
DX Research Group (DXRG)
SAFEdit: Does Multi-Agent Decomposition Resolve the Reliability Challenges of Instructed Code Editing?
Apr 28, 2026Noam Tarshish, Nofar Selouk, Daniel Hodisan +4Execution-Guided Code Generation Multi-Agent Coding
Ben-Gurion University of the Negev · Beer Sheva, Israel
Plausible but Wrong: A case study on Agentic Failures in Astrophysical Workflows
Apr 28, 2026Shivam Rawat, Lucie FlekAgent Failure Analysis AI for Science
Bonn-Aachen International Center for Information Technology, University of Bonn, Germany; Lamarr Institute for Machine Learning and Artificial Intelligence, Germany.
FAMA: Failure-Aware Meta-Agentic Framework for Open-Source LLMs in Interactive Tool Use Environments
Apr 28, 2026Amir Saeidi, Venkatesh Mishra, Souradeep Mukhopadhyay +4AI Agent Reliability Multi-Agent LLM Systems
Arizona State University · Cisco Research
FastOMOP: A Foundational Architecture for Reliable Agentic Real-World Evidence Generation on OMOP CDM data
Apr 27, 2026Niko Moeller-Grell, Shihao Shenzhang, Zhangshu Joshua Jiang +2Multi-Agent Orchestration Healthcare
EPSRC DRIVE-Health, Department of Biostatistics & Health Informatics, King’s College London, UK. · Lancashire Teaching Hospitals NHS Foundation Trust, UK. · Institute for Health Informatics, University College London, UK. +5
Beyond the Attention Stability Boundary: Agentic Self-Synthesizing Reasoning Protocols
Apr 27, 2026Dahlia Shehata, Ming LiInstruction Following Attention Mechanisms
University of Waterloo · Canada
From Prototype to Classroom: An Intelligent Tutoring System for Quantum Education
Apr 27, 2026Iizalaarab Elhaimeur, Nikos ChrisochoidesMulti-Agent LLM Systems Intelligent Tutoring Systems
Center for Real-Time Computing Computer Science Department Old Dominion University Norfolk, VA · Center for Real-Time Computing Computer Science and Physics Departments Old Dominion University Norfolk, VA
Analytica: Soft Propositional Reasoning for Robust and Scalable LLM-Driven Analysis
Apr 24, 2026Junyan Cheng, Kyle Richardson, Peter ChinLLM Agent Orchestration LLM Agents
Dartmouth Collegeα · Allen Institute for AIβ
Reliable Self-Harm Risk Screening via Adaptive Multi-Agent LLM Systems
Apr 24, 2026Meghana Karnam, Ananya JoshiAI Agent Reliability Multi-Agent LLM Systems
Johns Hopkins University Baltimore, USA
Sound Agentic Science Requires Adversarial Experiments
Apr 23, 2026Dionizije Fa, Marko CuljakAI-Assisted Scientific Research LLM Agent Evaluation
Entropic
How Consistent Are LLM Agents? Measuring Behavioral Reproducibility in Multi-Step Tool-Calling Pipelines
Apr 23, 2026Abel YagubyanAI Agent Reliability LLM Agent Evaluation
Independent Researcher
Efficient Agent Evaluation via Diversity-Guided User Simulation
Apr 23, 2026Itay Nakash, George Kour, Ateret Anaby-TavorAutomated Software Testing LLM Agent Evaluation
IBM Research

May 4, 2026, cs.NIJ/K move · Enter open · S save

Beyond State Machines: Executing Network Procedures with Agentic Tool-Calling Sequences

Purna Sai Garigipati, Onur Ayan, Kishor Chandra Joshi, Xueli An

Heisenberg Research Center, Huawei Technologies Duesseldorf GmbH, 80992 Munich, Germany · Eindhoven University of Technology, Eindhoven, The Netherlands

LLM Agent Orchestration Tool-Using Agents Long-Horizon LLM Agents LLM Agent Reliability

Open paper AskPDF arXiv

Agentic AI will be an essential enabling technology for designing future mobile communication systems, which could provide flexible and customized services, automate complex network operations, and drive autonomous decision-making across the network. This work studies how Large Language Model (LLM)-based network AI agents can be utilized to execute network procedures expressed as sequences of tool invocations. We investigate four approaches, which differ in how the agent obtains the procedure and in how execution is distributed between the agent and the underlying tools. We evaluated the latency and execution correctness across these approaches using a User Equipment (UE) IP allocation procedure as a case study. Furthermore, we conduct a stress test to examine how many sequential procedural steps an LLM agent can reliably execute before failure. Our results show that approaches relying on iterative agent-side reasoning incur higher latency and are more prone to execution errors, while approaches where the procedure is encapsulated within a single tool, which internally orchestrates the required steps by invoking other tools, reduce latency by limiting repeated reasoning. The stress-test results further show that the model with advanced tool-calling capability maintains reliable execution over longer procedures than the other evaluated models; however, all models exhibit reliability degradation as procedure length increases, revealing clear execution limits in multi-step tool-based workflows. To systematically analyze failures in procedure execution, we introduce a procedure-specific error taxonomy that categorizes deviations in multi-step procedural execution.

Momentum

Related topics

Latest papers 558