cs.MANov 4, 2026

More Agents, More Reliable? System-Level Reliability of Multi-Agent LLMs under Operational-Profile Uncertainty

Authors: Burkay Tunçtürk, Koorosh Aslansefat, Yiannis Papadopoulos, Bao Nguyen

Organizations: University of Hull, Hull, HU6 7RX, UK · Bilkent University, Ankara, 06800, Türkiye · University of Leeds, Leeds, LS2 9JT, UK

Abstract

Teams increasingly build software from cooperating large language model (LLM) agents, often assuming more agents mean more reliability. Yet the usual evidence, average benchmark accuracy, ignores the input mix a deployed system meets and the rule merging the agents' answers. Building on HIP-LLM, a hierarchical imprecise-probability assessment of single LLMs, we propose a framework combining it with a Bayesian network of the system's failure logic for multi-agent systems. Agent failure-probability intervals propagate through this network, where the combining rule can itself fail, into a reliability envelope showing which evidence limits it. In a pipeline extracting regulated chemical-hazard statements, majority voting over three models protected against the weakest but did not clearly exceed the best, and a voter failing on 5% of demands would largely erase its advantage over the middle model. More agents yield a more defensible reliability claim, not automatically a more reliable system.

Figures & tables

Explore similar work

CardsList
  1. AgentCollabBench: Diagnosing When Good Agents Make Bad Collaborators

    May 9, 2026Aritra Mazumder, Shubhashis Roy Dipta, Nusrat Jahan Lia +10AI Agent ReliabilityMulti-Agent Collaboration

  2. Who Broke the System? Failure Localization in LLM-Based Multi-Agent Systems

    Jul 8, 2026Yufei Xia, Anjun Gao, Yueyang Quan +2Agent Failure AnalysisMulti-Agent LLM Systems

  3. Understanding Communication Failures in Multi-Agent LLM Systems

    Oct 23, 2026Priya Pitre, Jun Hou, Yi Fang +2Multi-Agent LLM SystemsMulti-Agent Communication