CoE: Collaborative Entropy for Uncertainty Quantification in Agentic Multi-LLM Systems

Kangkang Sun; Jun Wu; Jianhua Li; Minyi Guo; Xiuzhen Che; Jianwei Huang

cs.AIMar 30, 2026

CoE: Collaborative Entropy for Uncertainty Quantification in Agentic Multi-LLM Systems

Authors: Kangkang Sun, Jun Wu, Jianhua Li, Minyi Guo, Xiuzhen Che, Jianwei Huang

Organizations: Shanghai Key Laboratory of Integrated Administration Technologies for Information Security School of Computer Science Shanghai Jiao Tong University Shanghai, China · School of Science and Engineering The Chinese University of Hong Kong, Shenzhen Shenzhen, China

Abstract

Uncertainty estimation in multi-LLM systems remains largely single-model-centric: existing methods quantify uncertainty within each model but do not adequately capture semantic disagreement across models. To address this gap, we propose Collaborative Entropy (CoE), a unified information-theoretic metric for semantic uncertainty in multi-LLM collaboration. CoE is defined on a shared semantic cluster space and combines two components: intra-model semantic entropy and inter-model divergence to the ensemble mean. CoE is not a weighted ensemble predictor; it is a system-level uncertainty measure that characterizes collaborative confidence and disagreement. We analyze several core properties of CoE, including non-negativity, zero-value certainty under perfect semantic consensus, and the behavior of CoE when individual models collapse to delta distributions. These results clarify when reducing per-model uncertainty is sufficient and when residual inter-model disagreement remains. We also present a simple CoE-guided, training-free post-hoc coordination heuristic as a practical application of the metric. Experiments on TriviaQA and SQuAD with LLaMA-3.1-8B-Instruct, Qwen-2.5-7B-Instruct, and Mistral-7B-Instruct show that CoE provides stronger uncertainty estimation than standard entropy- and divergence-based baselines, with gains becoming larger as additional heterogeneous models are introduced. Overall, CoE offers a useful uncertainty-aware perspective on multi-LLM collaboration.

Figures & tables

Figure 1: Overview of the proposed Collaborative Entropy (CoE) framework for agentic multi-LLM uncertainty quantification. Each LLM generates candidate responses that are clustered semantically to form probability distributions, which are then aggregated to minimize system-level collaborative entropy.

Figure 2: Visualization of the four CoE quadrants using Dirichlet distributions over three semantic clusters. Each simplex (colour intensity = probability density) and accompanying bar chart illustrates a characteristic combination of $\mathcal{U}_{\mathcal{A}}$ (horizontal axis) and $\mathcal{U}_{\mathcal{E}}$ (vertical axis).

Uncertainty Metric	Two LLMs (Llama + Qwen)						Three LLMs (Llama + Qwen + Mistral)
	Rejection Accuracy				AURAC	AUROC	Rejection Accuracy				AURAC	AUROC
	80%	90%	95%	100%	AURAC	AUROC	80%	90%	95%	100%	AURAC	AUROC
Semantic Entropy	0.625	0.583	0.579	0.575	0.176	0.670	0.394	0.416	0.468	0.375	0.123	0.687
Regular Entropy	0.562	0.583	0.605	0.575	0.175	0.475	0.342	0.333	0.343	0.375	0.103	0.416
$P_{\text{false}}$	0.575	0.575	0.575	0.575	0.172	0.586	0.375	0.405	0.405	0.375	0.117	0.540
$\mathcal{U}_{\mathcal{E}}$	0.625	0.583	0.589	0.575	0.177	0.661	0.368	0.388	0.437	0.375	0.116	0.716

Table 1: Selective prediction performance of uncertainty metrics in multi-LLM collaborative settings on TriviaQA (200 samples). For Semantic Entropy, Regular Entropy, and

P_{\text{false}}

, scores are reported as the arithmetic mean over constituent models.

\mathcal{U}_{\mathcal{A}}

and

\mathcal{U}_{\mathcal{E}}

are ablation baselines isolating each component of CoE. Bold denotes the best AUROC in each configuration.

Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

LLMs	TriviaQA Dataset		SQuAD Dataset
LLMs	Train Accuracy	Validation Accuracy	Train Accuracy	Validation Accuracy
Llama-3.1-8b-instruct	0.65	0.675	0.225	0.1
Qwen-2.5-7b-instruct	0.6	0.55	0.2	0.15
Mistral-7b-instruct	0.375	0.375	0.05	0.075

Appendix

Table 2: The performance of different single LLMs on different dataset (200 samples)

Panel of Figure 3 — Figure 3: Performance comparison of CoE with different divergence metrics on TriviaQA dataset (200 samples, KL divergence).

Panel of Figure 4 — Figure 4: Performance comparison of CoE with different divergence metrics on SQuAD dataset (200 samples, KL divergence).

Figure 5: Robustness of the CoE-reduction algorithm to the convergence threshold $\epsilon$ (Llama + Qwen ensemble on TriviaQA , 200 shots).

Panel of Figure 6 — Figure 6: Analysis of CoE-Reduction optimization algorithm under different sampling temperatures and generation counts (Llama + Qwen on TriviaQA dataset with 200 in-context examples).

Figure 7: Accuracy gain and residual CoE after applying the CoE-guided coordination heuristic under four divergence measures (Llama + Qwen on TriviaQA, 200 samples). Initial CoE (light bars) and optimised CoE (dark bars) are shown on the left axis; ensemble accuracy gain is annotated on each bar.

LLM Uncertainty Estimation

Large language model uncertainty estimation measures when outputs may be wrong, ambiguous, or unreliable to guide safer decisions.

See all 398 papers

LLMs Uncertainty Quantification via Adaptive Conformal Semantic Entropy
May 5, 2026Hamed Karimi, Vaishali Meyappan, Reza SamaviSelective Prediction Semantic Entropy
Toronto Metropolitan University, Toronto, Ontario, Canada · Vector Institute, Toronto, Ontario, Canada
Semantic Uncertainty-Guided Orchestration in Hierarchical Multi-Agent Systems
Aug 11, 2026John Knowlton, Aritra Guha, Risto MiikkulainenSemantic Uncertainty Estimation Multi-Agent Orchestration
The University of Texas at Austin, USA · AT&T Chief Data Office, USA · Cognizant AI Lab, USA
CoMet: Context and Multiplicity Decomposition for Multimodal Uncertainty Estimation
Jun 30, 2026Sanghyuk Chun, William Yang, Amaya Dharmasiri +1Uncertainty Quantification Multimodal Large Language Models
Princeton University

May 5, 2026J/K move · Enter open · S save

LLMs Uncertainty Quantification via Adaptive Conformal Semantic Entropy

Hamed Karimi, Vaishali Meyappan, Reza Samavi

Toronto Metropolitan University, Toronto, Ontario, Canada · Vector Institute, Toronto, Ontario, Canada

Selective Prediction Semantic Entropy LLM Uncertainty Estimation Uncertainty Calibration Semantic Uncertainty Estimation

Open paper AskPDF arXiv

LLMs' overconfidence, particularly when hallucinating, poses a significant challenge for the deployment of the models in safety-critical settings and makes a reliable estimation of uncertainty necessary. Existing approaches for uncertainty quantification typically prioritize lexical or probabilistic measures; however, these techniques often ignore the semantic variance of different responses with similar meaning. In this paper, we propose Adaptive Conformal Semantic Entropy (ACSE), a method for estimating prompt-level uncertainty by adaptively measuring semantic dispersion in LLMs outputs. Our uncertainty scoring function is based on clustering semantic entropy of multiple diverse responses to the same prompt. The function adaptively adjusts the uncertainty score based on semantic features of each cluster. To ensure statistical reliability of our score, we use conformal calibration to apply a decision rule to accept/abstain the prompts, providing a finite-sample, distribution-free guarantee such that the error rate among the accepted responses remains bounded by a user-specified tolerance. Our extensive experimental evaluations using different LLMs and datasets, demonstrate that our approach consistently outperforms state-of-the-art uncertainty quantification baselines using discriminative performance, conformal guarantees, and probabilistic calibration indicators. As a highlight, for TriviaQA dataset, AUROC of our approach is 0.88 compared to 0.65 produced by the token entropy approach.

Abstract

LLM Uncertainty Estimation

Explore similar work

LLMs Uncertainty Quantification via Adaptive Conformal Semantic Entropy

Semantic Uncertainty-Guided Orchestration in Hierarchical Multi-Agent Systems

CoMet: Context and Multiplicity Decomposition for Multimodal Uncertainty Estimation