Learning Calibrated Confidence for Multi-Agent Collaboration

Chenxiao Fan; Chongming Gao; Gangyi Zhang; Yaxin Gong; Leyang Shen; Jiakai Wang; Dong Wang; Yang Liu

cs.LGOct 29, 2026

Learning Calibrated Confidence for Multi-Agent Collaboration

Authors: Chenxiao Fan, Chongming Gao, Gangyi Zhang, Yaxin Gong, Leyang Shen, Jiakai Wang, Dong Wang, Yang Liu,

Organizations: University of Science and Technology of China · Qwen Business Unit of Alibaba · National University of Singapore

Abstract

Complex information seeking on the Web often requires collecting evidence from multiple documents and reasoning across them, as in multi-hop question answering. Multi-agent systems address such tasks by dividing work among specialized agents, where upstream agents produce intermediate answers and supporting evidence that downstream agents build upon. However, upstream answers vary in correctness, and most systems provide no quantitative signal about it, leaving the downstream agent to judge whether to trust or verify what it receives based solely on the textual evidence. In this paper, we propose MASCAL. Each agent learns to produce a correctness probability for its answer by jointly optimizing task performance and confidence calibration in reinforcement learning; the upstream agent passes this probability at handoff, giving the downstream agent a quantitative basis for its decisions, and the final answer also carries a calibrated probability, quantifying the reliability of the system output. To guide the downstream agent in using this signal during training, we introduce confidence-aware weighting, which increases the learning weight at positions responsive to the upstream probability, so that the agent learns more from decisions where it actively uses the signal. Across three collaboration architectures spanning Web retrieval and reasoning tasks, MASCAL improves task accuracy and calibration quality for both agents, and generalizes to deeper collaboration chains and different model scales. Intervention analysis confirms that upstream confidence affects downstream decision behavior, and the learned final confidence supports selective answering and targeted rechecking.

Figures & tables

Figure 1. Confidence at the agent handoff. Without a confidence signal, the downstream agent cannot distinguish a reliable report from an unreliable one. MASCAL attaches calibrated confidence to each report and to the final answer, guiding downstream decisions and enabling reliability assessment of the system output.

Figure 2. The MASCAL training pipeline. (a) Each rollout is split into a task segment and a confidence segment, trained with separate policy gradients and a BCE loss on the readout $c$ . (b) Masking the upstream confidence reveals which downstream tokens depend on it; those positions are up-weighted during task learning.

Figure 3. Collaboration architectures in our experiments.

Task	Main dataset	Train / Val. / Test	Additional tests ( $n$ )
Multi-hop QA	MuSiQue	12,938 / 719 / 1,252	HotpotQA	1,000
			2Wiki	1,000
Logic	LogiQA2	9,000 / 500 / 1,572	ReClor	500
			AR-LSAT	230
Math	DeepScaleR	9,000 / 500 / 500	MATH500	500
			AMC & AIME	181

Table 1. Dataset splits and question counts.

	Sequential handoff				Report fusion				Answer review (logic)				Answer review (math)
Method	MuSiQue	HotpotQA	2Wiki	Avg.	MuSiQue	HotpotQA	2Wiki	Avg.	LogiQA2	ReClor	AR-LSAT	Avg.	MATH500	AMC & AIME	DeepScaleR	Avg.
Base Single-Agent	24.65	42.67	40.90	36.07	24.65	42.67	40.90	36.07	69.94	80.30	35.65	61.96	83.20	37.43	60.60	60.41
Task-RL Single-Agent	47.26	52.27	51.87	50.47	47.26	52.27	51.87	50.47	76.91	85.65	52.07	71.54	91.30	51.31	75.70	72.77
Base MAS	30.35	43.93	50.27	41.52	27.69	46.77	51.37	41.94	68.73	78.10	34.84	60.56	82.85	37.50	60.35	60.23
Task-RL MAS	43.32	51.90	47.27	47.49	45.07	58.27	61.00	54.78	77.53	86.25	57.77	73.85	91.30	51.38	76.05	72.91
MAGRPO	44.12	53.43	49.07	48.87	43.93	57.77	67.20	56.30	73.58	82.95	49.73	68.75	88.80	48.96	73.40	70.39

Table 2. Task accuracy (%) comparing MASCAL with single-agent , MAS without confidence , and confidence-based MAS baselines. Best results are bolded .

	MuSiQue		Logic		Math
Method	Brier	AUROC	Brier	AUROC	Brier	AUROC
Base MAS	0.1496	0.7666	0.3472	0.6316	0.3726	0.7973
Task-RL MAS	0.1576	0.7887	0.1952	0.7081	0.2513	0.8361
MASCAL-Verbal 1 1 1 MASCAL-Verbal replaces the logit-based readout with verbalized confidence in the generated text, as in calibration RL methods such as RLCR and DCPO ( Damani et al., 2026 ; Ma et al., 2026 ) .	0.1440	0.6527	0.2403	0.5013	0.1092	0.7482
MASCAL	0.0972	0.8663	0.1516	0.7500	0.0827	0.8951

Table 3. Upstream confidence quality. Scores are averaged over test sets for logic and math (Brier

\downarrow

, AUROC

\uparrow

	Multi-hop QA		Answer review
Confidence	Handoff	Fusion	Logic	Math
Original	59.00 / 0.1947	61.45 / 0.1940	76.59 / 0.1489	78.69 / 0.0816
Hidden	50.00 / 0.3041	54.98 / 0.3251	76.28 / 0.2237	77.42 / 0.1646
Random	49.32 / 0.3055	54.80 / 0.3580	73.86 / 0.2470	73.07 / 0.2621

Table 4. Accuracy (%) and Brier score under upstream confidence interventions on a trained MASCAL system.

Figure 4. MASCAL’s behavior across ten confidence bins: additional search calls and answer revision rates. Fusion uses mean upstream confidence.

	Multi-hop QA		Answer review
Method	Handoff	Fusion	Logic	Math
Base Single-Agent	0.3689 / 0.7262		0.3472 / 0.6316	0.3726 / 0.7973
Task-RL Single-Agent	0.4275 / 0.6873		0.1952 / 0.7081	0.2513 / 0.8361
Base MAS	0.4111 / 0.6528	0.3999 / 0.6671	0.3008 / 0.7183	0.3560 / 0.7655
Task-RL MAS	0.4498 / 0.6272	0.3965 / 0.5920	0.2266 / 0.6929	0.2608 / 0.8106
ConfMAD	0.3933 / 0.6957	0.3936 / 0.6937	0.3339 / 0.5616	0.4645 / 0.5778

Table 5. Final confidence quality for single-agent , MAS without confidence , confidence-based MAS , and MASCAL variants , averaged over test sets (Brier

\downarrow

, AUROC

\uparrow

Figure 5. Selective answering by final confidence. Accuracy is measured on the most confident answers at each coverage level, then averaged over test sets.

	Initial	Rechecking budget
Scenario	Acc. (%)	5%	10%	20%
Handoff	59.00	+0.15 / -0.04	+0.37 / -0.09	+0.74 / -0.17
Fusion	61.45	+0.24 / +0.02	+0.39 / +0.05	+0.66 / +0.09

Table 6. Targeted rechecking in MASCAL under a fixed budget:

\Delta

Acc. (pp) when the budget is spent on the least-confident

k

% of answers versus a random

k

	Multi-hop QA		Answer review
Variant	Handoff	Fusion	Logic	Math
MASCAL	59.00 / 0.1947	61.45 / 0.1940	76.59 / 0.1489	78.69 / 0.0816
w/o weighting	55.45 / 0.2212	57.26 / 0.1970	73.92 / 0.1517	78.32 / 0.0865
w/o calibration loss	52.37 / 0.3092	51.81 / 0.4419	74.47 / 0.2413	78.05 / 0.0929
w/o confidence reward	57.55 / 0.2102	54.46 / 0.2306	74.01 / 0.1538	78.08 / 0.0888
MASCAL-Verbal	49.97 / 0.3022	56.30 / 0.3200	73.86 / 0.2496	77.42 / 0.1170

Table 7. Training ablations across four scenarios. Each pair reports Acc. (%) / Brier

\downarrow

, averaged over test sets.

	3-hop			4-hop
Method	Acc	Brier $\downarrow$	AUROC $\uparrow$	Acc	Brier $\downarrow$	AUROC $\uparrow$
Base MAS	15.61	0.3771	0.6767	6.10	0.3149	0.6279
Task-RL MAS	32.39	0.5209	0.6653	17.21	0.6737	0.5791
MASCAL-Unweighted	34.51	0.1914	0.7764	19.65	0.1759	0.7439
MASCAL	37.85	0.1903	0.7701	20.19	0.1741	0.7555

Table 8. Accuracy (%) and calibration on 3-hop and 4-hop MuSiQue questions, using checkpoints trained only on 2-hop data. Best results are bolded .

	1.7B			4B			8B
Method	Acc	Brier $\downarrow$	AUROC $\uparrow$	Acc	Brier $\downarrow$	AUROC $\uparrow$	Acc	Brier $\downarrow$	AUROC $\uparrow$
Base MAS	19.75	0.5964	0.4985	17.60	0.5794	0.6245	41.52	0.4111	0.6528
Task-RL MAS	40.01	0.5537	0.5036	41.08	0.4367	0.6892	47.49	0.4498	0.6272
MASCAL-Unweighted	44.50	0.2721	0.7000	45.73	0.2225	0.7460	55.45	0.2212	0.7297
MASCAL	46.82	0.2205	0.7115	51.24	0.2190	0.7481	59.00	0.1947	0.8183

Table 9. Accuracy (%) and calibration on sequential handoff across model scales. Best results per scale are bolded .

Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

Setting	Value	Setting	Value
Base model	Qwen3-8B	Rollouts $G$	8
Optimizer	AdamW (ZeRO-2)	PPO clipping $\epsilon$	0.2
Learning rate	$2\times 10^{-6}$	Calibration warmup	30 steps
Epochs	3	Weighting strength $\lambda$	0.5
Inputs per step	128	Train sampling	$T{=}1$ , top- $p{=}1$

Appendix

Table 10. Training hyperparameters shared across scenarios.

	Multi-hop QA		Answer review
Method	Handoff	Fusion	Logic	Math
Base Single-Agent	36.07 / 0.3689		61.96 / 0.3472	60.41 / 0.3726
Task-RL Single-Agent	50.47 / 0.4275		71.54 / 0.1952	72.77 / 0.2513
Calib-RL Single-Agent	52.97 / 0.2119		72.53 / 0.1516	73.11 / 0.0847
MASCAL	59.00 / 0.1947	61.45 / 0.1940	76.59 / 0.1489	78.69 / 0.0816

Appendix

Table 11. Single-agent objectives compared with MASCAL. Each pair reports Acc. (%) / Brier

\downarrow

, averaged over test sets.

Components			Multi-hop QA		Answer review
Cal.	Trans.	Wt.	Handoff	Fusion	Logic	Math
			47.49 / 0.4498	54.78 / 0.3965	73.85 / 0.2266	72.91 / 0.2608
$\checkmark$			53.28 / 0.2919	56.23 / 0.3016	73.89 / 0.1650	76.01 / 0.0918
$\checkmark$	$\checkmark$		55.45 / 0.2212	57.26 / 0.1970	73.92 / 0.1517	78.32 / 0.0865
$\checkmark$	$\checkmark$	$\checkmark$	59.00 / 0.1947	61.45 / 0.1940	76.59 / 0.1489	78.69 / 0.0816

Appendix

Table 12. Component analysis of MASCAL. Each pair reports Acc. (%) / Brier

\downarrow

, averaged over test sets.

	Time (h)
Scenario	w/o weighting	MASCAL	Extra time
Sequential handoff	8.13	9.28	+14%
Report fusion	9.95	10.29	+3%
Answer review (logic)	9.61	10.45	+9%
Answer review (math)	28.69	31.16	+9%

Appendix

Table 13. Training-iteration time with and without confidence-aware weighting.

Multi-Agent LLM Systems

Multi-agent large language model systems coordinate multiple AI agents to solve tasks through communication, specialization, shared tools, and memory.

See all 1,050 papers

Confidence Composition for Multiagent Language Model Systems
Jun 11, 2026Ali Elahi, Michael J. Curry, Barbara Di EugenioMulti-Agent LLM Systems Probability Calibration
University of Illinois Chicago
Sequential Probabilistic Uncertainty Estimation for Parallel Multi-Agent Reasoning Systems
Oct 6, 2026Tunyu Zhang, Zihao Zhao, Yusong Zhao +5LLM Uncertainty Estimation Multi-Agent LLM Systems
Rutgers University · University of Illinois Urbana-Champaign · Vanderbilt University +1
MARGIN: Runtime Confidence Calibration for Multi-Agent Foundation Model Coordination
May 21, 2026Joss ArmstrongAI Agent Reliability Multi-Agent Coordination
Ericsson, Athlone, Ireland

Jun 11, 2026J/K move · Enter open · S save

Confidence Composition for Multiagent Language Model Systems

Ali Elahi, Michael J. Curry, Barbara Di Eugenio

University of Illinois Chicago

Multi-Agent LLM Systems Probability Calibration Multi-Agent Debate LLM Agent Reliability

Open paper AskPDF arXiv

Multiagent language model systems, such as collaborative reasoning and debate, produce multiple correlated candidate answers and confidence signals. However, these signals are usually calibrated only at the individual agent level, and provide no principled confidence estimate for the system's final answer. We formulate this as a confidence composition problem where combining confidence across agents and reasoning stages while preserving both selective utility and probabilistic reliability. We study confidence-aware routing and log-odds pooling protocols that select among candidate answers and output a system-level confidence. Across five benchmarks, 30 heterogeneous and homogeneous model pairs, and two confidence estimators, our gated-fusion methods improve AUARC and reduce Brier score over single agent, standard debate, and selective debate baselines, while retaining competitive weighted F1-score as a correctness metric. We further show that our log-odds fusion is overconfident due to correlated intermediate signals. We propose a shared dependence discount that substantially improves reliability while preserving predictions.

Abstract

Multi-Agent LLM Systems

Explore similar work

Confidence Composition for Multiagent Language Model Systems

Sequential Probabilistic Uncertainty Estimation for Parallel Multi-Agent Reasoning Systems

MARGIN: Runtime Confidence Calibration for Multi-Agent Foundation Model Coordination