Multi-agent decision pipelines can outperform single agent workflows when complementarity holds, i.e., different agents bring unique information to the table to inform a final decision. We propose ComplLLM, a post-training framework based on decision theory that fine-tunes a decision-assistant LLM using complementary information as reward to output signals that complement existing agent decisions. We validate ComplLLM on synthetic and real-world tasks involving domain experts, demonstrating how the approach recovers known complementary information and produces plausible explanations of complementary signals to support downstream decision-makers.
Figures & tables
Figure 1: A graphical representation of how our work goes beyond simply identifying concepts that appear in text or signals that are predictive of a target state, by also ensuring that signals complement the existing recommendation. The diagonal stripes (twill pattern) represent the target of the corresponding methods.
Figure 2: The “two agents” setting in our framework. The recommending agent makes a recommendation Z based on their own features X , and the supervisor agent aggregates their own information T with Z to make a final decision D .
Notation
Radiology Diagnosis instantiation
Y
Payoff state ∈{0,1} for the presence of cardiac dysfunction.
Z
Prediction from the vision model on the probability of the presence of cardiac dysfunction ∈[0,1] .
X
Features of the vision model (X-ray image).
T
Features of the clinician (radiology report and patient’s medical history).
D
Final decision by the clinician ∈{0,1} (e.g., whether to order a biopsy).
Signals in T
Table 1: Notation summary (instantiated with the example of radiology-diagnosis).
Method
Surf.
F1
CompLLM
0.98
0.67
Zero-shot
0.91
0.42
Few-shot
0.95
0.38
BERTopic
0.84
0.17
HypotheSAE
0.90
0.38
Table 2: Results comparing average surface similarity and F1 score of the extracted signals by each method on the synthetic dataset.
Source
Total Acc.
# Sig
Significant Signals
Δ Acc.
Agent Decision
0.819
-
-
-
ComplLLM
0.839
12
negative pneumothorax
+0.0048
positive pleural effusion
+0.0040
negative pleural effusion
+0.0032
positive pulmonary congestion
+0.0024
positive cardiac silhouette enlargement
+0.0016
Table 3: Signals on MIMIC-CXR dataset that are significant with p<0.05 after Bonferroni correction. ‘ Δ Acc.’ represents the marginal accuracy improvement each signal sj marginally provides over the other signals and the agent recommendation. ‘Total Acc.’ shows the combined accuracy when all significant signals are included, i.e., sum of the signals’ marginal gain. ‘# Sig’ indicates the number of statistically significant signals discovered by each method.
Figure 3: Expected accuracy given the extracted signals and the agent’s recommendation by each method. Dashed lines represent agent decision accuracy and the accuracy of the benchmark method. Error bars depict bootstrapped 95% confidence intervals (N=5000).
Figure 4: Accuracy of LLM paper review decisions on the Review5K dataset. Error bars depict bootstrapped 95% confidence intervals (N=5000).
mentions presence of pulmonary edema mentions presence of interstitial edema mentions presence of interstitial lung disease or interstitial findings mentions no acute cardiopulmonary process mentions presence of pleural effusion and atelectasis mentions presence of pleural effusion mentions presence of bilateral pleural effusion mentions presence of interstitial pulmonary edema
BERTopic
15
Outlier/Noise Chest X-ray Findings Cardiac Enlargement Pancreatic Cancer and Chest Imaging Chest X-ray for TIA evaluation Chest X-ray for Upper GI Bleed Pulmonary Edema Monitoring Pulmonary Edema Evaluation Chest X-ray findings in edema Chest X-ray findings …
Table 4: Summary of Extracted Signals by Method on Synthetic Dataset.
Source
Total Acc.
# Sig
Significant Signals
Δ Acc.
Baseline Agent Decision Accuracy: 0.847
ComplLLM
0.878
4
promotes or condones violence
+0.0222
misinformation conspiracy or false theory
+0.0061
offers safe alternative
+0.0020
oblique or profane language
+0.0010
BERTopic
0.856
2
Outlier/Noise
+0.0051
Table 5: Signals on DICES dataset. Significant signals with p<0.05 after Bonferroni correction. ‘ Δ Acc.’ represents the incremental accuracy improvement each signal provides over the baseline agent decision (0.847). ‘Total Acc.’ shows the combined accuracy when all significant signals are included. ‘# Sig’ indicates the number of statistically significant signals discovered by each method.
Source
Total Acc.
# Sig
Significant Signals
Δ Acc.
Agent Decision
0.627
-
-
-
ComplLLM
0.692
8
clarity
+0.0310
novelty
+0.0179
experimental validation
+0.0110
theoretical contribution
+0.0024
clarity of presentation
+0.0021
Table 6: Signals on Review5k dataset. Significant signals with p<0.05 after Bonferroni correction. ‘ Δ Acc.’ represents the incremental accuracy improvement each signal provides over the baseline agent decision (0.627). ‘Total Acc.’ shows the combined accuracy when all significant signals are included. ‘# Sig’ indicates the number of statistically significant signals discovered by each method.
Appendix figures & tables10 assets
Supplementary material from the paper’s appendix.
Appendix
Biomarker
Age
Gender
Threshold
NT-proBNP
≤ 49
–
> 449
50–75
–
> 899
≥ 76
–
> 1799
Troponin
< 64
Female
≥ 0.014
≥ 65
Female
≥ 0.018
< 50
Male
≥ 0.019
Appendix
Table 7: Biomarker Thresholds by Age and Gender
Condition
Positive
Negative
Uncertain
Atelectasis
0.3384
0.3633
− 0.1758
Cardiomegaly
0.7462
− 0.3945
0.3062
Consolidation
0.9686
− 0.0945
0.6488
Enlarged Cardiomediastinum
0.2368
− 0.9585
0.1678
Fracture
0.3986
− 0.7871
0.2674
Lung Lesion
0.3194
0.1314
0.0970
Appendix
Table 8: Coefficients of the logistic regression model on the synthetic ground-truth labels. * indicates the labels are withheld from the model to generate the recommending agent’s decision.
Figure 5: Training curves of SFT and GRPO on the MIMIC-CXR dataset.
Figure 6: Training curves of SFT and GRPO on the DICES dataset.
Figure 7: Training curves of SFT and GRPO on the Review5K dataset.
Figure 8: Training curves of SFT and GRPO on the synthetic dataset.
Figure 9: Experimental results for ComplLLM with only SFT. Dashed lines represent agent decision accuracy and the accuracy of the benchmark method. Error bars depict bootstrapped 95% confidence intervals (N=5000).
Figure 10: An example of the radiology report and X-ray image shown to the physicians in the qualitative study.
Figure 11: An example of ComplLLM ’s output shown to the physicians in the qualitative study.
Figure 12: An example of ComplLLM ’s output and the updated probablistic prediction shown to the physicians in the qualitative study.
LLM decision-making studies how language models choose, act, seek information, and allocate resources under goals, uncertainty, constraints, and social pressures.
The rapid development of large language models (LLMs) has motivated research on decision-making in multi-agent systems, where multiple agents collaborate to achieve shared objectives. Existing aggregation approaches, such as voting and debate, are largely ad-hoc and lack formal guarantees regarding the informativeness of the resulting decisions. In this paper, we provide a principled approach to analyse decisions made in the multi-LLM setting using Blackwell's informativeness framework. Within the Blackwell information-structure abstraction, we show that voting and debate induce information structures that are no more informative than the pooled private information of all agents. This result identifies Bayesian pooled posterior maximisation as an information-theoretic upper-bound decision rule under the Blackwell ordering. Motivated by this theoretical analysis, we introduce a practical method for LLM-based question-answering (QA) tasks that estimates each agent's posterior and approximates the pooled posterior using a product-of-posteriors estimator. Extensive experiments on six QA benchmarks demonstrate that our approach outperforms state-of-the-art multi-LLM debate and voting methods.
Zheng Zhang, Cuong C. Nguyen, Kevin Wells +1
Centre for Vision, Speech and Signal Processing, University of Surrey
Effective collaboration among large language model (LLM) agents in multi-agent systems requires sharing relevant evidence, requesting missing information, and incorporating teammates' contributions into decisions. Training these behaviors is challenging because final task outcomes provide limited feedback on the value of individual messages. We introduce CoSIR (Collaboration via Surprisal-based Information Rewards), a post-training approach that uses a simple, verifiable environment to teach collaboration behaviors that generalize to more complex, unseen tasks. We propose a Bayesian evaluator that scores shared information by its reduction in team surprisal, defined as the sum of the negative log probabilities assigned to the correct option given each agent's available evidence. Requests are scored by the potential reduction in the same quantity if the requested information were truthfully provided. Separate rewards encourage agents to use the received information in their private choices and calculations. We post-train Qwen3-4B, Qwen3-8B, and Gemma-3-4B-it through self-play and evaluate transfer to seven unseen natural-language collaboration tasks. CoSIR achieves the highest aggregate score, while its performance in general tasks remains competitive. Behavioral analyses show increased disclosure of private evidence and fewer judged failures to use teammates' input.
Keru Chen, Sen Lin, Yingbin Liang +2
School of ECEE, Arizona State University · Computer Science Department, University of Houston · Department of ECE, The Ohio State University +1
Large language models (LLMs) are increasingly used to support decisions under incomplete information, where deciding what to look up and when to stop matters as much as the final answer. However, LLMs fall into an information trap: They pay for checks, i.e., costly pieces of evidence, that cannot change their decision or are not worth their cost, and they keep asking when the evidence is already sufficient. To this end, we propose Decision-Informed Value-of-information Acquisition (DIVA), a simple yet effective procedure for LLM information seeking based on the decision value of evidence. DIVA 1) lets the LLM specify a small decision model, i.e., how likely a bad outcome is and how each check would shift this probability, through short numeric questions and 2) values each candidate check by how much its imagined outcomes improve the decision net of its cost, stopping when no check is worth its cost. Its variant, DIVA-U, further accounts for the LLM's uncertain estimates of how strongly each check shifts this probability. DIVA is training-free and applies to any LLM. Across four domains and 20 LLMs, DIVA-U has lower regret, i.e., value lost relative to the best possible decision, than every LLM baseline on real-world data, mainly by stopping once evidence suffices, and DIVA cuts the regret of the best LLM baseline by 75% in synthetic scenarios where every check's reliability is given. Code: github.com/seunghan96/diva.