Semantic Cooperative Games for Contribution Attribution in LLM-Based Multi-Agent Systems

Pengyi Jiang; Xiaoguang Zhu; Quanyan Zhu

cs.AIMay 14, 2026

Semantic Cooperative Games for Contribution Attribution in LLM-Based Multi-Agent Systems

Authors: Pengyi Jiang, Xiaoguang Zhu, Quanyan Zhu

Organizations: New York University · University of California, Davis

Abstract

Contribution attribution has become a central problem in LLM-based multi-agent systems, where final outputs are produced through multiple agents, message exchanges, and ordered workflow dependencies. Existing attribution methods often rely on counterfactual valuation, such as removing agents or comparing score changes across altered agent subsets. In language-mediated workflows, these methods require repeated model calls, introduce high variance, and do not explicitly capture the intermediate semantic states through which agents produce, preserve, and transform task-relevant information. We propose Semantic Cooperative Games (SCG), a framework that represents a realized language flow as a semantic generation hypergraph and induces an agent-level semantic value function on this structure. We define the Semantic Shapley Value (SSV) to allocate contribution over semantic support logic, and introduce SLIC, a single-trajectory algorithm that constructs the semantic hypergraph, recovers minimal semantic supports, applies Boolean absorption, and computes SSV without rerunning agent subsets. We prove that SSV reduces to the classical Shapley value under standard set-based, fully observable, and no-order-dependence conditions. On a medical benchmark satisfying these conditions, SLIC reduces computation cost by 93.3% while remaining highly consistent with a Monte Carlo Shapley baseline. In more general multi-role workflows, SSV aligns with perturbation-induced score-drop profiles and exposes cases where semantic contribution and failure impact diverge. Overall, SLIC provides a fast, counterfactual-free, and interpretable attribution method for complex LLM-based multi-agent systems.

Figures & tables

Figure 1: Counterfactual vs. semantic routes for contribution attribution. Bottom left: The traditional counterfactual route estimates contributions by masking components and rerunning the workflow to measure value changes, requiring many model calls and high computational cost. Bottom right: The semantic route analyzes a single realized execution by tracing the language/semantic flow across agents. Contributions are computed directly from this dependency structure via the Semantic Shapley Value, with no reruns.

Figure 2: Overall SCG/SLIC pipeline for semantic attribution. The key idea is to compute agent contributions directly from a single realized execution by leveraging its semantic structure. A workflow is transformed into a semantic generation hypergraph $G$ , from which SLIC traces value dependencies, recovers minimal support logic, and computes the Semantic Shapley Value (SSV). This enables single-pass, structure-based attribution without counterfactual reruns.

Method	Kendall $\tau_{b}\uparrow$	L1 Error $\downarrow$	API Calls (per case) $\downarrow$	Cost Reduction $\uparrow$
Monte Carlo (GT)	1.000	0.00	$7R$ (avg 95.9)	Baseline
Leave-One-Out (LOO)	0.575	15.08	$4R$ (avg 54.8)	42.9%
Holistic LLM Judge	0.479	12.90	$R$ (avg 13.7)	85.7%
SLIC (Ours)	0.814	6.61	$R$ (avg 13.7)	85.7%

Table 1: Performance comparison on HealthBench.

R

denotes a normalized per-run cost unit.

Setting	Weak	Medium	Strong	LOO / C3-style
Meeting decision summarization	0.718	1.000	1.000	0.975
Table-text numerical reasoning	0.667	0.900	0.900	0.900
Rule application	0.500	0.700	0.700	0.700
HealthBench-style medical safety	0.564	0.872	0.872	0.667
Mean	0.612	0.868	0.868	0.810

Table 2: Shape alignment (Spearman

\rho

) between SSV and perturbation-induced score-drop profiles in Experiment 2.

Figure 3: Radar plots compare the normalized SSV profile with score-drop profiles induced by weak, medium, strong, and leave-one-out interventions across four evaluation settings. Greater overlap indicates stronger alignment between SSV and intervention-induced degradation patterns.

Appendix figures & tables24 assets

Supplementary material from the paper’s appendix.

Appendix

Relation	Planned	Valid	Confirmed	Rate [95% CI]	$\widehat{p}_{\mathrm{F}}$	$\widehat{p}_{-U}$	$\widehat{p}_{\mathrm{S}}$	Type match
Transformed	120	120	110	.917 [.867, .958]	.950	.028	.956	118/120
Preserved	40	40	39	.975 [.925, 1.000]	.983	.000	.975	40/40
New	160	159	154	.969 [.937, .994]	.969	.969	–	159/159

Appendix

Table 3: Counterfactual validation by semantic relation. Confirmation-rate intervals are 95% bootstrap confidence intervals over relation events. For new relations,

\widehat{p}_{-U}

denotes the anchored-null replay and the sham arm is not applicable.

Task family	$N_{\mathrm{sup}}$	Support conf.	$\widehat{p}_{\mathrm{F}}$	$\widehat{p}_{-U}$	$\widehat{p}_{\mathrm{S}}$	$N_{\mathrm{new}}$	New conf.
HealthBench-style	40	36/40	.917	.008	.925	39	39/39
LegalBench-style	40	37/40	.958	.025	.967	40	37/40
QMSum-style	40	38/40	.958	.000	.950	40	38/40
TAT-QA-style	40	38/40	1.000	.050	1.000	40	40/40
Overall	160	149/160	.958	.021	.960	159	154/159

Appendix

Table 4: Validation across task families. The support-removal columns combine transformed and preserved relations. “New conf.” reports confirmation under anchored-null replay.

Analyzer	SSV rank $\rho$	SSV $\ell_{1}$	Raw-link Jaccard	Link–agent Jaccard
Gemini 2.5 Flash-Lite	$0.831\,[0.814,\,0.848]$	$0.354\,[0.331,\,0.377]$	$0.056\,[0.049,\,0.063]$	$0.613\,[0.591,\,0.635]$
Gemini 2.5 Flash	$0.959\,[0.947,\,0.969]$	$0.112\,[0.098,\,0.126]$	$0.550\,[0.525,\,0.575]$	$0.850\,[0.833,\,0.867]$
Llama 4 Scout	$0.816\,[0.796,\,0.836]$	$0.389\,[0.367,\,0.412]$	$0.060\,[0.053,\,0.068]$	$0.610\,[0.586,\,0.634]$
Llama 4 Maverick	$0.853\,[0.831,\,0.873]$	$0.362\,[0.339,\,0.385]$	$0.099\,[0.089,\,0.110]$	$0.664\,[0.647,\,0.681]$

Appendix

Table 5: Sensitivity to the semantic analyzer. Each replacement analyzer is compared with Gemini 2.5 Pro on the same 200 workflows and 1,950 rubric-level graphs. Brackets show the reported uncertainty intervals.

Panel of Figure 4 — Figure 4: A linear workflow, its dual language flow, and the induced semantic generation hypergraph.

Panel of Figure 5 — Figure 5: A parallel workflow, its dual language flow, and the induced semantic generation hypergraph with multiple value nodes.

Figure 6: A supplementary example of link-level support.

Figure 7: A supplementary example of absorption from link-level logic to agent-level logic.

Figure 8: Workflow of the medical multi-agent collaboration.

Figure 9: The semantic graph for the medical case.

Method	Kendall $\tau_{b}\uparrow$	L1 Error $\downarrow$	API Calls (per case) $\downarrow$	Cost Reduction $\uparrow$
Monte Carlo (GT)	1.000	0.00	$7R$	Baseline
LOO (Duplicated)	0.000	21.18	$4R$	42.9%
Holistic LLM Judge	0.479	12.90	$R$	85.7%
SLIC (Ours)	0.814	6.61	$R$	85.7%

Appendix

Table 6: Method robustness under the redundant setting.

Figure 10: Method robustness under the redundant setting.

Case ID	Total Rubrics	Grand Coalition $2{:}1$ Count	Any-Subset $2{:}1$ Count
0	19	1	3
1	10	2	2
3	20	0	3
4	13	0	2
5	12	1	4
6	14	1	3

Appendix

Table 7: Case-wise of

2{:}1

rubric disagreements

Figure 11: Internal ambiguity distribution of the LLM judge under 3-run rubric evaluation (206 rubrics). A rubric is marked ambiguous when the three runs produce a $2{:}1$ vote split.

Method	API passes / case	API Calls	Kendall $\tau_{b}\uparrow$	L1 Error $\downarrow$
MC (GT)	15	207.0	1.000	0.000
LOO	5	69.0	0.443	14.433
LOO redundant endpoint	5	69.0	0.000	17.000
Holistic	1	13.8	0.499	7.467
SLIC	1	13.8	0.665	5.700

Appendix

Table 8: 4-agent extension under corrected-cost accounting

Figure 12: 4-agent extension under the standard setting. Left: rank consistency versus corrected cost. Right: attribution error versus corrected cost.

Figure 13: Results after introducing deliberate perturbations in the linear workflow.

Setting	$n$	Weak	Medium	Strong	LOO / C3-style
Meeting decision summarization	50	0.718	1.000	1.000	0.975
Table-text numerical reasoning	50	0.667	0.900	0.900	0.900
Rule application	50	0.500	0.700	0.700	0.700
HealthBench-style medical safety	50	0.564	0.872	0.872	0.667
Mean	200	0.612	0.868	0.868	0.810

Appendix

Table 9: Aligned-50 archive: per-setting average shape alignment.

Figure 14: In a branched workflow with clear early-stage specialization and a dedicated end node $F$ responsible for final formatting and integration, $F$ has a low SSV but causes a large system loss after deliberate perturbation.

Series	A	B	C	D
SSV	0.345	0.376	0.279	0.000
Weak	0.510	0.093	0.004	0.013
Medium	0.348	0.387	0.265	0.000
Strong	0.350	0.385	0.265	0.000
LOO	0.354	0.382	0.262	0.002

Appendix

Table 10: Meeting decision summarization: normalized profiles.

Series	A	B	C	D
SSV	0.539	0.342	0.117	0.001
Weak	0.048	0.578	0.134	0.000
Medium	0.536	0.137	0.306	0.020
Strong	0.526	0.138	0.314	0.022
LOO	0.893	0.082	0.006	0.019

Appendix

Table 11: Table-text numerical reasoning: normalized profiles.

Series	A	B	C	D
SSV	0.299	0.200	0.384	0.117
Weak	0.087	0.470	0.215	0.089
Medium	0.251	0.174	0.305	0.269
Strong	0.253	0.182	0.300	0.265
LOO	0.269	0.145	0.298	0.288

Appendix

Table 12: Rule application: normalized profiles.

Series	A	B	C	D
SSV	0.392	0.000	0.384	0.224
Weak	0.017	0.007	0.414	0.562
Medium	0.329	0.008	0.346	0.316
Strong	0.334	0.005	0.346	0.316
LOO	0.152	0.012	0.052	0.783

Appendix

Table 13: HealthBench-style medical safety: normalized profiles.

Setting	Flow	Agent responsibilities
Meeting decision summarization	A/B/C $\to$ D $\to$ E	A: Decision and scope B: Constraints and blockers C: Action items and owners D: Review, backup options, advice E: Transparent pass-through
Table-text numerical reasoning	A/B $\to$ C $\to$ D $\to$ E	A: Targets and values B: Computation rules C: A/B-based computation D: Explanation and final answer E: Transparent pass-through
Rule application	A/B/C $\to$ D $\to$ E	A: Claim focus B: Main rule C: Conditions and exceptions D: Rule application and final judgment E: Transparent pass-through
HealthBench medical safety	A/B $\to$ C $\to$ D $\to$ E	A: Raw case facts B: Formatting and remote-answer boundary C: Risk assessment D: Next-step action and safety boundary E: Transparent pass-through

Appendix

Table 14: Task settings and agent responsibilities in Experiment 2.

Experiment group	Function	Model
Four-scenario radar	Agent generation	models/gemini-2.5-flash
Four-scenario radar	Rubric judge	models/gemini-2.5-flash
Four-scenario radar	Semantic analyzer / link extraction	models/gemini-2.5-flash
Four-scenario radar	Attack / rerun downstream generation	models/gemini-2.5-flash
HealthBench Shapley-alignment	Agent generation / merge	models/gemini-2.5-flash
HealthBench Shapley-alignment	Rubric scoring / rejudge	models/gemini-2.5-pro

Appendix

Table 15: Model and decoding configurations for the main experiments.

Explore similar work

May 26, 2026cs.MA

Agents that Matter: Optimizing Multi-Agent LLMs via Removal-Based Attribution

As multi-agent systems (MAS) become increasingly complex, identifying the contributions of individual agents is critical for system optimization. However, existing approaches lack a rigorous, unified framework for credit assignment. In this work, we formalize agent attribution as a cooperative game, parameterized by the coalition distribution, removal protocol, and target metric. Using this framework, we show that Leave-One-Out (LOO) identifies bottleneck agents as effectively as combinatorial methods, but at a fraction of the computational cost. We also demonstrate that removal protocols induce distinct games: Agent ablation isolates structural bottlenecks, whereas introspective LLM judges fail to faithfully approximate this behavior. Furthermore, to evaluate the utility of specific agent backbones, we introduce attribution via model replacement. By substituting underlying models of low-contribution agents, we improve task performance by up to 17% while reducing cost by up to 35% across three benchmarks. Finally, we apply our framework to audit a medical MAS, revealing that agent contributions to diagnostic accuracy and ethical behavior are often decoupled. By intervening on counterproductive roles, we observe an increase in ethics alignment while maintaining diagnostic accuracy. Overall, this work provides a principled approach for cost-effective MAS attribution and intervention.

Mingyu Lu, Yushan Huang, Chris Lin +1

Paul G. Allen School of Computer Science & Engineering, University of Washington

Model-Based Multi-Agent Systems Multi-Agent Large Language Model Systems Coalition Formation Bottlenecks

Ask PDF

Nov 11, 2025cs.MA

Who Gets the Reward & Who Gets the Blame? Evaluation-Aligned Training Signals for Multi-LLM Agents

Large Language Models (LLMs) in multi-agent systems (MAS) have shown promise for complex tasks, yet current training methods lack principled ways to connect system-level evaluation with agent- and message-level learning. We propose a theoretical framework that unifies cooperative game-theoretic attribution with process reward modeling to transform system evaluation to agent credit to response-level signals. Unlike prior approaches that rely only on attribution (Shapley) or step-level labels (PRM), our method produces local, signed, and credit-conserving signals. In success cases, Shapley-based credit assignment fairly allocates outcomes across agents and is refined into per-message rewards that promote cooperation while discouraging redundancy or sabotage; in failure cases, first-error localization yields repair-aware preferences that penalize harmful steps while rewarding corrective attempts. The resulting signals are bounded, cooperative, and directly compatible with reinforcement- or preference-based post-training, providing a unified and auditable pathway from global evaluation to local supervision in LLM multi-agent training. Our contribution is conceptual: we present a theoretical foundation and training signals, leaving empirical validation for future work.

Chih-Hsuan, Yang, Tanwi Mallick +5

Bella · Argonne National Laboratory · & University of Chicago

Multi-Llm Agents Process Reward Model

Ask PDF

Sep 7, 2026cs.AI

DCFA: Dual-view Causal-inspired Attribution for Failure Reasoning in LLM-based Multi-agent Systems

Large language model (LLM)-based multi-agent systems have experienced rapid growth in recent years. Despite their promise, such systems remain fragile, frequently exhibiting reasoning and coordination errors that can lead to system-level failures. Failure attribution in such systems relies on tracing natural language interactions among agents to identify the decisive error, which refers to the earliest action whose correction can reverse system failure. There are two key challenges: 1) Shallow attribution: Existing methods often capture only minor deviations, such as incomplete retrievals or formatting errors, which verification mechanisms can correct, while missing the decisive cause of system failure. 2) Contextual degradation: As the length of the system traces increases, the model's reasoning ability rapidly deteriorates. To address these challenges, we propose DCFA, a training-free framework for failure attribution. DCFA integrates a global module that constructs structured causal-inspired dependency graphs from system traces to identify the initial decisive error, and a local module that applies local counterfactual-inspired reasoning to refine causal-inspired attribution. Experiments on the Who&When benchmark across six LLMs show that DCFA improves step-level accuracy by up to 8.27% over state-of-the-art baselines.

Zehao Wang, Lanjun Wang, Shilong Jin +2

College of Intelligence and Computing, Tianjin University, Tianjin, China · School of New Media and Communication, Tianjin University, Tianjin, China · College of Computer Science and Artificial Intelligence, Fudan University, Shanghai, China +1

Large Language Models Fail Latent Failure Patterns

Ask PDF