Knowing Whom to Trust is not Enough: The Trust--Agreement Gap in Repeated Multi-Party LLM Negotiation

Amit Sarker; Moumita Choudhury; Saaduddin Mahmud; Shlomo Zilberstein

cs.AIOct 22, 2026

Knowing Whom to Trust is not Enough: The Trust--Agreement Gap in Repeated Multi-Party LLM Negotiation

Authors: Amit Sarker, Moumita Choudhury, Saaduddin Mahmud, Shlomo Zilberstein

Organizations: University of Massachusetts Amherst

Abstract

Large language model (LLM) agents increasingly participate in negotiations on behalf of individuals and organizations, yet they remain vulnerable to manipulation, particularly in repeated multi-party interactions. Classical multi-agent systems address this challenge through explicit trust and reputation models, whereas existing work on LLM negotiation largely treats trust as an emergent phenomenon. We investigate whether LLM agents can form useful trust estimates from prior interactions and leverage them to improve future negotiation outcomes. We introduce a repeated, multi-party, multi-issue negotiation framework in which recurring agents participate in a sequence of negotiation games while one participant covertly attempts to prevent agreement. After each game, agents assess their counterparts and maintain pairwise trust estimates that carry over to subsequent negotiations. We compare trust-aware and trust-agnostic agents and analyze how explicit trust influences negotiation outcomes. Our results reveal a trust--agreement gap: trust-aware agents can identify the adversary within a few games, but accurate trust estimates do not consistently translate into better agreements. Trust increases agreement rates by up to 23 percentage points, but only when agents are explicitly guided to use it in their decisions. Explicit trust can therefore improve repeated LLM negotiations, but its value depends critically on how trust is assessed and incorporated into agents' reasoning.

Figures & tables

Figure 1: Repeated multi-party negotiation with explicit trust. Five parties negotiate a package over five issues with four options each; a deal needs four of the five votes. Each party holds private ( ) scores, minimum threshold to accept (BATNA, $\beta$ ), and a trust vector with one entry per counterpart. One party covertly prefers no agreement. After every game, each party updates trust with BRS, rubric-based, or free-form assessment, and the updated trust carries into the next game.

Model	Reasoning ( r )	Non-reasoning ( nr )
Gemini-2.5-Flash ( gemini )	47.0 / 0.0	—
Grok-4.1-Fast ( grok )	41.5 / 0.0	66.5 / 45.9
DeepSeek-V4-Flash ( deepseek )	94.5 / 0.5	—
GPT-5.4-Mini ( gpt )	95.5 / 14.7	99.5 / 67.8

Table 1: Baseline agreement rates (%) . Each entry reports total agreements / invalid agreements (below-

\beta

acceptance).

Figure 2: Adversary detection, misidentification, and trust separation in 2HB+2G scenarios. Shading is 95% CI.

	Condition	Agree $(\%)$ $\uparrow$	Surplus $\uparrow$	HB rej. $(\%)$ $\downarrow$	Conv. $(\%)$ $\uparrow$	R/deal $\downarrow$	Agree $(\%)$ $\uparrow$	Surplus $\uparrow$	HB rej. $(\%)$ $\downarrow$	Conv. $(\%)$ $\uparrow$	R/deal $\downarrow$
gemini-r	No trust	47.0 $\pm$ 7.2	8.56 $\pm$ 1.29	72.3 $\pm$ 1.5	9.0 $\pm$ 1.6	12.73 $\pm$ 0.64	36.5 $\pm$ 7.4	5.02 $\pm$ 0.98	66.8 $\pm$ 3.9	20.8 $\pm$ 6.1	11.45 $\pm$ 1.41
	$S_{0.5}{+}P$	41.5 $\pm$ 6.5	7.77 $\pm$ 1.10	79.0 $\pm$ 1.6	7.8 $\pm$ 1.6	12.47 $\pm$ 0.74	31.0 $\pm$ 7.1	4.18 $\pm$ 1.06	69.3 $\pm$ 2.9	12.0 $\pm$ 5.0	11.32 $\pm$ 1.30
	$S_{\mathrm{rand}}{+}P$	46.5 $\pm$ 6.1	8.45 $\pm$ 1.35	74.7 $\pm$ 2.0	9.6 $\pm$ 1.6	12.29 $\pm$ 0.90	33.5 $\pm$ 8.6	4.96 $\pm$ 1.42	69.0 $\pm$ 2.3	12.2 $\pm$ 2.4	10.99 $\pm$ 1.00
	BRS $(S)$	53.0 $\pm$ 11.6	9.84 $\pm$ 2.24	69.9 $\pm$ 2.0	9.6 $\pm$ 2.0	12.81 $\pm$ 0.88	45.0 $\pm$ 7.9	6.23 $\pm$ 1.16	62.9 $\pm$ 2.0	15.3 $\pm$ 5.4	11.32 $\pm$ 0.92
	BRS $(S{+}P)$	50.5 $\pm$ 5.5	9.36 $\pm$ 1.14	72.4 $\pm$ 1.3	9.2 $\pm$ 1.3	12.81 $\pm$ 0.91	46.0 $\pm$ 8.1	6.14 $\pm$ 1.06	61.9 $\pm$ 2.6	18.5 $\pm$ 4.2	10.96 $\pm$ 0.82
	Rubric $(S)$	48.5 $\pm$ 10.4	9.03 $\pm$ 1.96	72.5 $\pm$ 2.4	8.8 $\pm$ 2.4	12.84 $\pm$ 0.77	43.0 $\pm$ 7.8	5.92 $\pm$ 1.21	68.4 $\pm$ 2.2	18.5 $\pm$ 4.7	11.81 $\pm$ 0.89

Table 2: Negotiation outcomes for 2HB+2G (left) and 1HB+3G (right). Mixed FF rows use an adversary of the other model. Bold marks each model’s best mean among same-model conditions. Shaded rows are the baselines.

\pm

is the 95% CI. The outcome results of other models are in Appendix E.3 .

$\phi$	$\alpha_{S}$	$\alpha_{S\to P}$	$\alpha_{S+P}$	$\Delta_{S\to P}$	$\delta_{S\to P}$	$\Delta_{S+P}$	$\Delta_{\mathrm{trust}}$	$\alpha_{S}$	$\alpha_{S\to P}$	$\alpha_{S+P}$	$\Delta_{S\to P}$	$\delta_{S\to P}$	$\Delta_{S+P}$	$\Delta_{\mathrm{trust}}$
BRS	45.5	49.0	50.5	+3.5 $\pm$ 13.8	0.06	+5.0 $\pm$ 8.3	+1.5 $\pm$ 13.1	35.5	66.0	68.5	+30.5 $\pm$ 6.8	0.47	+33.0 $\pm$ 11.5	+2.5 $\pm$ 11.6
Rubric	41.0	49.0	52.0	+8.0 $\pm$ 10.6	0.14	+11.0 $\pm$ 13.5	+3.0 $\pm$ 9.0	37.0	65.0	58.0	+28.0 $\pm$ 10.4	0.44	+21.0 $\pm$ 13.3	$-$ 7.0 $\pm$ 9.3
FF	46.5	50.0	53.5	+3.5 $\pm$ 11.6	0.07	+7.0 $\pm$ 10.1	+3.5 $\pm$ 10.3	45.0	61.5	68.5	+16.5 $\pm$ 12.4	0.30	+23.5 $\pm$ 7.9	+7.0 $\pm$ 10.7

Table 3: Trust–agreement gap of grok-r . 2HB+2G (left) and 1HB+3G (right). Agreement rates

\alpha

are in %, and differences

\Delta

are in points

\pm

paired 95% CI; bold marks intervals that exclude zero. Shaded columns are the lower bounds on the gap.

Figure 3: Honest parties’ acceptance rate of a proposal when their trust in the proposer is low (

<0.45

) or high (

>0.55

) ; the neutral band is excluded. Error bars are 95% CI. The results for other models are in Appendix E.4 .

Figure 4: Declared free-form trust criteria (2HB+2G). Each bar shows the mean share of declared weight that honest assessors give each category under

S

S{+}P

. Appendix D.1 reports all six models.

Panel of Table 4 — Table 4: What trust changes in the agents’ private reasoning (RQ4). Each row gives an observation and excerpts. Color indicates the assessment method and the frame line indicates the guidance.

Appendix figures & tables15 assets

Supplementary material from the paper’s appendix.

Appendix

Structural property	2HB+2G	1HB+3G
Scenarios	20	20
Packages per scenario	1,024	1,024
Honest-feasible packages, mean [range]	9.45 [8, 10]	7.50 [7, 8]
Honest-feasible share of all packages (%)	0.923	0.732
Most constrained party: HB / G / tie (%)	79.9 / 13.2 / 6.9	26.7 / 73.3 / 0.0
Option changes, ideal $\to$ nearest feasible (HB / G)	1.33 / 0.05	2.00 / 1.00

Appendix

Table 5: Structural comparison of the two fixed scenario sets. An ideal package gives an agent its highest-scoring option on every issue. Values aggregate all 20 scenarios; option changes and utility losses are role means. HB: hard bargainer, G: general agent.

Figure 5: Declared free-form trust criteria for all models. Each bar shows the mean share of declared weight that honest assessors give each category under

S

S+P

Figure 6: Adversary detection, misidentification, and trust separation of gemini-r and grok-r in 1HB+3G scenarios. Shading is

95\%

CI.

Figure 7: Adversary detection, misidentification, and trust separation of the other models in 2HB+2G scenarios. Shading is

95\%

CI.

Figure 8: Trust in honest peers and in the adversary for all models in 2HB+2G scenarios. The dotted line marks the prior of

0.5

. Shading is

95\%

CI.

	Condition	Det. $(\%)$ $\uparrow$	MisID $(\%)$ $\downarrow$	Sep. $\uparrow$	Agree $(\%)$ $\uparrow$	Surplus $\uparrow$	HB rej. $(\%)$ $\downarrow$	Conv. $(\%)$ $\uparrow$	R/deal $\downarrow$
deepseek-r	No trust	–	–	–	94.5 $\pm$ 3.1	17.46 $\pm$ 0.55	58.4 $\pm$ 3.1	31.2 $\pm$ 2.6	7.01 $\pm$ 0.53
	BRS $(S)$	92.9 $\pm$ 4.8	2.5 $\pm$ 2.5	0.39 $\pm$ 0.05	94.0 $\pm$ 3.3	17.80 $\pm$ 0.74	54.5 $\pm$ 2.6	36.1 $\pm$ 3.1	6.37 $\pm$ 0.29
	BRS $(S{+}P)$	97.4 $\pm$ 0.9	0.4 $\pm$ 0.2	0.43 $\pm$ 0.03	94.0 $\pm$ 2.3	17.70 $\pm$ 0.49	47.2 $\pm$ 3.9	44.4 $\pm$ 4.8	6.03 $\pm$ 0.58
	Rubric $(S)$	96.3 $\pm$ 3.5	1.3 $\pm$ 1.2	0.26 $\pm$ 0.02	97.0 $\pm$ 2.5	17.99 $\pm$ 0.58	54.3 $\pm$ 2.2	34.4 $\pm$ 3.7	6.46 $\pm$ 0.28
	Rubric $(S{+}P)$	93.6 $\pm$ 4.7	2.9 $\pm$ 2.8	0.24 $\pm$ 0.03	92.5 $\pm$ 3.5	17.13 $\pm$ 0.45	53.8 $\pm$ 3.2	37.1 $\pm$ 5.3	6.48 $\pm$ 0.53
	FF $(S)$	96.3 $\pm$ 3.0	1.4 $\pm$ 1.4	0.33 $\pm$ 0.03	94.0 $\pm$ 3.7	17.69 $\pm$ 0.93	54.7 $\pm$ 2.1	32.9 $\pm$ 2.5	6.73 $\pm$ 0.54

Appendix

Table 6: Trust formation and negotiation outcomes of the other models in 2HB+2G scenarios. Det., MisID, and Sep. average all 20 episodes. Bold marks each model’s best mean. Shaded rows are the baselines.

\pm

is the

95\%

CI.

Figure 9: Paired effects of trust on negotiation outcomes of the other models. Each cell compares a trust-aware condition with no trust. Agree, HB rej., and Conv. change in points, and Surplus and R/deal change in percent of the no-trust mean. Blue is better and red is worse. Black borders mark

95\%

CIs that exclude zero.

Figure 10: Acceptance of a proposal by trust in its proposer for the other models. Bars show honest parties’ acceptance of proposals that meet their BATNA when trust in the proposer is low (

<0.45

) or high (

>0.55

). Bars from fewer than three trajectories are omitted. Error bars are

95\%

CI.

Figure 11: Trust in the adversary by the assessing agent’s role in 2HB+2G scenarios. The dotted line marks the prior of

0.5

. Shading is

95\%

CI.

Figure 12: Difference between hard-bargainer and general-agent trust in the adversary. Positive values mean that hard bargainers trust the adversary more. Shading is

95\%

CI.

Figure 13: EMA sensitivity for the two reasoning models. The panels overlay rubric-based and free-form

S

and

S+P

conditions at

\lambda\in\{0.5,0.7,0.9\}

. Solid and dashed lines denote rubric-based and free-form assessment; purple/pink, blue/green, and orange/red pairs denote

\lambda=0.5

0.7

, and

0.9

, with open markers for

S+P

. The episode signals are held fixed across decay values, and the

\lambda=0.7

curves reproduce the observed trust states. Averaged across models, conditions, and episodes,

\lambda=0.5

0.7

, and

0.9

respectively yield detection rates of

85.0\%

88.0\%

, and

92.2\%

, misidentification rates of

7.1\%

4.8\%

, and

3.0\%

, and trust separations of

0.327

0.304

, and

0.207

. We use

\lambda=0.7

because it improves ranking over

0.5

while retaining

47\%

more separation than

0.9

Figure 14: EMA sensitivity across all trust-assessment and guidance conditions. Each panel extends Figure 7 by overlaying the four rubric-based/free-form and

S

S+P

conditions at

\lambda\in\{0.5,0.7,0.9\}

. Solid and dashed lines denote rubric-based and free-form assessment, respectively. Purple/pink, blue/green, and orange/red pairs denote

\lambda=0.5

0.7

, and

0.9

; within each pair, open circles mark

S+P

. We recompute the trust trajectories offline from the inferred episode signals while holding negotiation behavior fixed. The

\lambda=0.7

curves reproduce Figure 7 ; the other curves show sensitivity to the decay value and do not represent negotiations conducted with those decay values.

Figure 15: Sensitivity of Grok-4.1-Fast (R) free-form trust formation to the initial trust prior. All conditions use

S+P

guidance. The panels report strict detection, misidentification, and trust separation after each episode for

\tau^{1}\in\{0.1,0.5,0.9\}

. The dotted references mark random adversary ranking and zero separation. The

0.1

and

0.9

conditions form the direct prior comparison;

0.5

is included as the main-paper reference. Each completed condition contains 10 independent seeds and 200 episodes; shaded bands show

95\%

confidence intervals across seeds.

Figure 16: BRS trust formation under different forgetting factors

\gamma

in 2HB+2G scenarios.

\gamma=0.9

reproduces the observed trust. Shading is

95\%

CI.

Condition	System-level trust text	Decision-level trust text	Post-episode update
No trust	None	None	None
Rubric (S)	Visibility	Scores	Rubrics
Rubric (S+P)	Visibility + guidance	Scores + reminder	Rubrics
Free-form (S)	Visibility	Scores	Free-form judgment
Free-form (S+P)	Visibility + guidance	Scores + reminder	Free-form judgment
BRS (S)	Visibility	Scores	Bayesian

Appendix

Table 8: Composition of the experimental prompt conditions. Metrics and free-form conditions use identical negotiation-decision prompts at the same guidance level and differ only in the post-episode trust-update prompt.

Multi-Agent LLM Systems

Multi-agent large language model systems coordinate multiple AI agents to solve tasks through communication, specialization, shared tools, and memory.

See all 1,040 papers

Counterparty Modeling is Not Strategy: The Limits of LLM Negotiators
May 15, 2026Romain Cosentino, Sarath Shekkizhar, Adam Earle +1Multi-Agent Negotiation Automated Negotiation
Salesforce AI Research
Used Car Salesbots? Honesty and Credulity of LLMs as Bargaining Agents under Partial Information
May 29, 2026Antonio Valerio Miceli-Barone, Vaishak Belle, Shay B. CohenMulti-Agent LLM Systems Multi-Agent Negotiation
University of Edinburgh
TERMS-Bench: Diagnosing LLM Negotiation Agents Beyond Deal Rate
May 13, 2026Erica Zhang, Fangzhao Zhang, Aneesh Pappu +5Multi-Agent Negotiation Automated Negotiation
♠Stanford School of Engineering · ♣Stanford Department of Economics · ♥Stanford Graduate School of Business +1

May 15, 2026J/K move · Enter open · S save

Counterparty Modeling is Not Strategy: The Limits of LLM Negotiators

Romain Cosentino, Sarath Shekkizhar, Adam Earle, Silvio Savarese

Salesforce AI Research

Multi-Agent Negotiation Automated Negotiation LLM Agents

Open paper AskPDF arXiv

Negotiation requires more than inferring what the other side wants: it requires using that information to make advantageous offers and counteroffers over multiple turns. We study whether large language model (LLM) agents do this in a controlled multi-attribute bargaining environment. We find that current LLM agents can model a counterparty's preferences, but do not reliably turn that knowledge into strategic bargaining. When given negotiating partner preference information, agents model it accurately and early in their reasoning traces, yet this does not reliably improve outcomes for the informed side. Turn-level analyses show why: agents often respond to what they believe the counterparty values, but do not consistently pair those moves with gains on their own high-value attributes. Sellers are more accommodating overall, and in asymmetric-information conditions, the informed side often makes the more weakly compensated concessions. Because agents fail to leverage this underlying utility structure for strategic advantage, their final agreements are heavily dictated by surface-level opening anchors rather than actual utility weights. Finally, requiring agents to explicitly state concession-for-reciprocity trades before making an offer makes individual turns look more strategic, but ultimately fails to improve the efficiency of the final agreements.

Abstract

Multi-Agent LLM Systems

Explore similar work

Counterparty Modeling is Not Strategy: The Limits of LLM Negotiators

Used Car Salesbots? Honesty and Credulity of LLMs as Bargaining Agents under Partial Information

TERMS-Bench: Diagnosing LLM Negotiation Agents Beyond Deal Rate