cs.MAMar 14, 2026

A Benchmark for Multi-Party Negotiation Games from Real Negotiation Data

Authors: Leo Benac, Jonas Raedler, Zilin Ma, Finale Doshi-Velez

Organizations: School of Engineering and Applied Sciences, Harvard University

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Many real-world multi-party negotiations unfold as sequences of binding, action-level commitments rather than a single final outcome, yet this regime remains under-studied in existing benchmarks. We introduce a benchmark and evaluation framework for this setting, combining a configurable negotiation game generator with document-grounded instances derived from a climate negotiation exercise. We also provide several baseline solvers. Exact evaluation on small games and comparative evaluation on larger instances show that no solver dominates across regimes; performance depends on the structural properties of the game. These results motivate the creation of novel negotiation methods that value partial commitments robustly across diverse strategic regimes. Code and data for the benchmark are available at: https://anonymous.4open.science/r/negotiation_MARL-46B8

Figures & tables

Figure 1: Algorithm performance on small games, measured by ℓ1\ell_{1} error to the exact optimal self-interested payoff (lower is better; columns use different y-scales). Points show mean ±\pm standard error over 50 random games. Columns show payoff regimes (hardest to easiest), rows show incentive alignment, and the x-axis shows the fraction of all-or-nothing goals. Myopic Reward performs best in balanced games, the Upper bound in negative-dominated regimes (especially with Poison Pill traps), and the Lower bound in positive-dominated games. We omit the Upper bound in the positive-dominated regime because its errors would dominate the scale. Error generally increases with non-linearity and is higher in adversarial settings.
 Game Regime
MethodBalancedNegative-Dom. w/ PPNegative-Dom.Positive-Dom.
Adversarial
Reward14.45±1.25\mathbf{14.45\pm 1.25}−12.81±2.65-12.81\pm 2.655.69±3.075.69\pm 3.0746.85±1.0146.85\pm 1.01
Lower−34.09±5.22-34.09\pm 5.22−196.56±14.32-196.56\pm 14.32−185.82±10.21-185.82\pm 10.2153.72±0.81\mathbf{53.72\pm 0.81}
Upper−14.48±13.09-14.48\pm 13.0911.55±3.90\mathbf{11.55\pm 3.90}12.39±4.64\mathbf{12.39\pm 4.64}−260.52±239.92-260.52\pm 239.92
Cooperative
Table 1: Aggregate improvement in terminal payoff relative to the No Negotiation baseline across payoff regimes and incentive alignments. Higher is better. For compactness, values are averaged over all fractions of binary goals and reported as mean ±\pm standard error. Bold entries denote the best- performing method within each regime–alignment setting. More detailed results are in App. J .
MethodDiff. from Baseline
Reward−7.60±5.71\hphantom{-}\mathbf{7.60\pm 5.71}
Lower−0.98±7.97-0.98\pm 7.97
Upper−9.54±12.83-9.54\pm 12.83
RFE (learning)−44.65±26.56-44.65\pm 26.56
PPO (learning)−46.00±26.38-46.00\pm 26.38
Table 2: Large Generator Games with PPO and RFE. Aggregate improvement in terminal payoff relative to the No Negotiation baseline (mean ±\pm standard error). Higher is better.
MethodDiff. from Baseline ( Δ\Delta )
Lower−26.61±9.82\hphantom{-}\mathbf{26.61}\pm\mathbf{9.82}
Reward−18.90±7.57\hphantom{-}18.90\pm 7.57
Upper−82.67±51.36-82.67\pm 51.36
Table 3: Real-Life Negotiation Games. Aggregate improvement in terminal payoff relative to the No Negotiation baseline on five real-world negotiation games (mean ±\pm standard error). Higher is better.
Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

  Game Regime
MethodBinary Frac.BalancedNegative-Dom. w/ PPNegative-Dom.Positive-Dom.
Adversarial Structure
Lower0.00−26.88±9.08-26.88\pm 9.08−189.04±11.82-189.04\pm 11.82−182.31±13.56-182.31\pm 13.5653.83±5.02\mathbf{53.83\pm 5.02}
0.15−33.99±9.09-33.99\pm 9.09−193.82±11.80-193.82\pm 11.80−183.76±13.40-183.76\pm 13.4052.87±4.68\mathbf{52.87\pm 4.68}
0.30−36.48±9.35-36.48\pm 9.35−185.90±11.40-185.90\pm 11.40−176.75±12.85-176.75\pm 12.8553.40±4.61\mathbf{53.40\pm 4.61}
0.50−39.00±9.50-39.00\pm 9.50−217.47±12.57-217.47\pm 12.57−200.45±13.49-200.45\pm 13.4954.78±4.27\mathbf{54.78\pm 4.27}
Appendix
Table 4: Comprehensive breakdown of total pairwise differences ( ∑Δn\sum\Delta_{n} ) across all game regimes, incentive alignments, and fractions of all-or-nothing (all-or-nothing) goals. Values represent the Mean ±\pm Standard Error of the Mean (SEM).
More in this topic

Multi-Agent Negotiation

Multi-agent negotiation studies how autonomous agents bargain, coordinate, and resolve conflicts over resources, contracts, and shared decisions.

See all 53 papers

Explore similar work

CardsList

  1. TERMS-Bench: Diagnosing LLM Negotiation Agents Beyond Deal Rate

    May 13, 2026Erica Zhang, Fangzhao Zhang, Aneesh Pappu +5Multi-Agent NegotiationAutomated Negotiation

    ♠Stanford School of Engineering · ♣Stanford Department of Economics · ♥Stanford Graduate School of Business +1

  2. [Re] Benchmarking LLM Capabilities in Negotiation through Scoreable Games

    Feb 20, 2026Jorge Carrasco Pollo, Ioannis Kapetangeorgis, Joshua Rosenthal +1LLM EvaluationAI Agent Benchmarks

    Informatics Institute University of Amsterdam

  3. DNQ: Deep Nash Q-Network for Partially Observable n-Player Games

    Jun 4, 2026Qintong Xie, Edward Koh, Xavier Cadet +1Nash EquilibriumReinforcement Learning