cs.AIFeb 5, 2026

PieArena: Ranking and Profiling Language Agents in Realistic Negotiation Scenarios

Authors: Chris Zhu, Sasha Cui, Will Sanok Dufallo, Runzhi Jin, Zhen Xu, Linjun Zhang, Daylian M. Cain

Organizations: Yale University · UC Berkeley · BloomBerg · Rutgers University

Abstract

We present an in-depth evaluation of LLMs' ability to negotiate, a central business task that requires strategic reasoning and theory of mind to create and claim economic value. To do so, we introduce PieArena, a large-scale negotiation benchmark grounded in multi-agent interactions over realistic scenarios adapted from MBA negotiation courses at an elite business school. We evaluate language agents across three pairing regimes: mirror-play, cross-play, and human-LM play. We develop a ranking model for continuous negotiation payoffs that yields order-invariant, uncertainty-quantified leaderboards while correcting for systematic experimental asymmetries. We further study the effects of shared-intentionality agentic scaffolding and find asymmetric gains, with large improvements for mid- and lower-tier LMs and diminishing returns for frontier LMs. As calibration anchors, we collect human-human and human-LM negotiation data from trained business school students, finding that a representative frontier language agent matches or exceeds this human baseline in our evaluation settings. Beyond deal outcomes, PieArena provides a multi-dimensional behavioral profile that reveals cross-model heterogeneity in instruction compliance, computation accuracy, as well as judge-assessed deception and reputation, illustrating the value of evaluation beyond outcome-only leaderboards.

Figures & tables

Appendix figures & tables35 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. TERMS-Bench: Diagnosing LLM Negotiation Agents Beyond Deal Rate

    May 13, 2026Erica Zhang, Fangzhao Zhang, Aneesh Pappu +5Multi-Agent NegotiationAutomated Negotiation