From Chats to Markets: AgenticPay for LLM-Powered Negotiation in Multi-Agent Commerce
Organizations: UC Berkeley.
Abstract
Large language model agents are increasingly expected to negotiate and transact autonomously, yet existing benchmarks rarely combine multimodal product grounding, multi-issue bargaining, and competitive multi-agent markets. We introduce AgenticPay, a unified benchmark for evaluating multimodal agents' ability to reach high-welfare, multidimensional agreements in realistic markets. AgenticPay includes 160 multimodal tasks across four business scenarios (E-commerce, Food Delivery, Ride-hailing, and Apartment Rental) and eight market topologies, from bilateral bargaining to many-to-many competition. Agents interpret product images, infer opponents' hidden preferences, and negotiate binding contract terms such as price, lease duration, return policy, and delivery speed through multi-round natural-language dialogue. A utility-based framework evaluates agreement quality and joint welfare. Experiments with proprietary (GPT-5.4, Claude Sonnet 4.6, Gemini 3.1 Pro Preview) and open-weight (Qwen3-VL-32B-Instruct, InternVL3-38B) multimodal models reveal substantial limitations in value creation and market reasoning. Even the strongest model, Gemini 3.1 Pro Preview, achieves a GlobalScore of only 42.3/100. As markets shift from bilateral to multi-sided, average GlobalScore falls by 5.7 points, with open-weight models declining by up to 11.5 points. AgenticPay provides a testbed for evaluating multimodal agents in realistic commerce. Code and data: https://github.com/SAIL-Research-Lab/AgenticPay.
Figures & tables
| Model | GlobalScore | BuyerScore | SellerScore | Timeout Rate | Avg. Rounds |
|---|---|---|---|---|---|
| Proprietary models | |||||
| Gemini 3.1 Pro Preview | 42.3 | 45.6 | 37.8 | 0.6% | 5.7 |
| GPT-5.4 | 35.6 | 34.9 | 32.8 | 3.1% | 4.5 |
| Claude Sonnet 4.6 | 34.8 | 30.7 | 34.6 | 7.5% | 6.6 |
| Avg. | 37.6 | 37.1 | 35.1 | 3.7% | 5.6 |
| Open-weight models | |||||
| GlobalScore | ||
|---|---|---|
| Model | Bilateral | Market |
| Gemini 3.1 Pro Preview | 47.3 | 40.6 6.7 |
| GPT-5.4 | 36.8 | 35.2 1.6 |
| Claude Sonnet 4.6 | 35.3 | 34.6 0.7 |
| Qwen3-VL-32B-Instruct | 30.9 | 23.2 7.7 |
| InternVL3-38B | 31.5 | 20.0 11.5 |
| GlobalScore | ||
|---|---|---|
| Method | Bilateral | Market |
| AgenticPay | 30.9 | 23.2 |
| AgenticPay w/o visual grounding | 25.8 5.1 | 19.6 3.6 |
| AgenticPay w/o mental model | 27.4 3.5 | 17.5 5.7 |
Appendix figures & tables10 assets
Supplementary material from the paper’s appendix.
Appendix
| Market Topology | E-commerce | Ride-hailing | Food Delivery | Apartment Rental | Total |
|---|---|---|---|---|---|
| Single Buyer/Product/Seller | 5 | 5 | 5 | 5 | 20 |
| Multi-Buyer Only | 5 | 5 | 5 | 5 | 20 |
| Multi-Seller Only | 5 | 5 | 5 | 5 | 20 |
| Multi-Product Only | 5 | 5 | 5 | 5 | 20 |
| Multi-Buyer + Multi-Seller | 5 | 5 | 5 | 5 | 20 |
| Multi-Product + Multi-Seller | 5 | 5 | 5 | 5 | 20 |
| Scenario | Sample Products / Services | Price Range | Source | Tasks |
|---|---|---|---|---|
| E-commerce | Beauty, Electronics, Apparel, Furniture, Beverage | 320 | Amazon [ 29 ] | 40 |
| Ride-hailing | NYC Manhattan taxi trips (Gramercy, Murray Hill, …) | 45 | NYC TLC [ 46 ] | 40 |
| Food Delivery | Karaage Chicken, Fries, Sprite, … | 23 | Kaggle [ 25 ] | 40 |
| Apartment Rental | Studios & villas in NYC, Barcelona, Rio, Sydney | 12.4k/mo | Airbnb [ 17 ] | 40 |
| Total | 160 | |||
| Section | Buyer prompt content |
|---|---|
| Role | You are {name} , a buyer looking for a good deal. {role_description} {personality_section} |
| Context inputs | Context Information: {context} |
| Current Negotiation State: {current_state} | |
| Conversation History: {conversation_history} | |
| Current product information: {product_info} |
| Section | Seller prompt content |
|---|---|
| Role | You are {name} , a seller trying to maximize profit while being reasonable. {role_description} {personality_section} |
| Context inputs | Context Information: {context} |
| Current Negotiation State: {current_state} | |
| Conversation History: {conversation_history} | |
| Current product information: {product_info} |
| Model | GlobalScore | BuyerScore | SellerScore |
|---|---|---|---|
| Gemini 3.1 Pro Preview | 42.1 0.9 | 45.4 1.1 | 37.9 1.0 |
| GPT-5.4 | 35.4 1.2 | 34.7 1.3 | 33.0 0.8 |
| Claude Sonnet 4.6 | 34.9 1.0 | 30.9 0.7 | 34.4 1.1 |
| Qwen3-VL-32B-Instruct | 24.8 1.7 | 23.3 1.8 | 30.6 1.4 |
| InternVL3-38B | 23.1 2.0 | 21.6 2.1 | 23.5 1.5 |
| Model | Temp. 0.0 | Temp. 0.7 (default) | Temp. 1.0 |
|---|---|---|---|
| Gemini 3.1 Pro Preview | 42.4 | 42.6 | 41.9 |
| Qwen3-VL-32B-Instruct | 24.2 | 24.5 | 24.1 |
| Method | GlobalScore |
|---|---|
| w/o mental model (simpler) | 20.3 |
| Full baseline (default) | 24.5 |
| + opponent simulation [ 38 ] (stronger) | 25.9 |
| Model | GlobalScore | Nash | Social | Egalitarian |
|---|---|---|---|---|
| (0–100) | (0–0.25) | (0–1) | (0–0.5) | |
| Gemini 3.1 Pro Preview | 42.3 | 0.075 | 0.54 | 0.148 |
| GPT-5.4 | 35.6 | 0.060 | 0.41 | 0.119 |
| Claude Sonnet 4.6 | 34.8 | 0.064 | 0.43 | 0.126 |
| Qwen3-VL-32B-Instruct | 25.1 | 0.043 | 0.35 | 0.083 |
| InternVL3-38B | 22.9 | 0.044 | 0.29 | 0.085 |
| Aggregation | Bilateral | Market |
|---|---|---|
| GlobalScore (0–100) | 47.3 | 40.6 |
| Nash product (0–0.25) | 0.086 | 0.071 |
| Social welfare (0–1) | 0.58 | 0.52 |
| Egalitarian (0–0.5) | 0.172 | 0.139 |
| Model | Linear (default) | Attr. | Attr. | Concave ( ) |
|---|---|---|---|---|
| Gemini 3.1 Pro Preview | 0.075 | 0.066 | 0.079 | 0.068 |
| GPT-5.4 | 0.060 | 0.052 | 0.063 | 0.058 |
| Claude Sonnet 4.6 | 0.064 | 0.057 | 0.067 | 0.071 |
| Qwen3-VL-32B-Instruct | 0.043 | 0.037 | 0.045 | 0.041 |
| InternVL3-38B | 0.044 | 0.038 | 0.046 | 0.042 |
Multi-Agent Negotiation
Multi-agent negotiation studies how autonomous agents bargain, coordinate, and resolve conflicts over resources, contracts, and shared decisions.
See all 63 papers