AI-driven Prices for Externalities and Sustainability in Production Markets

Panayiotis Danassis; Aris Filos-Ratsikas; Haipeng Chen; Milind Tambe; Boi Faltings

cs.MAJun 10, 2021

AI-driven Prices for Externalities and Sustainability in Production Markets

Authors: Panayiotis Danassis, Aris Filos-Ratsikas, Haipeng Chen, Milind Tambe, Boi Faltings

Organizations: University of Southampton, Southampton, UK. · University of Edinburgh, Edinburgh, UK. · Imperial College London, London, UK. · William & Mary, Williamsburg, Virginia, USA. ·

Abstract

Traditional competitive markets do not account for negative externalities; indirect costs that some participants impose on others, such as the cost of over-appropriating a common-pool resource (which diminishes future stock, and thus harvest, for everyone). Quantifying appropriate interventions to market prices has proven to be quite challenging. We propose a practical approach to computing market prices and allocations via a deep reinforcement learning policymaker agent, operating in an environment of other learning agents. Our policymaker allows us to tune the prices with regard to diverse objectives such as sustainability and resource wastefulness, fairness, buyers' and sellers' welfare, etc. As a highlight of our findings, our policymaker is significantly more successful in maintaining resource sustainability, compared to the market equilibrium outcome, in scarce resource environments.

Figures & tables

Panel of Figure 1 — Figure 1: Evolution of several metrics over the number of training episodes. The orange line is the baseline (market equilibrium prices). The blue line refers to the vanilla policymaker where each objective in the reward function has the same weight (see Section 4 ). The green line refers to the policymaker that only optimizes the specific objective of each figure (i.e., in 1(a) we set $w_{h}=1$ , in 1(b) we set $w_{b}=1$ , and in 1(d) , 1(e) , and 1(f) we set $w_{s}=1$ and the rest of the weights to 0). The red and purple lines in 1(c) refer to a policymaker with obfuscated valuations (see Section 3.4.1 ). In 1(f) we have a scarce resource setting (see Section 2.4 ). Shaded areas represent one standard deviation.

Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

Parameter	Value
Learning Rate ( $\alpha$ )	0.0001
Clipping Parameter	0.3
Value Function Clipping Parameter	10.0
KL Target	0.01
Discount Factor ( $\gamma$ )	0.99
GAE Parameter Lambda	1.0

Appendix

Table 1: List of hyperparameters.

Figure 2: Evolution of the mean leftover budget over the number of training episodes. The orange line is the baseline (market equilibrium prices). The blue line refers to the vanilla policymaker where each objective in the reward function has the same weight. The green line refers to the policymaker that optimizes for the harvesters’ reward (i.e.,

w_{h}=1

and the rest of the weights to 0). Shaded areas represent one standard deviation.

	Vanilla Policymaker
	$N=B=8$		$N=B=12$
	$R=4$	p-value	$R=6$	p-value
Harvesters’ Social Welfare	-7.44	1.21e-08	-11.2	4.15e-08
Buyers’ Social Welfare	-7.01	2.26e-06	-4.51	5.26e-05
Stock Difference	-15.30	2.72e-09	-19.31	2.24e-16
Harvesters’ Fairness Jain	-0.61	3.71e-03	-0.45	6.37e-04

Appendix

Table 2: Numerical results of the last 400 episodes of each training trial (averaged over the 8 trials). Each odd column represents the relative difference (

\%

) of the vanilla policymaker, as compared to the market equilibrium prices (

100(X_{\text{policymaker}}-Y_{\text{market eq.}})/Y_{\text{market eq.}}

), for each of the metrics presented in each row. Each even column shows the Student’s T-test p-values. The first two columns present the results for an environment with 8 harvesters, 8 buyer classes, and 4 resources (

N=8,R=4,B=8

), while the last two represent a larger scale scenario with 12 harvesters, 12 buyer classes, and 6 resources (

N=12,R=6,B=12

). Note that the results of the first two columns are the same as the ones in Table 10 , and have only been included to facilitate comparison between the two test-cases.

	Policymaker
	Vanilla	Interventions
Harvesters’ Social Welfare	64.38	65.59
Buyers’ Social Welfare	75.66	81.45
Stock Difference	-28.69	-33.64
Wasted Percentage	14.70	2.64
Harvesters’ Fairness Jain	0.98	0.99

Appendix

Table 3: Numerical results of the last 400 episodes of each training trial (averaged over the 8 trials). The first column refers to the vanilla policymaker, while the second to the vanilla policymaker that also optimizes for the extent of the intervention to the market (see Section 4.7 ).

Fair Resource Allocation

Fair resource allocation studies how to distribute scarce goods, services, and computing capacity equitably while accounting for efficiency and constraints.

See all 57 papers

Price of Anarchy of First-Price Auctions in Agentic Markets
Oct 14, 2026Shubhendu Trivedi, Jacob Eisenstein, Alekh Agarwal +3Price of Anarchy Mechanism Design
Google DeepMind · Google Research
Strategyproof Deep Reinforcement Learning for Online Resource Allocation
Oct 27, 2026Fariba Dehghan, Sebastian Stein, Vahid Yazdanpanah +1Online Resource Allocation Mechanism Design
University of Southampton
Learning to Harvest Without Collapse in a Regenerative Commons: A Lagrangian Framework
Sep 29, 2026Jose Tupayachi, Xueping Li, Soham DasConstrained MARL Multi-Agent Control
Oak Ridge National Laboratory Department of Industrial and Systems Engineering, University of Tennessee, Knoxville · Department of Industrial and Systems Engineering, University of Tennessee, Knoxville

Oct 14, 2026J/K move · Enter open · S save

Price of Anarchy of First-Price Auctions in Agentic Markets

Shubhendu Trivedi, Jacob Eisenstein, Alekh Agarwal, Ian Gemp+2

Google DeepMind · Google Research

Price of Anarchy Mechanism Design Game Theory Automated Bidding

Open paper AskPDF arXiv

An agentic market is a decentralized serving-time market where AI agents bid to claim each user prompt. An agent's bid is based on its expected value for the prompt, which is the expectation of the reward for its response minus its inference costs. First-price auctions (FPAs), in which the highest bidder pays its bid and receives the prompt, are a natural way to allocate prompts in such a market, since they are simple and transparent to run, and do not require central knowledge of private capabilities and costs. However, FPAs are not truthful: participants may bid less than their true value in the hope of a larger profit, and the prompt may then be allocated to someone other than the bidder who values it the most. This welfare loss is quantified by the

\textit{price-of-anarchy}

(PoA). In this paper, we argue that agentic markets are likely to possess features that reduce the price-of-anarchy. The first is a

\textit{competitive floor}

, which occurs when multiple bidders usually value the prompt above some baseline share of the optimal welfare (e.g., because the best models are often distilled into cheaper alternatives, which perform similarly). We show that competition among them then limits the welfare loss, which can be driven to zero as the floor rises. The second feature is

\textit{near-independence}

of values after conditioning on the prompt, which occurs when most of the predictable variation in the expected value is determined by the difficulty of the prompt. As the value distribution approaches conditional independence, we can derive an efficiency guarantee that approaches a factor of

1 - 1/e^2

of the optimal welfare. Thus, both features give rise to welfare lower bounds that are significantly better than the well-known general case of

1 - 1/e

. We validate these results empirically using auctions with strategic bidders simulated on the EmbedLLM prompt routing dataset.

Abstract

Fair Resource Allocation

Explore similar work

Price of Anarchy of First-Price Auctions in Agentic Markets

Strategyproof Deep Reinforcement Learning for Online Resource Allocation

Learning to Harvest Without Collapse in a Regenerative Commons: A Lagrangian Framework