The Shadow Price of Intelligence: Quality Degradation in LLM Inference as a Supply Chain Problem

Elioth Sanabria

math.OCAug 25, 2026

The Shadow Price of Intelligence: Quality Degradation in LLM Inference as a Supply Chain Problem

Authors: Elioth Sanabria

Organizations: Department of Decision and Technology Analytics Lehigh University - College of Business

Abstract

Large language model providers are compute constrained, and a common response to congestion is to degrade service. A degraded answer fails with some probability, and a failed answer either returns as a retry or departs as churn, destroying LTV on an unaccounted ledger. We model inference allocation as a newsvendor whose stockout cost is churned lifetime value, a geometric retry multiplier in which the recycled product is dissatisfaction, and a two-regime transient fluid queue whose arrival rate is made endogenous by retries. Statically, there is a regime in which a cheaper model saves energy per initiated task while consuming strictly more capacity per initiated task, so the discount inverts when capacity binds. Dynamically, a reactive throttle fired during a surge can cross an ignition threshold beyond which it manufactures more traffic than it sheds, and a release rule set below the degraded equilibrium converts a transient surge into a permanent degraded regime. With heterogeneous customers, throttling is a transportation problem in retry-inflated load whose optimal policy rations intelligence by critical ratio, and whose dual, the shadow price of intelligence, prices a marginal query by class and by hour. Under congestion, throttling is not a cost lever but a demand lever.

Figures & tables

Panel of Figure 1 — Figure 1 : The two frontiers (schematic). The posted frontier, per raw query, is monotone: weaker is cheaper, which is what every pricing page implies. The realized frontier, per initiated task, carries the multiplier and the churn charge of ( 5 ) and bends back up as $d_{j}\rho$ grows; past the energy break-even the ordering of the tiers inverts. The red arrow is the inversion at the distilled model: the vertical gap between its posted and realized cost, the retry storm plus the churn, is the cost of confusing a query with an answer.

provider	always strongest	reactive threshold	scheduled	index rule	fluid DP	tree	best/DP
Anthropic	div †	$991.88M 9v	div †	$1.17B 3v	$9.40M	$9.44M	NA
OpenAI	$117.70M 24v	$8.73M 21v	$61.84M 17v	$1.59M	$1.78M	$1.78M	0.89 $\times$
Google	$7.26B 3v	$1.75B 2v	$7.26B 3v	$7.26B 3v	$9.21M	$8.87M	NA
DeepSeek	div †	$27.87M 23v	div †	$2.03M	$2.03M	$2.03M	1.00 $\times$
Kimi	$3.44M	$3.44M	$1.62B 12v	$3.44M	$3.44M	$3.44M	1.00 $\times$

Table 1 : Twenty-four-hour DES-estimated economic cost by policy and provider (30 replications; mean; lower is better), with the per-attempt SLA screen of Section 5.2 applied to the DP’s action menu. Demand is scaled to each provider’s own throughput, so dollar figures compare across policies within a row, not across providers. Baselines: always strongest serves every query at tier 0; the reactive threshold degrades everyone when total backlog crosses a trigger, releasing below a second threshold; scheduled degrades everyone during the published peak windows; the index rule (hourly) re-solves the class-aware transportation problem ( 8 ) hourly against the forecast. All are tuned per instance as in Section 5.2 . † div : the DES-estimated cost exceeds the

1000\times

reporting cutoff under the policy’s own retry feedback (Proposition 4 ); no finite estimate is reported. kv : the policy’s DES-estimated

\mbox{\sf P}(W_{\mathrm{attempt}}>300\,\mathrm{s})

exceeds

0.05

k

of 24 hours (point estimates), and the policy is marked infeasible. The final column reports the best feasible baseline as a multiple of the DP; where no baseline is feasible it reports NA. Latency is DES-estimated throughout; no policy is certified feasible by construction.

Figure 2 : Policy-conditioned sensitivities

\hat{\pi}_{x}(t)

, five providers: finite differences of the twenty-four-hour cost through the closed-form fluid forward pass under the DP’s own actions; dotted verticals mark hours where the optimal action switches, where the sensitivity may kink. These sensitivities agree with the optimized-value derivative ( 9 ) when the entire optimal action sequence and its feasibility remain unchanged under the perturbation; they may differ at action-switch hours. Top panels : the specialized class’s sensitivity decomposed into the direct cost of one more initiated task at the hour’s action,

M_{j}(x)\bigl(c_{j}+d_{j}(x)(1{-}\rho_{x})p_{r}\ell_{x}\bigr)

with the peak electricity multiplier, and the congestion rent above it. Bottom panels : the rent by class, in dollars. The direct floors differ across classes at every hour, so the flat relative price of one that any uniform policy implicitly charges is wrong around the clock, off peak by 3.9

\times

on Anthropic; and the pool charges a near-common rent when capacity binds. Rent panels are clipped where a single hour dominates, the value printed at the marker: on Google the marginal casual query at the surge carries

145 of service-level knock-ons, the sensitivity at the capacity wall. DeepSeek and Kimi hold one action all day, the static optimum: no switches, and rent moves only with load. Sensitivities in the last hours are biased low by the finite horizon, and the rent reaches the latency constraint through the smooth

C_{\mathrm{SLA}}$ term rather than a hard wall.

provider	fluid (s)	Euler (s)	speedup	DES policy-value gap
Anthropic	0.43	34	79 $\times$	-3.68%
OpenAI	2.24	275	123 $\times$	-12.15%
Google	1.29	414	321 $\times$	+2.11%
DeepSeek	0.18	14	75 $\times$	+0.00%
Kimi	0.05	4	74 $\times$	+0.00%

Table 2 : Runtime and DES-estimated accuracy of the fluid solver against a brute-force competitor: the identical dynamic program, grid, action set, and cost function, with the closed-form transition replaced by a converged Euler integration of the same fluid dynamics at the per-provider stable step. The last column is the DES-estimated policy-value gap,

100\times(\mathrm{Euler}-\mathrm{fluid})/\mathrm{Euler}

from the two greedy policies’ twenty-four-hour costs (30 replications each): negative means the Euler-solved policy is cheaper under simulation. It replaces the fluid-rollout policy-value comparison; pointwise Q-regret statistics are in Appendix C .

Appendix figures & tables14 assets

Supplementary material from the paper’s appendix.

Appendix

Panel of Figure 5 — Figure 5 : One transition of the backward induction, magnified. The induction fills the same $T\times|{\cal A}_{1}|\times g^{|{\cal X}|}$ value surface under either solver; the panels differ inside the transition arrow. (a) An explicit integrator resolves the recovering branch’s relaxation at rate $\mu^{\mathrm{eff}}_{j}$ , so each transition takes $(\max_{j}\mu^{\mathrm{eff}}_{j})\Delta/c$ substeps. (b) Lemma 2 gives the same trajectory as a linear leg and an exponential leg joined at the closed-form crossing time $t^{\star}$ : constant cost per transition, independent of the service-rate scale.

Figure 6 : Finite-horizon backlog-growth probability at 30 s against the dissatisfaction probability

d_{1}

(single class, Example 3.2 parameters,

N(0)=7C/6

, tier 1 from

t=0

; 200 replications per point, Wilson 95% CIs). The outcome is

N(30\,\mathrm{s})>17C/6

. Dotted verticals mark the fluid stability threshold

d^{\star}=(1-r/(k\mu_{1}m))/\rho=0.4175

and the fluid finite-horizon growth threshold

d^{\mathrm{fluid}}=0.4217

; the stochastic transition is centered near

d^{\mathrm{fluid}}

and sharpens with

C

, and no irreversibility is implied. Per-

C

detail in Appendix F .

	symbol	meaning
fleet	$k$ , $m$ , $mk$	servers; concurrent slots per server; total capacity in jobs
	$N_{t}$	jobs resident in the system at time $t$
	$\gamma$ , $w_{0}$ , $c_{m}$ , $\kappa$ , $C_{\mathrm{SLA}}$ , $\beta$	electricity price; idle draw per server; memory cost scale and exponent; service-level penalty scale (dollars per hour) and elasticity
tiers	$j\in\{0,\dots,J\}$	quality tier, $0$ the strongest; $\varphi$ a routing mix across tiers
	$\Delta w_{j}$ , ${\bf\sf E}[S_{j}]$ , $\mu_{j}$	power draw per active slot; mean service time; service rate $1/{\bf\sf E}[S_{j}]$
	$d_{j}$	dissatisfaction probability: the answer fails its user, increasing in $j$

Appendix

Table 3 : Notation. The symbols of the first three blocks are primitives, measurable from benchmarks, serving traces, and logged traffic; the derived quantities of the last block carry the retry feedback and are the objects every result compares.

Panel of Figure 7 — Figure 7 : The transportation LP ( 8 ) for two classes and two tiers. Solid arrows are the dominant flows in the calibrated instance; dotted arrows are the cross-assignments the LP considers and rejects. Capacity is charged in effective slot-time ${\tilde{S}}_{j}(x)=M_{j}(x)\,{\bf\sf E}[S_{j}]$ , not posted ${\bf\sf E}[S_{j}]$ : a heavy-retry class consumes capacity its service time conceals, and in the trap region the capacity constraint tightens under degradation.

Panel of Figure 8 — Figure 8 : The boomerang of Example 3.2 , in the example’s units of thousands of jobs. Three levels anchor the vertical axis: capacity $mk=3$ (dotted), the reactive trigger $\widehat{N}_{\mathrm{fire}}=3.5$ (dash-dotted; the reactive policy of Proposition 6 instead fires at $\widehat{N}_{\mathrm{fire}}=mk$ , where the spiral already ignites), and the Little’s-law equilibrium $\bar{N}=\textstyle\sum_{x}\lambda_{x}\,{\tilde{S}}(x)\approx 2.94$ of the anticipatory mix, which sits below capacity. Reactive throttling fires when $N$ crosses the trigger and ignites the retry spiral; the anticipatory path relaxes toward $\bar{N}$ and never saturates. The cost rate spikes before it saves, and the churned lifetime value accrues on a separate, monotone ledger that no cost chart displays.

Figure 9 : Demand and backlog under the policies of Table 1 , five providers, one shared diurnal profile scaled to each provider’s throughput (hour of day; the day is a finite horizon, not a cycle, so the two edges need not match). Top panels : fresh demand, with the static-optimal-mix throughput

\theta_{\mathrm{mix}}

dashed; the morning and afternoon plateaus are the published peak windows, and the overnight spike at 03:00 is the surge. Bottom panels : the resulting backlogs (log scale), the closed-form fluid forward pass of ( 7 ) under each policy’s logged hourly actions; under the DP’s own policy the simulation’s mean hourly backlogs agree with the forward pass to a maximum relative gap of 42.5% over all providers and hours (mean 11.80%). The distilled tree is omitted: visually identical to the fluid DP on every panel. “static LP (hourly)” in the legend is the index rule of Section 5.2 .

provider	perturbation	DP cost	feas.	best baseline/DP	index rule (hourly)/DP
Anthropic	base	$9.38M	feasible	NA	124 $\times$
Anthropic	$\rho_{x}\!\uparrow$ (cap 1)	$5.75M	feasible	NA	212 $\times$
Anthropic	$\rho_{x}\times 0.75$	$16.47M	feasible	NA	60.9 $\times$
Anthropic	$p_{r}\ell\times 0.5$	$7.16M	feasible	NA	163 $\times$
Anthropic	$p_{r}\ell\times 2$	$13.54M	feasible	NA	86.7 $\times$
Anthropic	$C_{\mathrm{SLA}}\times 0.5$	$9.38M	feasible	NA	62.8 $\times$

Appendix

Table 4 : One-at-a-time sensitivity of the DES-estimated comparison (Anthropic and Google; demand profile fixed in every row except the peak/

\theta_{\mathrm{mix}}

provisioning rows). DP cost is the fluid rollout on the perturbed instance (mean and 95% CI over 20 forecast-noise seeds where applicable); feasibility is the DES criterion of Section 5.2 (no hour with DES-estimated

\mbox{\sf P}(W_{\mathrm{attempt}}>300\,\mathrm{s})>0.05

) evaluated on true demand; best-baseline/DP uses feasible baselines only (NA where none is feasible); index rule (hourly)/DP compares the hourly re-solved index rule against the DP. The index rule was not re-solved on noisy forecasts, so its column reads “not run” on the forecast-noise rows (

\sigma

rows carry no archived index-rule value). The full grid is in Appendix E .

customer classes (shared across providers)
	casual	special.	agentic
share A	0.60	0.25	0.15	share of fresh demand
$\rho_{x}$ A	0.30	0.85	0.98	probability an unsatisfied user re-asks
$d_{0,x}$ A	0.02	0.05	0.10	dissatisfaction at the strongest tier
$\beta_{x}$ A	0.8	2.5	3.0	quality sensitivity: $d_{j}(x)=\min\{0.95,\,d_{0,x}+\beta_{x}(1-I_{j}/I_{0})\}$
$p_{r}$ A	0.03	0.08	0.05	churn probability upon abandonment

Appendix

Table 5 : Calibrated instances. Markers give each number’s provenance: B benchmark (Artificial Analysis), P published by the provider, A assumption anchored to public prices, C calibration choice, D derived, S full joint-action count

\to

static-ledger count

\to

post-screen count; the per-attempt screen of Section 5.2 removes tiers only on OpenAI (the

221

s Sol (max), Terra (max) and the

172

s Luna (max)), so the two stages coincide on the other providers. Symbols as in Table 3 , by class

x

(Section 3.3 ).

provider	mean	p95	occ. mean	feas. disagr.
Anthropic	1.83%	7.79%	3.33%	5.77%
OpenAI	1.04%	3.98%	1.56%	0.00%
Google	8.05%	39.53%	11.66%	1.51%
DeepSeek	1.23%	1.83%	1.39%	0.00%
Kimi	0.00%	0.00%	0.00%	0.00%

Appendix

Table 6 : Pointwise Q-regret of the fluid policy under the converged Q-function, on mutually feasible state–action pairs (uniform mean and p95, occupancy-weighted mean), with the feasibility-disagreement rate under the heuristic surrogate. Tails concentrate off-trajectory in deep-saturation grid states on the shared-pool, high-

\mu

providers; the deployment-relevant comparison is the DES-estimated policy-value gap of Table 2 .

provider	policy	mean	95% CI	hours $>0.05$
Anthropic	always strongest	div †	div †	24
Anthropic	reactive threshold	$991.88M	[ $984.08M,$ 999.69M]	9
Anthropic	scheduled (peak windows)	div †	div †	19
Anthropic	index rule (hourly)	$1.17B	[ $1.16B,$ 1.18B]	3
Anthropic	fluid DP	$9.40M	[ $9.39M,$ 9.41M]	0
Anthropic	distilled tree	$9.44M	[ $9.44M,$ 9.45M]	0

Appendix

Table 7 : DES-estimated 24-hour cost: mean and 95% CI over 30 replications per policy and provider, with the number of hours in which the hourly point estimate of

\mbox{\sf P}(W_{\mathrm{attempt}}>300\,\mathrm{s})

exceeds

0.05

. † div: the DES-estimated cost exceeds the

1000\times

reporting cutoff under the policy’s own retry feedback, so no finite estimate is reported — the same convention as Table 1 . Implementation: batched discrete-event simulation at per-provider step

\Delta t

with

\mu_{\max}\Delta t\leq 0.1

(Anthropic

0.996

s, DeepSeek

1

s, Google

0.241

s, Kimi

1

s, OpenAI

0.679

s), per-class Poisson fresh-arrival streams, exponential services by per-job thinning, instantaneous retries, per-class FCFS queues with wait-age buckets, and the fluid model’s proportional capacity shares (per-tier pools where published), all 30 replications simulated in parallel.

provider	cell	DP cost	P0	P1	P2	P5	DP	best base/DP
Anthropic	base	$9.38M	div †	$991.88M	div †	$1.17B	$9.40M	NA
Anthropic	$\rho_{x}\!\uparrow$ (cap 1)	$5.75M	div †	$1.44B	div †	$1.22B	$5.76M	NA
Anthropic	$\rho_{x}\times 0.75$	$16.47M	div †	$1.05B	div †	$1.00B	$16.48M	NA
Anthropic	$p_{r}\ell\times 0.5$	$7.16M	div †	$976.77M	div †	$1.17B	$7.17M	NA
Anthropic	$p_{r}\ell\times 2$	$13.54M	div †	$1.06B	div †	$1.17B	$13.53M	NA
Anthropic	$C_{\mathrm{SLA}}\times 0.5$	$9.38M	div †	$538.03M	div †	$591.51M	$9.42M	NA

Appendix

Table 8 : Full one-at-a-time sensitivity grid (Anthropic and Google): DES-estimated 24-hour mean cost by policy, the DP’s fluid rollout cost, and the best feasible baseline/DP ratio (feasibility by the Table 7 criterion: no hour with DES-estimated

\mbox{\sf P}(W_{\mathrm{attempt}}>300\,\mathrm{s})>0.05

). The base row is taken from the same 30-replication run as Table 7 ; the deterministic perturbation rows use their own 30-replication DES sets and the two forecast-noise rows use 20 seeds, so base and perturbed cells differ by seeds as well as by the perturbation. † As in Table 7 : the cost exceeds

1000\times

the DP’s mean, so no finite estimate is reported. The index rule was not re-solved on noisy forecasts, so its column reads “not run” on the forecast-noise rows.

$d_{1}$	95% CI	$d_{1}$	prob.	95% CI
0.3000	[0.000, 0.019]	0.4175	0.065	[0.038, 0.108]
0.3200	[0.000, 0.019]	0.4180	0.100	[0.066, 0.149]
0.3400	[0.000, 0.019]	0.4185	0.100	[0.066, 0.149]
0.3600	[0.000, 0.019]	0.4190	0.140	[0.099, 0.195]
0.3800	[0.000, 0.019]	0.4195	0.235	[0.182, 0.298]
0.3975	[0.000, 0.019]	0.4200	0.285	[0.227, 0.351]

Appendix

Table 9 : Finite-horizon backlog-growth probability,

C=mk=750

(200 replications per point; Wilson 95% CIs; fluid threshold

d^{\star}=0.4175

; fluid growth threshold

d^{\mathrm{fluid}}=0.4217

; DES step

\mu_{1}\Delta t=0.083

). Grid: coarse spacing

0.02

plus refined rows at spacing

0.0005

within

\pm 0.02

d^{\star}

; the grid is split into two side-by-side halves so that every row from

0.3000

through

0.5000

prints.

$d_{1}$	95% CI	$d_{1}$	prob.	95% CI
0.3000	[0.000, 0.019]	0.4175	0.000	[0.000, 0.019]
0.3200	[0.000, 0.019]	0.4180	0.000	[0.000, 0.019]
0.3400	[0.000, 0.019]	0.4185	0.000	[0.000, 0.019]
0.3600	[0.000, 0.019]	0.4190	0.015	[0.005, 0.043]
0.3800	[0.000, 0.019]	0.4195	0.025	[0.011, 0.057]
0.3975	[0.000, 0.019]	0.4200	0.055	[0.031, 0.096]

Appendix

Table 10 : Finite-horizon backlog-growth probability,

C=mk=3000

(200 replications per point; Wilson 95% CIs; fluid threshold

d^{\star}=0.4175

; fluid growth threshold

d^{\mathrm{fluid}}=0.4217

; DES step

\mu_{1}\Delta t=0.083

). Grid: coarse spacing

0.02

plus refined rows at spacing

0.0005

within

\pm 0.02

d^{\star}

; the grid is split into two side-by-side halves so that every row from

0.3000

through

0.5000

prints.

$d_{1}$	95% CI	$d_{1}$	95% CI
0.3000	[0.000, 0.019]	0.4175	[0.000, 0.019]
0.3200	[0.000, 0.019]	0.4180	[0.000, 0.019]
0.3400	[0.000, 0.019]	0.4185	[0.000, 0.019]
0.3600	[0.000, 0.019]	0.4190	[0.000, 0.019]
0.3800	[0.000, 0.019]	0.4195	[0.000, 0.019]
0.3975	[0.000, 0.019]	0.4200	[0.000, 0.019]

Appendix

Table 11 : Finite-horizon backlog-growth probability,

C=mk=12000

(200 replications per point; Wilson 95% CIs; fluid threshold

d^{\star}=0.4175

; fluid growth threshold

d^{\mathrm{fluid}}=0.4217

; DES step

\mu_{1}\Delta t=0.083

). Grid: coarse spacing

0.02

plus refined rows at spacing

0.0005

within

\pm 0.02

d^{\star}

; the grid is split into two side-by-side halves so that every row from

0.3000

through

0.5000

prints.

Explore similar work

Jun 2, 2026cs.AI

The Shadow Price of Reasoning: Economic Perspective on Optimal Budget Allocation for LLMs

Inference-time scaling has emerged as a critical avenue for enhancing Large Language Models' performance, yet real-world deployment is constrained by strict computational budgets. In this work, we formulate inference budget allocation as a global constrained optimization problem governed by economic principles. By modeling per-query reasoning utility with a shifted-surge function, we derive an optimal allocation policy based on a global shadow price that equilibrates marginal utility under resource scarcity. Based on this theory, we propose Constrained Latent-utility Equilibrium Allocation for Reasoning (CLEAR). It performs rational abandonment and reallocates resources from insolvent queries to solvable queries near their emergence thresholds. Extensive experiments on several reasoning tasks with different traffic streams demonstrate that CLEAR significantly improves the Pareto frontier of total token cost versus mean accuracy. In resource-scarce regimes, CLEAR achieves up to a 3x improvement in global accuracy compared to uniform allocation.

Xu Wan, Speed Zhu, Jianwei Cai +4

1Zhejiang University · 2Tecent HY Team · 3Peking University

Token Budget Allocation Economies

Ask PDF

Jun 14, 2026math.OC

Service-Induced Congestion in Memory-Constrained LLM Serving

In large language model (LLM) serving, each request accumulates persistent graphics processing unit (GPU) memory during service as its key-value cache grows with every generated token. Under high concurrency, aggregate memory usage therefore increases endogenously over time: the service process itself creates future capacity pressure. When memory capacity is exceeded, systems evict active requests, discarding cached state and restarting them later, which wastes computation and reduces throughput. We develop a discrete-time dynamical model of memory-constrained LLM inference that captures admission, memory growth, and eviction under continuous batching. In the saturated-input regime, the system admits both eviction-free fixed points and limit cycles with evictions. For homogeneous workloads, we show that the eviction-free equilibrium is unstable and that, except for a Lebesgue-measure-zero exact-capture set, the system converges to a unique worst-case limit cycle that is asymptotically stable outside this exceptional set, with throughput losses as large as 50%. For heterogeneous workloads, we prove a stability criterion in the two-class common-input setting and explain how the survival-polynomial mechanism generalizes to multiple classes and heterogeneous-input lengths. Under an input-dominated scaling regime, coprime decoding lengths stabilize the eviction-free equilibrium, while non-coprime lengths create synchronized modes that drive instability. These results characterize when workload heterogeneity desynchronizes completions and helps stabilize memory-constrained serving. More broadly, we identify service-induced congestion as a structural instability mechanism and derive scheduling design principles for sustaining high throughput.

Ruicheng Ao, Jing Dong, Gan Luo +1

Institute for Data, Systems, and Society, Massachusetts Institute of Technology, Cambridge, MA 02139 · Columbia Business School, Columbia University, New York, NY 10027 · School of Mathematical Sciences, Peking University, Beijing, China

Large Language Model Serving Large Language Model Memory LLM Inference Optimization Workload Congestion Key-Value Cache+2

Ask PDF

Sep 23, 2026cs.AI

Learning the Cost of Reliable Inference

Benchmarking and routing platforms increasingly act as intermediaries connecting large language model providers with end-users. However, providers on these platforms typically use a fixed price per token, preventing users from achieving the most competitive price for their tasks. In this work, we design a procurement platform where token prices for each task are driven by provider competition, enabling users to secure competitive pricing for guaranteed quality levels. To this end, the platform sequentially routes queries via a reverse second-price auction that incentivizes model providers to truthfully bid their best estimate of the average cost to serve a user's query. As it routes queries, the platform learns the quality offered by each provider and progressively routes queries to the most cost-competitive provider among those meeting a desired quality threshold. To validate our design, we conduct experiments with multiple LLMs from the Llama and Qwen families on popular mathematical reasoning and question-answering benchmarks. The results show that the pricing margin of the most cost-competitive provider on our platform varies significantly---from

10\%

71\%

---depending on the task and quality threshold. This suggests a substantial inefficiency in the current fixed-price market, and it demonstrates that our platform may enable users to capture maximum savings whenever competitive market conditions permit.

Dimitrios Rontogiannis, Ander Artola Velasco, Manuel Gomez Rodriguez

Max Planck Institute for Software Systems Kaiserslautern, Germany

Question-Answering Benchmarks

Ask PDF