The Price Reversal Phenomenon: When Cheaper Reasoning Models Cost More

Lingjiao Chen; Chi Zhang; Yeye He; Ion Stoica; Matei Zaharia; James Zou

cs.CLMar 25, 2026

The Price Reversal Phenomenon: When Cheaper Reasoning Models Cost More

Authors: Lingjiao Chen, Chi Zhang, Yeye He, Ion Stoica, Matei Zaharia, James Zou

Organizations: Stanford University · Microsoft Research · CMU · UC Berkeley

Abstract

Developers and consumers increasingly choose reasoning models (RMs) based on their listed API prices. However, how accurately do these prices reflect actual inference costs? We conduct the first systematic study of this question, evaluating 8 frontier RMs across 12 diverse tasks covering competition math, science QA, code generation, and multi-domain agents. We uncover the pricing reversal phenomenon: in 32% of model-pair comparisons, the model with a lower listed price actually incurs a higher total cost, with reversal magnitude reaching up to 28x. For example, Gemini 3 Flash's listed price is 80% cheaper than GPT-5.4's, yet its actual cost across all tasks is 38% higher. We build a formal cost attribution framework based on Shapley value, and leverage it to trace the dominating contributors to vast heterogeneity in thinking token consumption and number of interaction turns: on the same query, one model may use 900% more thinking tokens than another, or 10x more turns of environment interactions. We further show that per-query cost prediction is fundamentally difficult: repeated runs of the same query yield thinking token variation up to 9.7x, establishing an irreducible noise floor for any predictor. Thus, we propose cost distribution prediction as an open challenge. Our findings demonstrate that listed API pricing is an unreliable proxy for actual cost, calling for cost-aware model selection and transparent per-request cost monitoring.

Figures & tables

Figure 1: The phenomenon of mismatch between RM pricing and their actual costs. The actual cost is aggregated over all 12 tasks, including AIME, MMLUPro, Human Last Exam, SimpleQA, GAIA, and Terminal-Bench 2.0. Overall, models with lower listed prices may incur much higher expenses than those with higher prices. For example, Gemini 3 Flash’s list price (

3.5/1 million tokens) is 80% cheaper than GPT-5.4 (

17.5), but its actual cost (

705) is actually 38% higher than GPT-5.4 (

509). This dramatically changes the cost ranking and poses a pressing challenge to cost-sensitive users. For example, one might choose Gemini 3 Flash over Kimi K2.6 due to its lower price, but recognize later that it is much more expensive.

Figure 2: The pricing reversal phenomenon. (a) Across 8 models evaluated on 12 diverse tasks, we observe that the listed price rankings systematically mismatch the actual costs. On average, 32% of model pairs exhibit pricing reversal, i.e., a model with a lower listed price ends up with a higher cost. The actual cost rankings vary substantially across tasks, too. (b) For example, on MMLU-pro, Gemini 3.1 Flash’s actual cost is the highest, though it is ranked 7th by listed price. (c) However, Kimi K2.6 becomes the most expensive one on Cybench despite its low listed price. This suggests that naively assessing cost by a fixed API pricing Table is thus not reliable.

Symbol	Description	Symbol	Description
$T$	Total turns	$\bar{n}_{\text{think}}$	Avg. thinking tokens/turn
$\bar{n}_{\text{in}}$	Avg. fresh input tokens/turn	$p_{\text{in}}$	Price of fresh input tokens
$\bar{n}_{\text{out}}$	Avg. output tokens/turn	$p_{\text{out}}$	Price of output (incl. thinking)
$\bar{n}_{\text{cache}}$	Avg. cached tokens/turn	$p_{\text{cache}}$	Price of cached tokens

Table 1: The eight cost factors.

Figure 3: Cost contributions assigned by our cost attribution framework. Overall, the number of thinking tokens is the dominating factor for single-turn tasks. More than 95% of the cost difference among reversed model pairs is attributed to thinking tokens. For multi-turn tasks, on the other hand, both the number of turns and the cached input tokens contribute substantially. This suggests that the historical context difference is a unique factor in agentic systems.

Figure 4: Cost attribution breakdown by task difficulty. On easy tasks, the cost gap is often small, as most models do not need too much effort. On hard questions, the gap becomes substantial, especially for multi-turn tasks. In particular, the impact of cached tokens also becomes more visible.

Figure 5: Per-query cost distributions for a fixed (model, query) pair on a signle-turn task (AIME) and a multi-turn task (Terminal-Bench 2.0). Even after fixing both, repeated rollouts span up to $9.7\times$ between the cheapest and most expensive run. This suggests that the cost itself is a big

Appendix figures & tables9 assets

Supplementary material from the paper’s appendix.

Appendix

Dataset	Category	# tasks	Description
Single-turn workloads
AIME Art of Problem Solving [2025]	Math	$60$	Competition math (AMC/AIME)
ARC-AGI Chollet [2019]	Reasoning	$400$	Visual / abstract reasoning
ArenaHard Li et al. [2024]	Chat	$750$	Open-ended hard prompts
GPQA Rein et al. [2023]	Science	$198$	Graduate-level QA (diamond)
HLE Phan et al. [2025]	Mixed	$500$	Humanity’s Last Exam (subset of $2{,}056$ )

Appendix

Table 2: Details of evaluated datasets.

Model	Provider	API model code	$p_{i}$	$p_{o}$	$p_{\text{cache}}$	$L_{m}$
GPT-5.4	OpenAI	gpt-5.4	2.50	15.00	0.25	17.50
GPT-5.4 Mini	OpenAI	gpt-5.4-mini	0.75	4.50	0.075	5.25
Gemini 3.1 Pro	Google	gemini-3.1-pro-preview	2.00	12.00	0.20	14.00
Gemini 3 Flash	Google	gemini-3-flash-preview	0.50	3.00	0.05	3.50
Claude Opus 4.7	Anthropic	claude-opus-4-7	5.00	25.00	0.50	30.00
Claude Haiku 4.5	Anthropic	claude-haiku-4-5	1.00	5.00	0.10	6.00

Appendix

Table 3: API model identifiers and pricing (USD per MTok), recorded May 1, 2026 .

Model	$T$	top- $p$	max_tokens	Reasoning configuration
GPT-5.4	default	default	default	reasoning_effort="high"
GPT-5.4 Mini	default	default	default	reasoning_effort="high"
Gemini 3.1 Pro	default	default	default	built-in adaptive thinking
Gemini 3 Flash	default	default	default	built-in adaptive thinking
Claude Opus 4.7	default	default	$32{,}000$	adaptive thinking
Claude Haiku 4.5	default	default	default	extended thinking

Appendix

Table 4: Decoding and reasoning configuration. We use default decoding parameters. For OpenAI models, we set a high reasoning effort since these tasks are often considered challenging. Single-turn requests use a

1{,}800

s per-call timeout; multi-turn requests use a 5-hour timeout per task.

Figure 6: Listed-price vs. actual-cost ranking on all

12

benchmarks (

9

single-turn

+

3

multi-turn). Crossings between left- and right-column ranks are reversed model pairs.

Model	AIME	ARC	Arena	GPQA	HLE	LCB	LMB	MMLU	SQA
GPT-5.4	389	4,596	1,090	585	17,309	3,284	289	1,804	4,722
GPT-5.4 Mini	272	3,617	1,140	482	8,766	2,097	284	2,964	3,530
Gemini 3.1 Pro	646	5,340	2,061	1,147	35,735	8,231	586	8,007	30,122
Gemini 3 Flash	994	12,579	2,087	3,619	57,797	53,338	1,754	71,489	5,024
Claude Opus 4.7	321	2,797	1,019	614	13,599	2,405	279	1,827	1,336
Claude Haiku 4.5	1,074	7,267	2,146	1,751	6,047	9,023	1,086	9,301	121

Appendix

Table 5: Total thinking tokens (in thousands) by (model, single-turn dataset). Abbreviations: ARC = ARC-AGI, Arena = ArenaHard, HLE=Human Last Exam, LCB = LiveCodeBench, LMB = LiveMathBench, MMLU = MMLU-Pro, SQA = SimpleQA.

Figure 7: Per-dataset Shapley breakdown across all

12

benchmarks. The dominant cost-driving factor varies systematically with workload.

Figure 8: Per-query cost distribution by (model, dataset). Long-tailed in every cell.

Single-turn
Dataset	MEAN	Q1	Q10	Q20	Q40	Q50	Q70	Q90	Q99
AIME	32.1	50.0	53.6	57.1	53.6	50.0	42.9	42.9	17.9
ARC-AGI	14.3	50.0	42.9	42.9	25.0	21.4	17.9	17.9	17.9
ArenaHard	10.7	28.6	17.9	17.9	17.9	14.3	10.7	10.7	17.9
GPQA	35.7	50.0	53.6	50.0	60.7	60.7	53.6	25.0	17.9
HLE	28.6	32.1	42.9	50.0	46.4	42.9	28.6	21.4	17.9

Appendix

Table 6: Per-dataset listed-price reversal rate (%) by cost statistic. Last row is the pooled (unweighted) mean across the

12

datasets, matching Fig. 9 of the main text.

Figure 9: Listed-price reversal rate vs. which cost statistic the user cares about, pooled over

12{\times}28{=}336

ordered model pairs. (a) Global pooled across all

12

datasets: mean reversal is

31.5\%

, but the body

Q_{15}

–

Q_{40}

rises to a

39

–

40\%

plateau (peak

Q_{40}

39.6\%

); the high tail

Q_{99}

is the lowest at

25.0\%

, so the mean tracks the high tail rather than the body. (b–d) Local workload regimes underlying the pooled curve: reasoning (single-turn: AIME, GPQA, HLE, LiveMathBench, LiveCodeBench) is a clean inverted-U; short-answer / chat (single-turn: ARC-AGI, ArenaHard, MMLU-Pro, SimpleQA) is roughly flat near

30\%

; agentic (multi-turn: TerminalBench 2, Cybench, GAIA) is bimodal with a body peak at

Q_{40}

–

Q_{85}

. The mean is a lossy summary in every regime.

Cost-Aware Inference

Cost-aware inference allocates models, computation, and serving resources to meet quality goals at lower latency, energy use, and monetary cost.

See all 281 papers

The Price of Thinking: Reasoning Effort as a Model-Specific API Contract
Aug 16, 2026Yeabin MoonLLM Inference Cost-Aware Inference
Brandeis School of Business and Economics Brandeis University
Learning the Cost of Reliable Inference
Sep 23, 2026Dimitrios Rontogiannis, Ander Artola Velasco, Manuel Gomez RodriguezCost-Aware Inference LLM Routing
Max Planck Institute for Software Systems Kaiserslautern, Germany
PyroDash: Cost-Efficient Token-Level Small-Large Language Model Collaborative Inference
Jul 22, 2026Niqi Lyu, Pengtao Shi, Wei Qiu +4RL for Language Model Reasoning Cost-Aware Inference
Pyromind Dynamics Inc.

Aug 16, 2026J/K move · Enter open · S save

The Price of Thinking: Reasoning Effort as a Model-Specific API Contract

Yeabin Moon

Brandeis School of Business and Economics Brandeis University

LLM Inference Cost-Aware Inference

Open paper AskPDF arXiv

API buyers purchase a dated contract, not a model name alone: the contract includes the requested and served model, reasoning-effort term or its omission, output rail, service product, prompt, and price schedule. We study the reasoning-effort term through a registered paired contrast of Sonnet 5 with explicit high effort against the same model with effort omitted, using 30 AIME 2026 items and five calls per item. Every paid attempt was assigned one frozen terminal category, and inference resampled items while retaining their repeated calls. Mean delivered cost was $0.01031 per call higher under the explicit-high contract than under the omitted contract [+$0.00204, +$0.01974]. The corresponding accuracy contrast was +0.0133 [-0.0267, +0.0467]; we did not detect an accuracy difference, and the interval permits a gain of up to 4.67 percentage points that this design cannot rule out. Cost per correct answer was $0.08665 under the high-effort contract and $0.07662 under the omitted contract, as registered point estimates. A dated contract census, Models-API metadata, and preregistered raw-response probes further documented model-specific omission semantics, including within a provider; claims remained at documentation grade when raw structure was indeterminate. The request registry, parser, terminal taxonomy, statistical plan, and analysis pipeline were frozen before outcomes were examined; the resulting claims are bounded to the model, task, and collection date studied.

Abstract

Cost-Aware Inference

Explore similar work

The Price of Thinking: Reasoning Effort as a Model-Specific API Contract

Learning the Cost of Reliable Inference

PyroDash: Cost-Efficient Token-Level Small-Large Language Model Collaborative Inference