Think More or Read More? A Token-Budget Theory of Reasoning with Shared Context

Manh Nguyen; Sunil Gupta; Hung Le

cs.CLOct 16, 2026

Think More or Read More? A Token-Budget Theory of Reasoning with Shared Context

Authors: Manh Nguyen, Sunil Gupta, Hung Le

Organizations: Applied Artificial Intelligence Initiative, Deakin University, Australia

Abstract

Reasoning systems increasingly improve answers by incorporating additional context into the model's input, from multi-agent debate, where agents read their peers' responses, to self-refinement, where a model revisits its own drafts. In these systems, communication consumes context as one agent's output becomes another's input. We study the regime where token reading and generation draw on one fixed token window, so every token spent reading is a token that cannot be spent reasoning. We develop a token-budget theory that formalizes this trade-off, modeling accuracy as a function of the reasoning budget remaining after reading and aggregating votes with a Condorcet jury model that permits correlated errors. For multi-agent debate, the theory gives a closed-form prediction for the optimal number of agents. This quantity is estimated from single-agent measurements, allowing the optimal group size to be determined without running debate, and the same accounting extends to a heterogeneous set of different models. We empirically verify our theory and show that, at the predicted group size, our method outperforms strong baselines under the tightest calibration spend, where debate-calibrated rules can afford only one or two questions. Self-refinement obeys the same accounting, with its own draft in place of a peer message. The theory further predicts that communication pays only when the gain in the accuracy ceiling exceeds the reasoning room spent on reading, with this condition tightening as the window shrinks. Across five open models and four reasoning tasks, self-consistency matches or beats deployed debate at $93$ % of $980$ matched window-and-group-size settings, with the gap being largest under tight windows and nearly disappearing at a loose one, consistent with the theory. Under a shared window, how much room a model has left to think matters more than what it reads.

Figures & tables

Figure 1: Overview of our method (Qwen2.5-7B, GSM8K). One window

W

pays for both reading (question plus peer messages) and thinking (generation

g

), and each extra agent adds

m

tokens of reading. The per-agent edge

\delta

is the gap between the probability of the correct answer and that of the strongest wrong answer: larger

\delta

means a stronger correct-answer margin, while

\delta\leq 0

means that a wrong answer is at least as likely. Left : The dashed line marks the smallest thinking room at which

\delta>0

; a group that reads past it (here

n{=}8

) leaves too few tokens for reasoning and its accuracy collapses. Right : The effective sample size

n_{\mathrm{eff}}

measures the independent-vote equivalent of

n

correlated agents, while

\delta

falls as thinking room shrinks. Their product

S

, the synergy index, therefore captures the trade-off between per-agent accuracy and voting amplification and peaks at

n^{\ast}=4.9

. The shaded band marks

\delta{\leq}0

, where votes amplify a wrong majority and extra agents cannot help.

Form	$\phi(g)$
Hill ( Hill, 1910 )	$\dfrac{g^{\kappa}}{g^{\kappa}+g_{0}^{\kappa}}$
Burr XII ( Burr, 1942 )	$1-\left(1+\left(\frac{g}{g_{0}}\right)^{\kappa}\right)^{-\lambda}$
Weibull ( Weibull, 1951 )	$1-e^{-\left(\frac{g}{g_{0}}\right)^{\kappa}}$
Power law ( Pareto, 1964 )	$1-(1+\frac{g}{g_{0}})^{-\kappa}$
Monotone ( Robertson et al., 1988 )	isotonic step

Table 1: Candidate forms for the thinking gain

\phi

, adopted form in bold. Right: all five fitted to one measured cap sweep (Qwen2.5-7B/GSM8K).

Debate-calibrated
Method	1.0M budget					2.0M budget					Avg.
Method	L-3B	Q-7B	F-7B	G2-9B	G3-12B	L-3B	Q-7B	F-7B	G2-9B	G3-12B	Avg.
Successive Halving	7.13	12.05	9.18	5.63	11.39	5.52	4.93	6.39	3.38	8.84	7.44
Empirical Argmax	4.88	3.17	3.37	3.19	1.41	4.48	2.38	2.77	3.05	1.62	3.03
Quadratic Surface	4.20	2.46	2.43	2.98	2.27	4.25	2.24	2.95	2.07	0.91	2.68
No calibration

Table 2: Held-out regret against the per-cell oracle (percentage points) when each method chooses

n

at an equal token spend. Bold is the lowest regret in a column and underline the second lowest, the Avg. column included. Per-cell values for all calibration spend in Appendix Tables 24 – 25 .

Figure 2: Two ways to fix a calibration budget, equal questions (left and middle) and equal spend (right, dotted verticals at the two budgets of Table 2 ), and two readings of the equal-questions sweep: cost-adjusted (left) and plain regret (middle).

Appendix figures & tables38 assets

Supplementary material from the paper’s appendix.

Appendix

Benchmark	Task	Sample size	HuggingFace path
GSM8K	Grade-school math (open)	500	openai/gsm8k
MATH500	Competition math (open)	500	HuggingFaceH4/MATH-500
MMLU-Pro	Multi-domain QA (10-way)	500	TIGER-Lab/MMLU-Pro
CSQA	Commonsense QA (5-way)	500	tau/commonsense_qa

Appendix

Table 3: Benchmark details. All benchmarks use 500 samples, including the first 40% used for calibration and the remaining 60% for evaluation.

Parameter	Value
Window $W$ (tokens)	$\{1.2,1.6,2.0,2.5,3.2,4.5,6.5\}$ k
Agents $n$	$\{1,\dots,8\}$
Question length $q$ (mean, tokens)	GSM8K $128$ , MATH500 $146$ , MMLU-Pro $258$ , CSQA $105$
Per-message length $m$	$320$
Overhead $c$	$64$
Revision rounds $R$	$1$ (so $R{+}1{=}2$ generation passes)

Appendix

Table 4: Operating grid. Generation room is the leftover

g=\mathrm{clip}(W-H-c,g_{\min},g_{\mathrm{cap}})

, so a larger group at fixed

W

thinks less. The probe sweep runs to 4096. Deployment clips

g

g_{\mathrm{cap}}=2048

, and the out-of-range test in Table 9 fits on caps up to

2048

and predicts at

4096

. Table 17 ’s long-CoT comparison raises

g_{\mathrm{cap}}

8192

to give the single agent every advantage.

Model	Task	$p_{\max}$	95% CI	$g_{0}$	$\kappa$	RMSE	LOO	$R^{2}$
Qwen2.5-7B	GSM8K	87.4	[86.8, 88.0]	256	6.9	0.17	1.89	1.000
	MATH500	67.6	[66.5, 68.9]	433	3.7	0.21	0.27	1.000
	MMLU-Pro	49.9	[48.8, 51.1]	437	3.7	0.37	1.88	1.000
	CSQA	80.5	[79.7, 81.4]	246	9.8	0.09	8.32	1.000
Falcon3-7B	GSM8K	81.0	[80.4, 81.7]	244	4.7	0.37	2.43	1.000
	MATH500	61.0	[59.7, 62.4]	475	3.5	0.49	1.68	1.000

Appendix

Table 5: Single-agent Hill fits on the cap sweep, all four tasks.

p_{\max}

, its 95% bootstrap CI (resampling sweep records,

500

refits), RMSE and LOO are on the accuracy scale, reported as percentages; LOO is the held-out RMSE, leaving out one swept cap at a time.

	GSM8K			MMLU-Pro
Model	$\hat{g}_{0}$	median len.	ratio	$\hat{g}_{0}$	median len.	ratio
Qwen2.5-7B	256	264	0.97	437	490	0.89
Falcon3-7B	244	254	0.96	414	484	0.85
Mean ratio	0.96			0.87

Appendix

Table 6: Fitted Hill midpoint

\hat{g}_{0}

against the measured median completion length, per model and task. Median lengths are read off the largest generation cap (

4096

tokens), where under

1\%

of samples are truncated. ratio is

\hat{g}_{0}

over the median.

		s.d. over the three per-seed fits			s.d. of $n^{\ast}$	regret (pp)
Model	Task	$\hat{p}_{\max}$	$\hat{g}_{0}$	$\hat{\kappa}$	mean	mean	s.d.
Qwen2.5-7B	GSM8K	0.02	0.1	0.30	0.026	2.01	0.00
	MATH500	0.31	1.1	0.05	0.020	0.78	0.00
	MMLU-Pro	0.22	3.7	0.05	0.023	1.09	0.00
	CSQA	0.07	1.7	1.66	0.064	0.70	0.00
Falcon3-7B	GSM8K	0.22	0.7	0.07	0.024	0.20	0.00

Appendix

Table 7: Seed-to-seed spread of the ability fit, of the resulting

n^{\ast}

, and of the regret that spread costs, as standard deviations over three per-seed refits.

\hat{p}_{\max}

and regret are in percentage points,

\hat{g}_{0}

in tokens,

\hat{\kappa}

and

n^{\ast}

dimensionless. Each per-seed fit uses that seed’s full thinking sweep and its regret is scored against the per-cell oracle on the full bank. Mean regret is scored from a fit refit on a single held-out seed at a time, not the pooled (all-seed) fit the rest of the paper uses, so it is not directly comparable to the pooled-fit regret in Table 21 ; the near-zero s.d. shows that which seed is dropped does not matter, not that single-seed refitting matches the pooled result.

Model	Task	1.2	1.6	2.0	2.5	3.2	4.5	6.5	range W	range n
		$\hat{\rho}$ at $W$ ( $\times 10^{3}$ tokens)
Qwen2.5-7B	GSM8K	0.36	0.35	0.36	0.35	0.34	0.35	0.35	0.01	0.02
	MATH500	0.71	0.69	0.68	0.68	0.68	0.68	0.68	0.04	0.01
	MMLU-Pro	0.60	0.57	0.57	0.56	0.56	0.56	0.56	0.04	0.02
	CSQA	0.58	0.58	0.58	0.58	0.58	0.58	0.58	0.01	0.03
Falcon3-7B	GSM8K	0.41	0.41	0.40	0.40	0.41	0.41	0.40	0.01	0.01

Appendix

Table 8: Round-0 error correlation

\hat{\rho}

re-estimated at each window. range W and range n are

\max\hat{\rho}-\min\hat{\rho}

over the swept windows and over the group sizes.

Selection criterion	hill	weib	pow	burr	iso
Leave-one-out RMSE	3.61	5.21	11.74	4.49	10.02
Out-of-range RMSE	0.38	0.75	6.85	0.33	0.31
Total	3.99	5.96	18.59	4.82	10.33

Appendix

Table 9: Fitting performance of thinking gain

\phi

, over five models and four datasets, per cell in Table 5 , Appendix 9 . RMSE is on the accuracy scale in percent; leave-one-out holds out one swept cap at a time and out-of-range predicts at

4096

from caps up to

2048

. Adopted form in bold.

Model	probe	debate	probe	debate	probe	debate	probe	debate	ratio
	GSM8K		MATH500		CSQA		MMLU-Pro
Llama3.2-3B	6	21	11	34	6	38	11	71	5 $\times$
Qwen2.5-7B	8	25	13	43	7	55	13	93	5 $\times$
Falcon3-7B	8	31	17	54	6	57	14	105	6 $\times$
Gemma2-9B	7	29	13	62	6	68	13	147	8 $\times$
Gemma3-12B	12	62	34	146	10	171	31	336	9 $\times$

Appendix

Table 10: Calibration wall-clock in minutes,

200

questions. Probe is our single-agent sweep; debate (empirical argmax) is the full

(n,W)

grid. ratio is debate over probe per dataset, then averaged over the four.

Calibration rule	Input	Output
Empirical Argmax	$49\times$	$11\times$
Quadratic Surface	$49\times$	$11\times$
Successive Halving	$33\times$	$9\times$
Difficulty Mixture	$0.9\times$	$0.8\times$
Round-0 probe ( $\rho$ , $c_{w}$ )	$0.1\times$	$0.1\times$
Probe sweep (TB-MAD, learned $\beta_{\mathrm{eff}}$ , Star Exponent)	$1\times$	$1\times$

Appendix

Table 11: Input and output tokens each calibration rule spends, as a multiple of the single-agent probe sweep, summed over the twenty cells at the full

200

-question pool. The probe (except Difficulty Mixture) multiple is

1\times

by definition; debate re-reads every peer message, so its input multiple is the largest.

Debate-calibrated
Method	1.0M budget				2.0M budget				Avg.
Method	Q+G3	Q+L	F+G2	G3+L	Q+G3	Q+L	F+G2	G3+L	Avg.
Successive Halving	8.96	10.88	7.97	7.47	4.75	6.87	4.82	5.50	7.15
Quadratic Surface	2.61	7.01	2.68	2.63	4.60	6.62	2.79	4.05	4.13
Empirical Argmax	2.93	7.09	3.30	3.07	2.96	5.18	3.64	3.19	3.92
No calibration

Appendix

Table 12: Held-out regret against the per-cell oracle (percentage points) when each method chooses

n

on the heterogeneous model-set bank at a matched token budget, the model-set analogue of Table 2 with the same marks. Per-cell values in Tables 31 – 32 .

Debate-calibrated
Method	1.0M budget				2.0M budget				Avg.
Method	Q+G3+L	Q+F+G2	Q+F+G2+G3	Q+F+G3+L	Q+G3+L	Q+F+G2	Q+F+G2+G3	Q+F+G3+L	Avg.
Successive Halving	2.97	3.09	4.48	3.66	2.41	1.68	2.77	2.84	2.99
Quadratic Surface	2.58	2.41	1.81	1.64	1.90	1.81	1.91	1.71	1.97
Empirical Argmax	2.14	1.88	2.39	2.16	1.75	1.79	2.02	1.73	1.98
No calibration

Appendix

Table 13: Held-out regret (percentage points) on the balanced-

K

sets, two

K{=}3

sets on grid

\{3,6\}

and two

K{=}4

\{4,8\}

, at the budgets of Table 2 with the same marks. Each naive column is a fixed

n

, blank where a set’s grid omits it; a row spanning one

K

gets no average. Per-cell values in Tables 29 – 30 .

Figure 3: Self-refinement accuracy against the first draft (

\Delta{=}A_{R}-A_{1}

, in points) as the window grows,

R\in\{1,2,3\}

;

R{=}1

is the base generation without refinement. The dotted vertical line marks the saturation window: from there on, both

R{=}2

and

R{=}3

stay within

7\%

of the first draft.

Model	$A_{1}$	gain ∗	$R^{\ast}$	drop	$A_{1}$	gain ∗	$R^{\ast}$	drop	$A_{1}$	gain ∗	$R^{\ast}$	drop	$A_{1}$	gain ∗	$R^{\ast}$	drop
	GSM8K				MATH500				MMLU-Pro				CSQA
Qwen2.5-7B	87.6	0.0	1	18.3	64.7	1.3	1	44.3	47.0	2.5	1	40.0	81.4	1.1	1	4.7
Falcon3-7B	81.3	4.3	2	10.6	55.1	3.5	1	31.3	48.3	6.7	3	23.5	74.1	0.7	2	1.3
Gemma-2-9B	81.7	2.0	2	1.6	38.2	3.3	2	3.8	50.5	3.2	1	13.7	77.1	0.0	1	1.3
Gemma-3-12B	92.7	0.9	1	7.5	65.7	2.3	1	27.1	54.3	2.9	2	33.3	80.5	0.2	1	2.3
Llama-3.2-3B	69.9	0.0	1	28.8	33.7	0.0	1	15.3	27.3	0.0	1	20.3	66.7	0.0	1	14.1

Appendix

Table 14: Self-refine over

R\in\{1,2,3\}

rounds across settings.

A_{1}

: first-draft accuracy; gain ∗ : oracle (grid-best) refine gain over

A_{1}

;

R^{\ast}

: the round most often best across the swept windows; drop:

A_{1}

minus the

R{=}3

accuracy at the smallest window.

Qwen2.5-7B	GSM8K	86.7	84.7	-1.97
		No peer	With peer	$\widehat{\Delta p_{\max}}$
Model	Task	(round 1)	(round 1)
Qwen2.5-7B	MMLU-Pro	49.8	46.0	-3.78
Falcon3-7B	GSM8K	80.6	82.9	+2.33
Falcon3-7B	MMLU-Pro	46.1	50.2	+4.11

Appendix

Table 15: Per-agent round-

1

accuracy (%) with no peer read ( No peer ,

n{=}1

) against with one peer read ( With peer ,

n{=}2

), fixed

g{=}2048

for both, averaged over

3

seeds.

\widehat{\Delta p_{\max}}

(%) is the difference.

$W$	$\hat{n}_{\mathrm{deb}}$	debate	$\hat{n}_{\mathrm{SC}}$	self-cons.	margin	matched $n$
6k	13	93.67	13	94.07	$-0.40\pm 1.11$	$-0.40$
9k	13	93.80	13	94.07	$-0.27\pm 0.46$	$-0.27$
12k	13	94.00	21	94.07	$-0.07\pm 0.81$	$+0.00$
16k	21	94.33	13	93.67	$+0.67\pm 1.22$	$+0.73$
24k	13	94.07	21	94.00	$+0.07\pm 1.47$	$+0.13$
32k	21	93.87	21	93.67	$+0.20\pm 0.53$	$+0.20$

Appendix

Table 16: Qwen2.5-7B/GSM8K, self-consistency against debate at large

W

, each method read at its own best group size

\hat{n}

over the extended grid

n\in\{1,2,4,8,13,21,32\}

(columns 2–5). margin is debate minus self-consistency (

\pm

seed s.d. over

3

seeds), and matched

n

is the same gap when self-consistency is instead read at debate’s

\hat{n}

. Every margin lies inside its own seed s.d., so the two methods are indistinguishable from

W{=}6\mathrm{k}

onward.

Model	Task	Long-CoT $A$	gen. tokens	Debate $A$	Debate $-$ long-CoT
Qwen2.5-7B	GSM8K	88.9	276	93.7	$\mathbf{+4.9}$
Qwen2.5-7B	MMLU-Pro	49.5	533	49.8	$\mathbf{+0.2}$
Falcon3-7B	GSM8K	81.5	274	91.4	$\mathbf{+9.9}$
Falcon3-7B	MMLU-Pro	46.5	540	55.3	$\mathbf{+8.9}$

Appendix

Table 17: Single-agent long-CoT (

n{=}1

, room grown to

g_{\mathrm{cap}}{=}8192

) against the debate group (

n{\geq}2

), matched at the tokens debate spends at its own best cell. Gen. tokens instead gives long-CoT’s own best window, where more thinking stops improving its accuracy.

		theory			averaged over the grid		at the oracle $n^{\ast}(W)$		group vs. single agent			SC vs. debate
Model	Task	$p_{\max}$	$\bar{\delta}$	$\bar{A}_{1}$	$\bar{A}_{\text{SC}}$	$\bar{A}_{\text{deb}}$	$A_{\text{SC}}$	$A_{\text{deb}}$	$\delta\leq 0$ share	single $\geq$ group	predicted	SC $\geq$ deb	SC sig. $W$
Llama3.2-3B	GSM8K	69.0	+0.23	67.7	80.3	69.1	82.0	80.7	16/49	16/49	80	49/49	4/7
	MATH500	34.4	-0.05	34.1	42.0	32.9	42.1	41.5	21/49	22/49	86	36/49	4/7
	MMLU-Pro	32.8	-0.23	31.8	37.1	27.9	37.0	35.7	23/49	22/49	47	42/49	4/7
	CSQA	67.8	+0.11	65.1	69.9	60.1	71.7	70.2	16/49	15/49	78	49/49	3/7
Qwen2.5-7B	GSM8K	87.4	+0.37	84.7	92.2	73.5	92.5	92.3	15/49	15/49	96	48/49	2/7

Appendix

Table 18: Accuracy (%) over the swept windows with

n\geq 2

. Barred columns average over the grid, unbarred read both methods at debate’s size

n^{\ast}(W)

\delta\leq 0

share compares each cell’s debate accuracy against the single-agent accuracy pooled over all windows; single

\geq

group is the stricter, per-window version of the same comparison, the observed share of the

49

cells with

A(1,W)\geq A(n,W)

at matching

W

; predicted is the fraction of the

\delta\leq 0

-share signs the closed form recovers. SC sig.

W

is how many of the

7

swept windows have SC significantly ahead (paired bootstrap) at a majority of the

7

group sizes.

Figure 4: Debate (red, solid) against self-consistency (blue, dashed) as the window grows, one panel per task (Qwen2.5-7B). Marker size is the group size

n\in\{2,8\}

. The gap closes as the window grows on every panel (Prop. 2 ).

Figure 5: Same as Figure 4 , on Falcon3-7B: the gap closes with the window on every task there too.

Figure 6: Accuracy against cost per question. Lines are the Pareto frontier (left axis) and steps are the cost-optimal group

n^{\ast}(B)

(right axis), under symmetric pricing and under cheaper input at API rates.

Method	$c_{\mathrm{out}}/c_{\mathrm{in}}=1.0$					$c_{\mathrm{out}}/c_{\mathrm{in}}=2.0$					$c_{\mathrm{out}}/c_{\mathrm{in}}=2.5$					$c_{\mathrm{out}}/c_{\mathrm{in}}=4.0$
Budget (M)	1	2	4	8	16	1	2	4	8	16	1	2	4	8	16	1	2	4	8	16
Empirical Argmax	3.20	2.86	2.18	1.67	1.35	3.11	2.51	1.83	1.42	1.08	2.91	2.45	1.89	1.45	1.10	2.87	1.88	1.57	1.25	0.97
Successive Halving	9.08	5.81	3.04	1.65	1.12	6.54	4.56	1.83	1.26	1.00	6.31	4.45	1.73	1.24	0.92	5.55	2.01	1.39	1.10	0.95
Quadratic Surface	2.87	2.48	2.25	2.31	2.29	2.55	2.34	2.22	2.25	2.31	2.35	2.32	2.18	2.27	2.36	2.50	2.26	2.45	2.32	2.27
Learned $\beta_{\mathrm{eff}}$	1.68	1.49	1.56	1.52	1.52	1.65	1.78	1.50	1.52	1.52	1.63	1.80	1.50	1.52	1.52	1.76	1.70	1.52	1.52	1.52
Star Exponent	1.95	1.96	1.88	1.87	1.87	2.05	1.93	1.82	1.87	1.87	1.95	2.08	1.89	1.87	1.87	1.93	2.03	1.87	1.87	1.87

Appendix

Table 19: Held-out regret (percentage points) at four input/output price ratios and every swept budget. Bold is the column minimum, underline indicates an entry fixed at its large-budget value as the whole question is already used for calibration. First group:

c_{\mathrm{out}}/c_{\mathrm{in}}{=}1

, i.e. Table 2 .

Figure 7: Predicted

n^{\ast}(W)

(red) against empirical

\hat{n}^{\ast}(W)

(green) at 1M calibration tokens.

	Debate-calibrated			No calibration		Probe-calibrated
$W$	Halv.	Emp.	Surf.	$n_{\max}$	$n{=}1$	Mix	Star	Learn	TB-MAD
1200	1.03	0.82	2.38	58.82	1.56	1.65	2.16	1.91	1.12
1600	1.16	0.98	2.44	54.23	2.58	1.34	2.27	1.97	0.96
2000	1.09	1.17	2.80	42.72	3.81	1.89	2.59	1.34	1.49
2500	0.59	1.04	3.18	25.57	4.16	3.20	2.22	1.30	1.75
3200	0.62	1.12	3.32	2.93	5.36	3.93	1.59	0.72	0.87

Appendix

Table 20: Held-out regret (percentage points) by calibration window

W

, each method’s own chosen

n

averaged over all five models and four tasks. Complements Table 2 , which aggregates over

W

instead. Bold is the lowest regret in a row.

Figure 8: Synergy gap

A(n,W)-A(1,W)

against

n

under the shared window

W

of the standard debate protocol. Stars mark the empirical best size

\hat{n}^{\ast}(W)

, measured directly on the grid; open circles mark

n^{\ast}(W)

predicted from single-agent probes alone, with no debate data.

		$\hat{c}_{w}$ by samples per question			$\hat{c}_{w}$ range		$n^{\ast}$	regret (pp)
Model	Task	$1$	$8$	$48$	over $g$	over $n$	moved	$c_{w}$ fixed	$c_{w}(n)$
Llama3.2-3B	GSM8K	0.00	0.00	0.14	0.00–0.34	0.29–0.33	0/7	0.28	0.28
	MATH500	0.41	0.27	0.20	0.24–0.28	0.24–0.28	4/7	2.71	0.79
	MMLU-Pro	0.03	0.02	0.21	0.02–0.44	0.42–0.45	6/7	4.40	0.65
	CSQA	0.00	0.00	0.32	0.00–0.77	0.72–0.77	0/7	0.52	0.52
Qwen2.5-7B	GSM8K	0.05	0.05	0.15	0.05–0.49	0.35–0.43	0/7	0.50	0.50

Appendix

Table 21: Range of

c_{w}

over grid of

n

and

g

. The first three columns are the same pooled estimate at

1

8

and

48

samples per question. The last two columns are held-out regret against the per-cell oracle when

n^{\ast}

is chosen with the fixed cell-level

c_{w}

against the per-

n

estimate

c_{w}(n)

Model	Task	Moved	$\hat{n}^{\ast}_{\mathrm{derived}}-\hat{n}^{\ast}_{\mathrm{const}}$ by $W$
			1200	1600	2000	2500	3200	4500	6500
Llama3.2-3B	GSM8K	0/7	0	0	0	0	0	0	0
	MATH500	1/7	0	0	0	+1	0	0	0
	MMLU-Pro	0/7	0	0	0	0	0	0	0
	CSQA	0/7	0	0	0	0	0	0	0
Qwen2.5-7B	GSM8K	1/7	0	-1	0	0	0	0	0

Appendix

Table 22: Where the arg max of Algorithm 1 moves when the fitted constant

\sigma

is replaced by the derived

\sigma(n)

, per model-task cell across the

7

swept

W

(columns, in tokens). Moved counts the windows where

\hat{n}^{\ast}

differs; each numbered column gives the signed shift at that

W

(

\hat{n}^{\ast}_{\mathrm{derived}}-\hat{n}^{\ast}_{\mathrm{const}}

0

where the two choices of

\sigma

agree.

Figure 9: Predicted

n^{\ast}(W)

from the closed form against empirical argmax and the true held-out best (oracle), all matched to a

1

M-token calibration budget.

Figure 10: Accuracy against the generation floor

g_{\min}

at the worst cell (

n{=}8,W{=}1200

), Qwen2.5-7B and Falcon3-7B on GSM8K and MMLU-Pro.

		grid mean			collapse cell $(n{=}8,W{=}1200)$
Model	Task	$A_{\mathrm{qf}}$	$A_{\mathrm{qb}}$	$\Delta$	$A_{\mathrm{qf}}$	$A_{\mathrm{qb}}$	$\Delta$
Falcon3-7B	GSM8K	79.2	77.9	+1.24	9.0	0.2	+8.80
Falcon3-7B	MMLU-Pro	42.1	41.6	+0.48	6.9	1.3	+5.67
Qwen2.5-7B	GSM8K	78.0	78.2	-0.24	0.0	0.1	-0.07
Qwen2.5-7B	MMLU-Pro	37.4	36.9	+0.50	0.7	0.5	+0.20
Average		59.2	58.7	+0.49	4.2	0.5	+3.65

Appendix

Table 23: Effect of question position: question first ( qf ) versus buried after the peer messages ( qb ).

\Delta{=}A_{\mathrm{qf}}-A_{\mathrm{qb}}

(%).

Model	Dataset	QS	EA	SH	$n{=}n_{\max}$	$n{=}1$	DM	SE	LA	TB	Oracle
Llama3.2-3B	GSM8K	5.20	7.04	5.84	19.22	13.08	7.81	5.70	2.26	0.41	79.34
	MATH500	2.76	5.57	10.13	15.38	8.23	8.23	3.31	3.93	3.48	43.43
	MMLU-Pro	4.52	3.33	6.21	14.74	4.42	4.43	3.71	3.60	4.42	37.22
	CSQA	4.35	3.59	6.32	22.09	5.80	2.77	2.56	1.08	0.87	69.87
Qwen2.5-7B	GSM8K	4.14	6.57	11.20	39.78	7.83	0.91	1.81	1.40	2.30	92.10
	MATH500	3.97	3.45	20.37	38.52	2.89	2.89	1.28	1.38	1.12	70.46

Appendix

Table 24: Every cell behind Table 2 at the

1.0

M budget: held-out regret (percentage points) against the per-cell Oracle in the last column. Columns are SH successive halving, EA empirical argmax, QS quadratic surface, LA learned ability, SE star exponent, DM difficulty mixture, TB ours.

Model	Dataset	QS	EA	SH	$n{=}n_{\max}$	$n{=}1$	DM	SE	LA	TB	Oracle
Llama3.2-3B	GSM8K	5.04	4.26	3.40	19.22	13.08	5.15	5.70	1.80	0.41	79.34
	MATH500	4.83	6.22	6.31	15.38	8.23	8.23	3.54	2.69	1.80	43.43
	MMLU-Pro	2.58	3.40	9.52	14.74	4.42	4.42	4.42	4.05	4.42	37.22
	CSQA	4.54	4.04	2.84	22.09	5.80	3.67	2.50	1.03	0.87	69.87
Qwen2.5-7B	GSM8K	4.81	4.03	1.44	39.78	7.83	1.06	1.80	1.39	0.46	92.10
	MATH500	1.95	2.34	6.05	38.52	2.89	2.89	1.28	1.31	1.10	70.46

Appendix

Table 25: Every cell behind Table 2 at the

2.0

M budget. Columns and entries as in Table 24 .

Model	Dataset	QS	EA	SH	$n{=}n_{\max}$	$n{=}1$	DM	SE	LA	TB	Oracle
Llama3.2-3B	GSM8K	3.22	4.34	2.16	19.22	13.08	4.30	5.70	1.59	0.41	79.34
	MATH500	3.93	1.85	3.50	15.38	8.23	8.23	3.20	3.04	1.92	43.43
	MMLU-Pro	2.60	2.17	2.15	14.74	4.42	4.42	4.42	4.32	4.42	37.22
	CSQA	3.51	3.36	1.76	22.09	5.80	2.77	2.49	0.96	0.87	69.87
Qwen2.5-7B	GSM8K	4.57	2.69	2.43	39.78	7.83	0.99	1.80	1.98	0.43	92.10
	MATH500	2.10	2.51	1.77	38.52	2.89	2.89	1.28	1.51	1.09	70.46

Appendix

Table 26: Every cell behind Table 19 at the

4.0

M budget. Columns and entries as in Table 24 .

Model	Dataset	QS	EA	SH	$n{=}n_{\max}$	$n{=}1$	DM	SE	LA	TB	Oracle
Llama3.2-3B	GSM8K	5.11	2.08	2.49	19.22	13.08	4.30	5.70	1.59	0.41	79.34
	MATH500	3.77	2.74	2.76	15.38	8.23	8.23	3.20	3.04	2.62	43.43
	MMLU-Pro	2.74	1.99	3.65	14.74	4.42	4.42	4.42	4.32	4.42	37.22
	CSQA	4.80	2.80	2.42	22.09	5.80	2.77	2.49	0.96	0.87	69.87
Qwen2.5-7B	GSM8K	4.92	1.80	1.09	39.78	7.83	0.99	1.80	1.98	0.43	92.10
	MATH500	2.08	1.76	1.03	38.52	2.89	2.89	1.28	1.51	1.10	70.46

Appendix

Table 27: Every cell behind Table 19 at the

8.0

M budget. Columns and entries as in Table 24 .

Model	Dataset	QS	EA	SH	$n{=}n_{\max}$	$n{=}1$	DM	SE	LA	TB	Oracle
Llama3.2-3B	GSM8K	3.68	1.13	0.75	19.22	13.08	4.30	5.70	1.59	0.41	79.34
	MATH500	4.34	1.84	1.41	15.38	8.23	8.23	3.20	3.04	2.62	43.43
	MMLU-Pro	2.35	1.08	1.67	14.74	4.42	4.42	4.42	4.32	4.42	37.22
	CSQA	4.21	1.86	1.53	22.09	5.80	2.77	2.49	0.96	0.87	69.87
Qwen2.5-7B	GSM8K	4.74	1.37	0.94	39.78	7.83	0.99	1.80	1.98	0.43	92.10
	MATH500	2.17	1.39	0.99	38.52	2.89	2.89	1.28	1.51	1.10	70.46

Appendix

Table 28: Every cell behind Table 19 at the

16.0

M budget. Columns and entries as in Table 24 .

Model set	Dataset	SH	QS	EA	$n{=}8$	$n{=}6$	$n{=}3$	$n{=}4$	TB w/o gate	DM	SE	LA	TB
Qwen2.5-7B+Gemma3-12B+Llama3.2-3B	GSM8K	0.47	0.59	0.80	–	15.18	0.93	–	1.50	0.17	0.42	0.13	0.13
	MATH500	3.65	2.52	3.16	–	19.65	3.54	–	1.98	3.54	2.10	1.27	1.03
	MMLU-Pro	6.61	6.31	3.48	–	11.78	5.59	–	3.39	5.59	3.42	3.02	3.39
	CSQA	1.15	0.91	1.13	–	11.99	1.47	–	8.19	1.10	0.62	0.50	0.50
Qwen2.5-7B+Falcon3-7B+Gemma2-9B	GSM8K	2.02	0.79	0.62	–	13.06	1.17	–	1.63	0.00	0.40	0.02	0.02
	MATH500	5.10	3.98	4.22	–	20.73	4.14	–	1.91	4.13	3.51	3.38	2.00

Appendix

Table 29: Every cell behind Table 13 at the

1.0

M budget: held-out regret (percentage points) against the widened per-cell oracle (set-best or any single member). Abbreviations as in Table 24 ,

n{=}3

–

8

the fixed-size rules.

Model set	Dataset	SH	QS	EA	$n{=}8$	$n{=}6$	$n{=}3$	$n{=}4$	TB w/o gate	DM	SE	LA	TB
Qwen2.5-7B+Gemma3-12B+Llama3.2-3B	GSM8K	1.03	0.59	0.64	–	15.18	0.93	–	1.01	0.24	0.42	0.13	0.17
	MATH500	2.17	2.65	2.11	–	19.65	3.54	–	1.48	3.54	2.10	1.27	0.83
	MMLU-Pro	6.03	3.42	3.66	–	11.78	5.59	–	1.38	4.91	3.42	2.99	1.38
	CSQA	0.39	0.96	0.58	–	11.99	1.47	–	3.79	0.74	0.62	0.50	1.60
Qwen2.5-7B+Falcon3-7B+Gemma2-9B	GSM8K	0.56	0.83	0.87	–	13.06	1.17	–	1.89	0.22	0.40	0.02	0.02
	MATH500	3.98	3.94	3.52	–	20.73	4.14	–	0.83	4.14	3.51	3.43	2.43

Appendix

Table 30: Every cell behind Table 13 at the

2.0

M budget. Columns and entries as in Table 29 .

Model pair	Dataset	SH	QS	EA	$n{=}n_{\max}$	$n{=}2$	DM	SE	LA	TB w/o gate	TB
Qwen2.5-7B+Gemma3-12B	GSM8K	6.34	1.78	2.34	32.07	2.34	0.26	0.80	0.61	0.47	0.51
	MATH500	12.75	2.80	3.46	36.14	3.99	3.17	0.84	0.80	2.28	0.84
	MMLU-Pro	10.71	3.47	3.26	28.72	5.65	4.71	4.36	3.85	1.83	1.75
	CSQA	6.05	2.37	2.67	26.34	2.67	1.64	1.69	0.97	0.77	0.77
Qwen2.5-7B+Llama3.2-3B	GSM8K	7.42	4.82	5.04	31.07	6.82	1.34	2.17	1.09	0.37	1.03
	MATH500	15.36	8.67	9.42	33.24	7.74	6.67	4.96	4.59	0.27	1.77

Appendix

Table 31: Every cell behind Table 12 at the

1.0

M budget: held-out regret (percentage points) against the widened per-cell oracle (set-best or any single member). Abbreviations as in Table 24 .

Model pair	Dataset	SH	QS	EA	$n{=}n_{\max}$	$n{=}2$	DM	SE	LA	TB w/o gate	TB
Qwen2.5-7B+Gemma3-12B	GSM8K	2.88	1.58	2.05	32.07	2.34	0.54	0.89	0.56	1.19	0.51
	MATH500	5.57	2.87	3.23	36.14	3.99	3.99	0.84	0.95	2.89	0.61
	MMLU-Pro	8.15	11.57	4.28	28.72	5.65	4.91	1.82	2.15	1.06	1.33
	CSQA	2.40	2.40	2.26	26.34	2.67	1.78	1.69	0.86	0.77	0.77
Qwen2.5-7B+Llama3.2-3B	GSM8K	3.14	3.77	3.80	31.07	6.82	1.61	1.61	1.03	0.60	1.16
	MATH500	12.61	6.69	6.83	33.24	7.74	7.51	4.96	4.51	0.22	1.42

Appendix

Table 32: Every cell behind Table 12 at the

2.0

M budget. Columns and entries as in Table 31 .

Efficient Language Model Reasoning

Efficient language model reasoning studies how to improve problem-solving accuracy while reducing inference, memory, latency, and training costs.

See all 369 papers

May 8, 2026

The Coupling Tax: How Shared Token Budgets Undermine Visible Chain-of-Thought Under Fixed Output Limits

Chain-of-thought reasoning is often treated as a monotone way to improve language-model accuracy by letting a model think longer. We identify a countervailing effect, the coupling tax: when reasoning traces and final answers share one output-token budget, long traces can crowd out the answer they are meant to support. Across GSM8K, MATH-500, and five BIG-Bench Hard tasks with Qwen3 models at three scales, non-thinking mode matches or outperforms thinking mode on GSM8K and MATH-500 at every budget up to 2048 tokens, while harder tasks shift the crossover to larger budgets. We derive a truncation-waste decomposition,

\mathrm{Acc}_{\mathrm{think}}(b)=α_c F_L(b)+α_t(1-F_L(b))

, that predicts this crossover from chain-length and accuracy statistics and explains inverse scaling within the Qwen family. A DeepSeek-R1-Distill-Llama-8B replication shows the same pattern under a different thinking interface. As a mitigation, split-budget generation decouples reasoning and answer budgets; on full MATH-500, IRIS reaches 74.0% accuracy, a strengthened extraction variant reaches 78.8%, and a fixed non-oracle SC+IRIS gate reaches 83.6%. The results show that test-time reasoning should be evaluated as a budget-allocation problem, not only as a question of whether longer traces are available.

Wenhua Nie, Junlin Liu, Jianan Wu +5

National Taiwan University

CoT Evaluation CoT Reasoning Efficient Language Model Reasoning Reasoning Trace Analysis

Ask PDF

Aug 27, 2026

Thinking Costs Tokens: When More Structure is Worth the Price

Adding inference structure to a language model lets it search, verify, and revise, but these actions consume the very budget they are supposed to use well. In this paper, we investigate whether there exists a token-budget threshold, below which the overhead of planning and verification hurts performance and above which it helps. We evaluate two systems on FinQA and TAT-QA financial reasoning tasks, using GPT-5.4 mini across 14 budget tiers ranging from 250 to 42,000 output-equivalent tokens. The first system is a monolith, which is a single LLM call. The second is a verified search architecture that adds planning, label-blind checking, and repair capabilities. We run 1,000 cases for a total of 28,000 completed cells. Both systems score 0% at the two lowest tiers, where neither can fit a complete prompt. At 1,000 tokens, the monolith reaches 18% accuracy while verified search scores near 0%, since the planning overhead leaves no room for an answer. From 1,500 tokens onward, verified search surpasses the monolith and maintains a consistent advantage, reaching approximately 44% at the highest tiers while the monolith reaches approximately 40%. The crossover occurs between 1,000 and 1,500 output-equivalent tokens, confirmed by a strict intersection-union test (

p \le 0.001

at both endpoints).

Thomas Nolasque, John Grey, Calista Pham +1

Royal Bank of Canada

Efficient Language Model Reasoning Test-Time Scaling Numerical Reasoning in Language Models

Ask PDF

Sep 26, 2026

Beyond Symmetric Agents: Cognitive Diversity and Multi-Agent Debate in Small Language Models

Multi-agent debate (MAD) reportedly improves reasoning and factuality over single-model inference, but prior work treats agents as symmetric peers, leaving open what drives the gains. We test the hypothesis that cognitive diversity among agents is the driver, in the setting where the question is still measurable: small open-weight models with benchmark headroom. Across 23 models from eleven vendor families, five tasks, and 5,500+ debate and control runs, we vary diversity along three axes - personas, sampling temperature, and model identity - pairing every debate configuration with a generation-budget-matched majority-vote control. The hypothesis is rejected on every axis. Debate beats single-agent inference (3--7 points where tasks have headroom) but at matched budget conditions it ties or even loses to self-consistency sampling at 1.6

\times

the wall-clock and 3.4

\times

the token cost. Persona prompting reduces accuracy and a dose-response experiment over each model's full combinatorial persona space shows the cost is a persona tax, not a diversity tax: redundant personas hurt most, while maximally-diverse teams recover part of the loss. Furthermore, mixed-model teams lose to majority votes over their own rosters, with accuracy tracking member capability rather than heterogeneity, and nearly all of debate's benefit comes from the first exchange of answers. We further identify a pervasive measurement hazard in which debate transcripts silently overflow serving context windows, whose correction alone moves our debate-versus-sampling comparison from

-1.8

points to parity. Our results recast reported MAD gains as an ensemble-sampling effect and provide the budget-matched, contamination-checked baseline bar that future debate mechanisms should be required to clear.

Leonardo Ferreira, Gardenia Liu, Kaden Zheng

Boston Children’s Hospital, Harvard Medical School · Harvard School of Engineering and Applied Sciences, Harvard College

Multi-Agent Debate Multi-Agent Systems Language Model Ensembles

Ask PDF

Abstract

Efficient Language Model Reasoning

Explore similar work

The Coupling Tax: How Shared Token Budgets Undermine Visible Chain-of-Thought Under Fixed Output Limits

Thinking Costs Tokens: When More Structure is Worth the Price

Beyond Symmetric Agents: Cognitive Diversity and Multi-Agent Debate in Small Language Models