A cumulative token cap can fall inside a mathematical derivation, forcing a test-time controller to choose between stopping at the cap (strict) and allowing the current attempt to finish (advisory). We measure this boundary choice with paired offline replays of 19,200 public traces: 120 AIME, BrUMO and HMMT problems and two archive configurations of one model. Candidate order and a 16-attempt cap are fixed, and answer selection is blind to reference answers and correctness labels. Three findings emerge. First, at the 4k cap, most advisory accuracy gains replace abstention with a correct answer; strict stopping pays for an unfinished prefix that the completed-only selector cannot use. Second, comparisons along realized cost differ from same-cap comparisons: advisory 4k in low has higher accuracy than strict 8k at comparable mean completion cost, while in high its observed accuracy is 0.42 points below strict 32k using 59% of its mean tokens. These aggregate comparisons do not establish equal-compute superiority or accuracy equivalence. Third, increased candidate coverage does not guarantee higher answer accuracy: a log-probability selector loses accuracy while coverage rises, including after a source-grade consistency repair. Same-cap majority-accuracy differences shrink below 1.3 percentage points at 32k. Budget curves should jointly state the cap, realized cost, eligible candidates, stopping rule and selector information.
Figures & tables
Figure 1: Paired offline evaluation. A fixes the archived bank and order; B shows a boundary-crossing attempt under each completed-only rule; C applies the same eligibility and selection separately to each pool. Grades enter evaluation only; coverage is an oracle diagnostic. Exact boundaries or bank exhaustion need not overshoot. Timeline widths are illustrative, not to scale. Costs are completion tokens.
Accuracy (%)
Coverage (%)
Cost/ B
Archive
B
S
A
Δ [95% CI]
S
A
S
A
low
4k
37.25
50.13
12.88 [9.79, 16.17]
41.08
58.67
0.99
1.87
8k
46.96
52.79
5.83 [4.08, 7.79]
55.13
63.88
0.99
1.43
16k
52.83
55.50
2.67 [1.75, 3.67]
66.04
69.00
0.96
1.17
32k
57.00
57.63
0.63 [0.21, 1.08]
71.08
71.96
0.81
0.89
high
4k
18.54
61.58
43.04 [37.08, 49.17]
18.54
62.25
1.00
4.68
Table 1: Paired boundary audit: 120 problems, K=16 , 20 orders. S: strict; A: advisory. Accuracy and oracle coverage use archived grades. The paired accuracy change is in percentage points, with a 95% question-bootstrap interval. Cost is mean realized completion tokens divided by B ; these are not matched-compute comparisons. Bank exhaustion can give cost/ B<1 .
Appendix figures & tables9 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 2: Strict and advisory majority accuracy (top) and realized completion cost divided by the nominal cap B (bottom), with K=16 . Curves average 20 orders within each of 120 problems; shading gives problem-bootstrap 95% intervals for each curve, not the paired difference. Bottom-row intervals are also divided by B . Dotted lines at one mark the nominal cap; bank exhaustion can yield ratios below one.
Archive
Competition
No box
Length
Parser agreement
Median tokens
low
AIME24
132
122
2268
2924.0
low
AIME25
123
121
2277
3471.0
low
BrUMO25
16
0
2307
1180.0
low
HMMT25
34
1
2366
1307.5
high
AIME24
400
413
2000
6994.0
high
AIME25
508
543
1892
8880.0
Appendix
Table 2: Source-record audit, each row containing 2,400 attempts. “No box” includes records without a complete nonempty parsed final box; “Length” records remain costly but ineligible, even if a box appears earlier. The columns overlap and must not be added as disjoint exclusions. Agreement compares primary boxed normalization with archive extraction, with no correctness-based filtering.
Archive
B
Source grade
Repaired grade
Change in effect
low
4k
12.875
12.958
0.083
8k
5.833
5.917
0.083
16k
2.667
2.667
0.000
32k
0.625
0.625
0.000
high
4k
43.042
43.375
0.333
8k
28.208
28.458
0.250
Appendix
Table 3: Paired majority change (advisory minus strict, pp) under primary source grades and exploratory formatting-repaired grades. The repaired values are a robustness check, not replacements for the prespecified source-grade table.
Figure 3: Paired majority changes (advisory minus strict, percentage points) in each archive’s shorter and longer halves. Points average the 20 paired orders within each problem; intervals use the same stratified question-bootstrap procedure. Length groups differ in task and difficulty, so these are not causal effects of making a solution longer.
Mean tokens
Accuracy (%)
Archive
S cap
A
S
A
S
Δ pp [95% CI]
low
8k
7,485
7,940
50.13
46.96
3.17 [1.42, 5.04]
high
16k
18,729
16,000
61.58
47.21
14.38 [11.54, 17.50]
32k
18,729
31,867
61.58
62.00
−0.42 [ −1.17 , 0.25]
Appendix
Table 4: Existing budget points compared by mean realized completion cost. A always uses a nominal 4k cap; the S cap varies. Accuracy and the paired A minus S difference use archived grades. These are different-cost comparisons.
low
high
B
U (tokens)
f (%)
U (tokens)
f (%)
4k
1,585
39.83
3,436
85.90
8k
2,138
26.92
5,873
73.42
16k
2,875
18.80
9,879
61.74
32k
3,436
13.29
15,095
47.37
Appendix
Table 5: Strict cost assigned to unfinished terminal prefixes. U is mean charged prefix tokens; f divides total prefix tokens by total strict cost over 2,400 problem–order replays. It excludes fully returned but ineligible responses and is not a mean of per-replay fractions.
Archive
B
S eligible
A eligible
A one return (%)
A P95 tokens
P95/ B
low
4k
2.05
3.01
21.21
25,042
6.26
low
8k
4.28
5.21
11.17
29,654
3.71
low
16k
7.88
8.68
6.04
32,768
2.05
low
32k
11.44
11.86
2.29
46,855
1.46
high
4k
0.26
0.94
80.50
32,768
8.19
high
8k
0.68
1.34
64.29
32,768
4.10
Appendix
Table 6: Effective pools and advisory tail cost, K=16 . S and A denote strict and advisory. “One return” includes a single returned attempt even if it is length-capped or has no valid box. Eligible counts exclude such responses. P95 is the 95th percentile across 2,400 replay costs.
Archive
B
Boundary events
Examined
Unverified
Any box
Last box
low
4k
2,380
2,297
83
41
40
low
8k
2,347
2,252
95
45
45
low
16k
2,054
1,967
87
37
37
low
32k
1,185
1,124
61
24
24
high
4k
2,399
2,271
128
146
142
high
8k
2,400
2,264
136
249
248
Appendix
Table 7: Answer-format availability in reconstructed strict boundary prefixes. Counts are replay events and may reuse a trace across orders. Unverified = boundary events minus examined events. No correctness claim is made for prefix boxes.
Archive
B
K=8
K=16
K=32
Canonical
Order range
low
4k
12.875
12.875
12.875
15.000
[5.833, 17.500]
low
8k
5.792
5.833
5.833
8.333
[0.000, 10.000]
low
16k
2.708
2.667
2.667
5.833
[0.000, 5.833]
low
32k
0.542
0.625
0.667
1.667
[-0.833, 1.667]
high
4k
43.042
43.042
43.042
37.500
[39.167, 48.333]
high
8k
28.208
28.208
28.208
23.333
[22.500, 34.167]
Appendix
Table 8: Advisory minus strict majority accuracy (pp). The first three numeric columns average the same 20 predetermined orders per problem. Canonical uses ascending archive seed and K=16 . The final column spans the 20 individual-order mean effects at K=16 .
Dyson School of Design Engineering Imperial College London London, United Kingdom · Department of Electrical and Electronic Engineering Imperial College London London, United Kingdom