Does the level of test-time processing effort affect a language model's computational capability? A formal grammar-lens benchmark method, derived from the theory of computation, generates novel data at specified levels, each successively more computationally powerful than the levels below. Using this data, we tested GPT-5.5, and GPT-5.5-Pro-Medium, -High, and -XHigh, showing incrementally increased acceptance of test data: 16, 40, 43, and 52%, respectively, illustrating the benchmark's ability to discern fine-grained differences among models' added test-time reasoning levels. Additionally, as test-time compute effort increases, model responses diverge from human responses to the same data. This increasing disagreement may turn out to be broadly characteristic of more capable language models, introducing the potential of a general capability-versus-alignment conflict.
Figures & tables
Figure 1: The grammar–automata hierarchy equivalence classes ( left ) with formal grammars (brown) and automata (black) in order of increasing computational power, from finite state machines (FSM) to Turing machines (TM). Single-stack mechanisms (green) are distinguished from higher order of stacks (yellow) and from tape-based automata (red). ( right ) External memory associated with each automata class. (From Graham and Granger Graham and Granger (2026) )
Figure 2: Acceptance rate by grammar generator and by reasoning-effort level, with color scale reflecting average elapsed query time. IXG-generated data acceptance is at ceiling. CSG acceptance scales monotonically with reasoning effort. Average query time (colors) also scales monotonically, both with grammar class and with reasoning effort.
Grammar
Model/configuration
Trials
Acceptance (%)
Seconds/query
IXG
GPT 5.5
50
96.92±1.29
3.87±0.98
GPT 5.5 Pro (Med)
50
97.84±0.55
14.71±1.44
GPT 5.5 Pro (High)
50
97.91±0.57
20.80±5.02
GPT 5.5 Pro (XHigh)
50
97.64±2.00
50.33±25.53
CSG
GPT 5.5
50
16.08±3.31
7.10±2.27
GPT 5.5 Pro (Med)
50
40.35±4.64
105.21±12.51
Table 1: Final results: means ± SD across repeated trials over the 50-string sets.
Grammar
Comparison
Δ
pt
pW
d
CSG
5.5 → Pro Med
+24.27
1.6×10−33
7.3×10−10
−4.31
5.5 → Pro High
+26.68
1.2×10−37
7.2×10−10
−5.28
5.5 → Pro XHigh
+35.90
4.9×10−34
7.3×10−10
−4.42
Med → High
+2.41
1.1×10−2
1.4×10−2
−0.37
Med → XHigh
+11.63
5.7×10−13
3.5×10−9
−1.37
High → XHigh
+9.22
5.9×10−10
8.6×10−9
−1.09
Table 2: Pairwise acceptance-rate comparisons. Δ represents the percentage-point difference (second − first), pt the paired t -test, pW the Wilcoxon signed-rank, and d denotes Cohen’s d .
Figure 3: A valid parse in terms of the grammar generator Gix of its string generation: Leibniz who invites Riemann who conjectures Galileo who rests solves the mountains that eat or fall.
Figure 4: A valid parse in terms of the grammar generator Gcs of its string generation: The dogs that silently fade a person that deeply writes Laplace conjecture elegantly prove.
Grammar
Comparison
Δ (s)
pt
pW
d
CSG
5.5 → Pro Med
+98.11
1.5×10−45
1.8×10−15
−7.72
5.5 → Pro High
+126.64
2.2×10−37
1.8×10−15
−5.21
5.5 → Pro XHigh
+233.90
6.2×10−27
1.8×10−15
−3.09
Med → High
+28.53
1.7×10−9
1.1×10−7
−1.04
Med → XHigh
+135.79
1.6×10−17
3.6×10−15
−1.84
High → XHigh
+107.26
5.1×10−13
3.6×10−15
−1.37
Appendix
Table 4: Pairwise Query Latency comparisons. Δ represents the mean difference in seconds (second − first), pt the paired t -test, pW the Wilcoxon signed-rank, and d denotes Cohen’s d .