Existing adaptive-reasoning methods shorten the explicit reasoning a model writes, or skip it by switching modes; either way the answer is still written out token by token. Rather than deciding how much to think, we explore a different paradigm, \emph{zero-or-T thinking}: a single model either answers from the implicit thinking inside one forward pass, generating no token, or performs full autoregressive reasoning at a cost of T tokens when that is worth paying. We realise it in a hybrid decision model (HDM), where one frozen backbone serves both modes and a lightweight self-routing block chooses between them. The block estimates the value of thinking, the utility the slow mode is expected to add over the fast one, from what that implicit thinking has already produced, and the slow mode runs only where that value exceeds a price λ set per request. Where no per-decision label exists, as in episodic agentic tasks, we introduce Calibrated Decision Policy Optimisation (CDPO), a reinforcement-learning recipe that trains the block from episode-level returns. We evaluate HDM on MMLU-Pro, AGIEval, GPQA-Diamond, JevBench, BANKING77 and MiniWoB. Specifically, on JevBench HDM-4B scores 0.857 against Jev's 0.851 (+0.7%) at a median 0.179,s against 0.652 (−72.5%). On the agentic venue MiniWoB HDM-27B cuts token cost by 80.1%.
Figures & tables
Figure 1: HDM’s zero-or- T thinking on two JevBench questions. LEFT, a case where the fast mode is sufficient: the predicted value of thinking is below λ , the block answers at no token, and running both modes afterwards confirms the slow mode would not have changed the answer. RIGHT, a case that should go to the slow mode: the predicted value is above λ , routing to the slow mode, and running both modes afterwards confirms the fast answer would have been wrong. Appendix G gives both thinking blocks in full.
Figure 2: Backbone sketch of HDM. The frozen backbone encodes a typed request with no token generated; the self-routing block, which holds the only learned weights, reads the readout state hi and eight statistics of the fast answer pi and returns the joint outcome of the two modes, from which ci and vi follow. Routing occurs once: the model either returns the cached fast answer or decodes autoregressively with the same backbone. The dashed feedback arrow returns each generated token to the shared context, reusing the KV cache; only the completed answer enters the response.
MMLU-Pro
AGIEval
GPQA-Diamond
Policy
λ
Esc.
Acc.
Tok.
Sec.
Esc.
Acc.
Tok.
Sec.
Esc.
Acc.
Tok.
Sec.
Qwen3.5-4B
Fast only
–
0%
0.543
0 ( − 100%)
0.27 ( − 98%)
0%
0.606
0 ( − 100%)
0.31 ( − 98%)
0%
0.429
0 ( − 100%)
0.27 ( − 99%)
Slow only
–
100%
0.853
2281
12.30
100%
0.842
2364
12.79
100%
0.717
3685
19.75
HDM
−0.1
97%
0.844
2220 ( − 2.7%)
11.93 ( − 3.0%)
100%
0.842
2364 (0%)
12.75 ( − 0.4%)
100%
0.717
3685 (0%)
19.56 ( − 1.0%)
HDM
0
66%
0.844
1845 ( − 19%)
9.92 ( − 19%)
71%
0.830
1801 ( − 24%)
9.75 ( − 24%)
93%
0.707
3491 ( − 5.3%)
18.58 ( − 5.9%)
Table 1: Evaluating how far HDM compresses token and wall-clock cost on general-purpose question answering. Accuracy, escalation and cost at five prices. Escalation is the share of questions sent to the slow-thinking mode; tokens and seconds are per question. All six columns are at B=4096 , layer −1 , one question to a request. Brackets give a cost’s change against that block’s Slow only row, and bold marks our rows. λ prices one escalation, so it is comparable down a column; a slow pass costs a different number of tokens on each benchmark and on each backbone, and the same λ is a different price across either.
System
Esc.
Accuracy
P50 (s)
Laya, 421M
—
0.584
—
smalljev semantic-v9
—
0.606
—
jeff, 400M
—
0.628
—
Open-Jev, 2B
—
0.645
—
kev, 4B
—
0.662
—
kev, 8B
—
0.714
—
Table 2: Accuracy and latency on JevBench v1’s 231 public items, under the coded readout of Appendix D . Our rows are HDM-4B (Qwen3.5-4B) and HDM-27B (Qwen3.8-27B) at B=4096 , each with a block trained on MMLU-Pro and nothing retrained here. Escalation is the share of items sent to the slow-thinking mode and P50 the median wall-clock seconds per item.
Policy
Esc.
Acc.
Tok.
Sec.
Tacit-9B + AnyJev
6.0%
0.913
140.2
3.82
Qwen3.8-27B + AnyJev τ=1.0
7.0%
0.923
33.6
—
Qwen3.8-27B + AnyJev τ=0.5
3.3%
0.933
16.9
—
Slow only
100%
0.913
135.3
3.53
Fast only
0%
0.923
0.0
0.57
HDM , λ=0
6.7%
0.920
14.1
0.83
Table 3: Accuracy and cost on BANKING77 as a 20-way intent task, 300 items, on Qwen3.8-27B at B=4096 , with a block trained on MMLU-Pro and nothing retrained here.
ALFWorld
MiniWoB
Policy
Esc.
Reward
Tok./ep
Sec./ep
Esc.
Reward
Tok./ep
Sec./ep
Fast only
0 %
0.463
0 ( − 100%)
55.4 ( − 70%)
0 %
0.407
0 ( − 100%)
21.3 ( − 78%)
Slow only
100 %
0.642
6254
185.2
100 %
0.519
3345
96.1
HDM
46 %
0.560
3015 ( − 52%)
108.4 ( − 41%)
26 %
0.469
664 ( − 80%)
31.3 ( − 67%)
Table 4: Evaluating CDPO-trained HDM on episodic agentic tasks, with one outcome an episode rather than one a decision. Reward, escalation and cost per episode on two agentic venues, Qwen3.8-27B throughout. Esc. is the share of steps sent to the slow-thinking mode, Reward the environment’s terminal signal, and brackets give the change against that venue’s Slow only row. ALFWorld is 134 held-out valid_unseen games at 256 thinking tokens a step, one run at λ=0 , its block fitted on one round of 150 episodes collected under random routing; MiniWoB is 27 held-out click tasks at 512 a step, its routed row pooled over three seeds, and its endpoints one run each.
CV (11 subjects)
Test (3 unseen)
Configuration
Dim
rescue
right
rescue
right
Stats only
8
0.671
0.763
0.701
0.840
Layer −1 , raw
2568
0.765
0.758
0.716
0.821
Layer −1 , PCA-64
72
0.745
0.764
0.692
0.847
Layer 16, raw
2568
0.733
0.710
0.742
0.816
Layer 16, PCA-64
72
0.719
0.737
0.726
0.845
Table 5: Evaluating the discriminative ability to filter out the questions thinking helps on, and which features supply it. Qwen3.5-4B, the 1,500 -question MMLU-Pro selection, B=1024 . Two AUROCs per configuration, in cross-validation over the 11 training subjects ( 1,154 questions) and on the three held-out categories ( 346 ): rescue , whether thinking turns the fast answer from wrong to right, the routing AUROC ; and right , whether the fast answer is already right. Dim is the block’s input width, hidden state plus eight statistics. The last row zeroes the statistics rather than dropping them. One seed.
Figure 3: The price-of-thinking curve. Qwen3.8-27B on MMLU-Pro’s 346 held-out questions. Solid line, left axis: accuracy. Dashed line, right axis: thinking tokens saved against always thinking. Dotted line: always thinking’s accuracy, 0.902 . Each line passes through all 111 prices swept at a step of 0.01 , marked at every fifth; the large marks are the five prices Table 1 reports.
Rule
Qwen3.8-27B
Qwen3.5-4B
always thinking
591
1709
random, p swept
213
991
confidence
138
945
ThinkSwitcher (reproduced)
352
1709
RouteLLM (reproduced)
112
822
HDM
103
792
Table 6: Comparison with non-parametric baselines and prior thinking-mode routing methods, in the thinking tokens needed to reach 90 % of always-thinking’s accuracy; lower is better. JevBench’s 231 public items, content readout at B=4096 , blocks fitted on MMLU-Pro’s training categories and transferred. Each cell is tokens per question, read off that rule’s monotone frontier.
Appendix figures & tables1 asset
Supplementary material from the paper’s appendix.
Appendix
Distribution statistics
max prob, norm. entropy, top-2 margin, concentration, logK , type one-hot
Optimiser
AdamW, lr 10−3 , weight decay 0.05, batch 256
Regularisation
dropout 0.3, early stopping (patience 6)
CDPO
λ∼U[−0.2,0.5] , τ=0.05 , clip ϵ=0.2 , mean-return baseline; same optimiser, early stopping
Slow-mode decoding
greedy, B=4096 , stop at </think>
Collection batch
m=10 questions a shared state
Agentic budget
256 tokens a step (ALFWorld), 512 (MiniWoB)
Appendix
Table 7: Settings used for every block and every run. Width w is the one entry that follows the block’s input rather than being fixed across configurations.
School of Data Science, Fudan University · Shanghai Institute of Artificial Intelligence for Education, East China Normal University · College of Computer Science and Artificial Intelligence, Fudan University +1