Compute scaling for LLM reasoning trades off exploring solution approaches (breadth) against refining promising ones (depth), yet why a given trade-off works, and why it often fails to transfer across models, remains unclear. We argue that the optimal strategy depends on the model's diversity profile, the spread of probability mass across solution approaches, and that this must be characterized before any exploration strategy is adopted. We formalize this with a framework decomposing reasoning uncertainty, deriving when depth-based refinement outperforms parallel sampling, and validate it across three model families at both inference and training. Our central finding is that the diversity regime dictates the strategy: low-diversity aligned models benefit from depth-based refinement with lightweight intrinsic signals, whereas high-diversity base models are often harmed by it, and instead need breadth or stronger signals to compensate.
Figures & tables
Model
Baseline
ENT
SR
ENT Lift
SR Lift
Qwen-3 4B base
0.9004
0.6377
0.6406
-29.18%
-28.85%
Qwen-3 4B instruct
0.9121
0.9180
0.9170
0.65%
0.54%
Qwen-3 4B RLVR
0.6885
0.7471
0.7373
9.34%
7.90%
OLMo-3 7B base
0.8506
0.9072
0.8828
6.65%
3.79%
OLMo-3 7B instruct
0.6797
0.8154
0.8086
19.96%
18.96%
OLMo-3 7B RLVR
0.7201
0.8300
0.7813
15.26%
8.50%
Table 1: Pass@ 16 for Baseline , ENT and SR . The Lift showcases the gain over the Baseline .
Figure 1: Question-level Vendi diversity with cosine similarity. Base models produce the largest effective response diversity, with Nemotron Base showing the strongest contrast against its Instruct counterpart.
Figure 2: Correctness-conditioned Vendi score. Each bar splits responses within a question and model cell into correct and incorrect subsets, computes Vendi separately for each, and averages over questions.
Model
Baseline
ENT
Lift
Qwen-3 1.7B base ( 0.10 )
0.27
0.16
-40.7%
Qwen-3 1.7B instruct ( 0.37 )
0.61
0.59
-3.2%
Table 2: Best validation accuracy during training for the GSPO baseline and the MCTS-entropy (ENT) variant, on Qwen-3 1.7B base and post-trained checkpoints. Starting validation accuracy is shown in brackets. Lift is the relative gain over the corresponding baseline.
Symbol
Meaning
Prefix-selection setting
N,K,L
Total rollout budget, number of selected rollouts, and prefix length.
x1:L(i)
Length- L prefix of rollout i .
ℓt(i),mt(i)
Chosen-token log-probability and top-1/top-2 log-probability gap at position t .
q(i),q^(i)
Prefix score and its normalized version.
Ut,disagree(t)
Number of distinct tokens and disagreement score at position t .
Table 3: Notation
Appendix figures & tables7 assets
Supplementary material from the paper’s appendix.
Appendix
Method
Quality signal
Diversity signal
Uses top- k ?
Random@ K
none
none
No
MMR
prefix logprob q^(i)
Jaccard over prefix tokens
No
Adaptive Dispersion
prefix logprob q^(i)
token-disagreement anchors + hybrid distance
Yes
Appendix
Table 4: Taxonomy of prefix-selection methods used in the main text. “Top- k ” indicates whether the method requires per-position top- k logprob vectors (beyond chosen-token logprobs).
Figure 3: Assumed-cluster spread. Base cells form wider response clusters than Instruct cells, showing that instruction tuning compresses both mode diversity and geometric spread.
Figure 4: Correctness-conditioned clustering. Incorrect responses are typically more spatially dispersed than correct responses, especially for Base models.
Figure 5: Critic rewards offer a direct signal on quality of exploration; the plot verifies the expected trend on Qwen family i.e. the performance of Entropy-based MCTS closely matches standard sampling for the instruct model, while significantly degrades on the base model.
Figure 6: The RBF kernel compresses the Vendi scale but preserves the main ordering: Base cells remain the most diverse.
Figure 7: Under the RBF kernel, incorrect traces still account for the largest correctness-conditioned diversity gaps.
Figure 8: Qwen shows the clearest Base–Instruct geometric separation, OLMo the weakest, and Nemotron an intermediate but still visible shift.