cs.AINov 4, 2026

Routes of Reasoning: Reading and Reshaping Test-Time Compute in MoE Models

Authors: Shengjie Wang, Mohsen Fayyaz, Nanyun Peng

Organizations: University of California, Los Angeles

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Reasoning models allocate test-time compute across distinct behaviors such as planning, deriving, and verifying. However, models frequently "overthink," continuing to verify and re-derive long after finding a correct answer. This wastes expensive compute and can even degrade performance. Existing interventions, such as token budgets or steering, primarily regulate reasoning duration rather than targeting specific behaviors. In this work, we demonstrate that Mixture-of-Experts (MoE) routers provide a natural, training-free interface for behavioral control. We first split reasoning traces into sentence-level segments and label each with one of six reasoning behaviors (planning, execution, restatement, fact, reflection, and conclusion), so that every token carries the behavior of its segment. Linear probes on router logits predict this label nearly as well as probes on the residual stream, the model's full hidden state and the standard target for probing, while using about half as many features. These signals generalize: experts associated with a reasoning behavior on one dataset predict the same behavior on the others, across six datasets spanning five domains (math, code, science, knowledge, and commonsense). Building on this, we selectively edit expert router scores during inference to curb overthinking. Across four models and four benchmarks, this intervention reduces the number of generated tokens by a median of 30.4% (and up to 63.3%). Crucially, it encourages the model to stop once the problem is solved, rather than skipping essential reasoning steps. Accuracy never drops by more than 0.5 percentage points, and frequently improves. Ultimately, MoE routing offers both an interpretable lens into reasoning states and a lightweight mechanism for optimizing test-time compute.

Figures & tables

Figure 1: Overview. Read : router logits reveal the behavior of each reasoning step (Section 4 ). Identify : behavior-associated experts recur across domains (Section 5 ). Steer : editing behavior-associated experts reduces generated tokens while largely preserving accuracy (Section 6 ).
BehaviorWhat the segment does
planningproposes how to proceed
executionderives a new result
restatementrepeats without new inference
factinvokes established knowledge
reflectionchecks or revises earlier work
conclusionstates a result or closes a subgoal
Table 1: Six reasoning behaviors.
Figure 2: Linear behavior readout. Curves use one layer; circles mark dev-selected layers. The dashed line concatenates router L8–L16. Bands: 95% problem-bootstrap CIs.
Figure 3: Expert-set transfer at layer 36. Rose: overall AUC and four reported behaviors on full segments. Blue-gray: the same ranking after dropping the first five tokens of each segment. Dashed line: chance.
Figure 4: Efficiency. Token change relative to the unmodified model; the table reports baseline accuracy and accuracy changes, with baseline tokens in gray. Rows above give each edit: behavior ( P planning, Rf reflection, and Rs restatement), direction, and number of layer–expert pairs.
Figure 5: Cut at the first answer.
Figure 6: Same prefix and seed, different next plan. More in Appendix E.2.3 .
Appendix figures & tables9 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 7: One annotated trace (Qwen3-30B-A3B, NuminaMath problem 00245; spans 7–27 and 85–88 of 108).
SourceTrainDevTest
GSM8K training split1002228
NuminaMath training split982329
AIME 20232145
Problems2194962
Segments52,00810,31516,632
Appendix
Table 2: Problem-grouped corpus splits, stratified by source.
LayersWidthToken testSegment test
L12 only1280.6080.708
L8–L16 ( ±4\pm 4 )1,1520.6590.731
L7–L17 ( ±5\pm 5 )1,4080.6560.731
L6–L18 ( ±6\pm 6 )1,6640.6550.733
L5–L19 ( ±7\pm 7 )1,9200.6530.732
Appendix
Table 3: Router layer windows centered on L12 (macro-F1).
ReadoutWidthMacro-F1
Token majority vote1280.608
Token probability average1280.613
Final token1280.702
Max pooling1280.682
Mean pooling1280.708
Mean + max pooling2560.716
Appendix
Table 4: Segment readout from router L12.
SourceDomainSegments
Mathematical mixtureMathematics78,955
MATH500 ( Hendrycks et al., 2021 )Mathematics79,816
LiveCodeBench ( Jain et al., 2024 )Code187,068
GPQA (non-Diamond) ( Rein et al., 2024 )Science112,825
MMLU-Pro ( Wang et al., 2024 )Knowledge53,770
CommonsenseQA ( Talmor et al., 2019 )Commonsense17,123
Appendix
Table 5: Cross-dataset association corpus, with 330 problems per source.
ModelLayersTotalActiveRoutedTop- kk & shared
Gemma-4-26B-A4B-it3026B4B1288
Qwen3-30B-A3B4830B3B1288
Qwen3.6-35B-A3B4035B3B2568 ++ 1 shared
GPT-OSS-120B36120B∼\sim 5.1B1284
Appendix
Table 6: MoE models and routing configurations used for steering.
  Δ\Delta tokens (%)Δ\Delta accuracy (pp)
ModelBenchmarkPromptOursPromptOurs
Gemma-4MATH500−27.3-27.3−32.2-32.2−0.6-0.6+0.0+0.0
 AIME+0.6+0.6−24.9-24.9−7.3-7.3+2.2+2.2
 GPQA-D−9.1-9.1−21.2-21.2+0.3+0.3−0.5-0.5
 AICrypto−18.6-18.6−44.9-44.9−0.7-0.7+2.2+2.2
Qwen3MATH500−17.4-17.4−23.1-23.1−0.6-0.6+0.6+0.6
Appendix
Table 7: Prompt baseline versus behavior-targeted steering for all models. Changes are relative to the unmodified model; token changes average the evaluated seeds, as in Figure 4 .
SuppressTokensAnswer atRe-checksDoubtsOpenersCorrect
none1,494[31]4710yes
planning ( ↓\downarrow 200)1,076[30]237yes
reflection ( ↓\downarrow 100)1,579[38]4315yes
restatement ( ↓\downarrow 100)1,866[43]5715yes
execution ( ↓\downarrow 30)1,758[64]4514yes
execution ( ↓\downarrow 100)16,384––1940no
Appendix
Table 8: One problem under each suppression. Parentheses give the number of suppressed layer–expert pairs. Re-checks: rounds of checking after the answer; Doubts and Openers: reflection and planning sentences in the trace.
Planning expertsWrap-upNew angleThinking ends
unmodified23235%
suppressed throughout681117%
suppressed for one window43216%
Appendix
Table 9: First plan after the cut (Qwen3.6, MATH500, 314 cut points). The last column is the share of continuations that end thinking within ∼170{\sim}170 tokens.
More in this topic

Efficient Language Model Reasoning

Efficient language model reasoning studies how to improve problem-solving accuracy while reducing inference, memory, latency, and training costs.

See all 388 papers

Explore similar work

CardsList
Oct 19, 2026

TempoMoE: Identifying and Steering Fast-Slow Thinking Experts for Efficient Multimodal Reasoning

Multimodal Mixture-of-Experts (MoE) reasoning models rely on long chain-of-thought (CoT) for complex tasks, yet often overthink easy inputs, increasing token cost and latency, and sometimes degrading accuracy. Existing methods optimize reasoning length during training or impose external control at inference, but leave the underlying reasoning mechanism a black box. We find that fast and slow thinking exhibit distinct expert-routing patterns: holding the input fixed and varying only the prompted reasoning tempo produces stable routing shifts. Suppressing slow-thinking experts shortens the reasoning chain, whereas suppressing fast-thinking experts lengthens it, providing causal evidence that these experts influence reasoning depth. Based on this finding, we propose TempoMoE, a training-free, two-stage framework. It identifies fast- and slow-thinking experts via tempo-contrastive routing analysis and redistributes routing weights within the original top-k set according to sample difficulty. It amplifies fast-thinking experts and suppresses slow-thinking experts on easy inputs and reverses this adjustment on hard ones, enabling difficulty-adaptive control over reasoning depth. We evaluate TempoMoE on Kimi-VL, Qwen3-VL, and ERNIE-4.5-VL across six benchmarks. Compared with Vanilla, TempoMoE consistently improves accuracy while reducing output tokens, with accuracy gains of up to 2.6 points and token reductions of up to 35.4%.

Xiyue Zhang, Keru Zhou, Ziyi Wang +2

Zhejiang University · Tsinghua University · Wenzhou Business College

Mixture-of-Experts InferenceEfficient Multimodal InferenceVLM ReasoningMultimodal CoT ReasoningMixture-of-Experts Interpretability
AskPDF
Aug 18, 2026

Beyond the Trace: Coupling an Interpretable Reasoning-State Readout to Native MoE Routing

What a reasoning model writes is only a partial record of the process that produces it. We introduce a two-level internal readout for mixture-of-experts reasoning. We first distill vocabulary-scale J-space into J64, a 64-axis semantic frame learned from the model's own reasoning states. J64 reveals readable process state that the emitted trace does not show: it separates inference effort from problem-induced strain. It also adds 0.096 to 0.135 held-out AUC over a baseline that reads the same rollout as token occupancy and aggregates it in exactly the same way. We then reconstruct J64 from native expert-routing statistics. The result is R64, a low-overhead proxy: its median per-axis correlation with J64 is 0.69 to 0.86 across three models and two families, and on gpt-oss-20b it preserves 95 to 100% of J64's predictive gain. The readout supports test-time decisions at two temporal resolutions. Over completed candidate sets, J64 and R64 improve single-branch selection, and R64-weighted voting improves plain majority voting in seven of eight settings. During generation, rolling readout windows drive a cumulative stop-and-resample policy whose operating point is fixed on training questions alone. J64 improves accuracy by 1.1 to 5.9 points over a sibling-permuted control, and the routing-only R64 proxy retains 0.9 to 3.2 of those points. Finally, router edits aimed at the mechanism J64 names induce the predicted reasoning behaviors and shift a diagnosed stall from numerical guessing toward exact symbolic execution. Together, J64 makes latent process state readable, while routing makes it deployable and actionable.

Kang Chen, Sihan Zhao, Yixin Cao +1

Fudan University · Shanghai Innovation Institute

Mixture-of-Experts InterpretabilityLLM InterpretabilityReasoning Trace AnalysisMixture-of-Experts ModelsExpert RoutingInference-Time Compute Allocation
AskPDF
Sep 6, 2026

From Concentration to Differentiation and Back: Routing Effective Rank in MoE Reasoning Cohorts

Test-time scaling produces cohorts of reasoning rollouts, yet there is no standard label-free account of how their internal computation reorganizes as inference unfolds. We introduce routing effective rank deff, the entropy-effective dimensionality of a cross-rollout graph built from MoE expert-routing similarity. Across ten MoE configurations and five math/science benchmarks, deff exhibits a reproducible low-high-low trajectory, with a prominent interior maximum in 98.5% of 3,105 model-question cohorts: routing similarity is concentrated early, maximally differentiated at intermediate budgets, and reconcentrated later, and the timing of this maximum varies systematically with architecture and reasoning effort. An exact decomposition separates cohort-wide common-mode mass from residual spectral dimensionality: common-mode reallocation accounts for about two thirds of the trajectory, while the residual spectrum contributes about one quarter and retains substantial variation beyond the common mode. The decomposition further localizes behavior: among non-unanimous cohorts, increases in common-mode concentration strongly predict same-answer recoverability, and higher reasoning effort delays the maximum by 2.59 octaves (doublings of the token budget) and consistently expands the high-rank period across all four tested architectures, locating the effort effect in timing and duration rather than peak amplitude. Correctness comparisons separate structural monitoring from answer selection, positioning routing effective rank as a decomposable, label-free diagnostic of cohort organization - a principled spectral lens on how MoE reasoning cohorts differentiate and reconcentrate over inference time.

Kang Chen, Sihan Zhao, Yixin Cao +1

Fudan University · Shanghai Innovation Institute

Mixture-of-Experts ModelsMixture of ExpertsInference-Time Scaling
AskPDF