Beyond the Trace: Coupling an Interpretable Reasoning-State Readout to Native MoE Routing

Kang Chen; Sihan Zhao; Yixin Cao; Yu-Gang Jiang

cs.AIAug 18, 2026

Beyond the Trace: Coupling an Interpretable Reasoning-State Readout to Native MoE Routing

Authors: Kang Chen, Sihan Zhao, Yixin Cao, Yu-Gang Jiang

Organizations: Fudan University · Shanghai Innovation Institute

Abstract

What a reasoning model writes is only a partial record of the process that produces it. We introduce a two-level internal readout for mixture-of-experts reasoning. We first distill vocabulary-scale J-space into J64, a 64-axis semantic frame learned from the model's own reasoning states. J64 reveals readable process state that the emitted trace does not show: it separates inference effort from problem-induced strain. It also adds 0.096 to 0.135 held-out AUC over a baseline that reads the same rollout as token occupancy and aggregates it in exactly the same way. We then reconstruct J64 from native expert-routing statistics. The result is R64, a low-overhead proxy: its median per-axis correlation with J64 is 0.69 to 0.86 across three models and two families, and on gpt-oss-20b it preserves 95 to 100% of J64's predictive gain. The readout supports test-time decisions at two temporal resolutions. Over completed candidate sets, J64 and R64 improve single-branch selection, and R64-weighted voting improves plain majority voting in seven of eight settings. During generation, rolling readout windows drive a cumulative stop-and-resample policy whose operating point is fixed on training questions alone. J64 improves accuracy by 1.1 to 5.9 points over a sibling-permuted control, and the routing-only R64 proxy retains 0.9 to 3.2 of those points. Finally, router edits aimed at the mechanism J64 names induce the predicted reasoning behaviors and shift a diagnosed stall from numerical guessing toward exact symbolic execution. Together, J64 makes latent process state readable, while routing makes it deployable and actionable.

Figures & tables

Figure 1: The internal readout at a glance, on one gpt-oss-20b High rollout of AIME-24 problem

29

. (a) A pair matched on the emitted text: rollout

1898

and a sibling rollout on the same problem emit an identical

15

-token span, yet their J64 readings over that span separate along named axes (bars, standardized within the problem). The continuations diverge accordingly, and only one rollout is correct. (b) The same rollout end to end on the specifics axis of (a) (

1024

-token centered average). An axis’s token family is its nearest

20

vocabulary directions. Not one of this axis’s family words is written in the

23

k tokens, and the same holds for

31

of the

64

axes, yet the reading keeps moving. (c) The silent axis of (b), reconstructed from native routing alone and plotted on the same token axis. A ridge map, fitted with question-held-out folds, recovers it from the layer-

20

expert usage spectrum of

256

-token windows. Median per-axis

r

0.69

over the

64

axes with

30

above

0.7

(

0.00

for shuffled routing). The axis shown ranks

12

th.

Figure 2: The instrument: a named readout, a routing proxy, three test-time decisions. (a) At a single token position we read the hidden state

h_{t}

against vocabulary-aligned directions, bringing out concepts the state supports whether or not the rollout ever writes them; two of the high readings are named in dashed callouts, and the rollout wrote neither word. A high reading for caution does not mean the model emitted the word. (b) Those directions are merged into the

64

named axes of the J64 frame, built once per model and named only after construction (§ 4.1 ). The same reading is reachable in two ways, at very different cost: from hidden states, which means replaying the rollout through the model, or from native expert routing, which generation already emits. The braces mark the two routes as reaching the same

64

coordinates, not two readouts that happen to resemble each other: a ridge map is fitted once, on J64 as its target, to reproduce J64 from routing without outcome labels, and recovers it at a median per-axis agreement of

r=0.69

–

0.86

(§ 4.2 ), which is what the

\approx

records. (c) Routing statistics are emitted by generation itself, so reading R64 costs one ridge projection rather than a second forward pass. R64 is therefore the only copy available at serve time, and the one that acts: rollout selection among completed rollouts (§ 5.1 ), online stopping on a rolling window during generation (§ 5.2 ), and router edits that move a named axis to test the readout’s causal semantics (§ 5.3 ).

Figure 3: J64 exposes readable latent process state and outcome signal beyond the matched text channel. (a) The posture coordinate separates the three effort settings. Dots are question means. (b) Pooled rollout -level accuracy across quintiles of the strain coordinate, one row per effort with High at the top and Low at the bottom, as in panel (a). Each row’s

r

is the rollout-level correlation with correctness:

-0.32

at High,

-0.28

at Medium and

-0.35

at Low. It is not the problem-level correlation with difficulty, which is

+0.41/+0.46/+0.41

at Low/Medium/High. (c) Adding J64 to the matched text representation raises held-out trajectory-outcome AUC at every effort. The bars are the two channels’ AUC; the

+

value above each pair is the paired increment, estimated per question and question-clustered, with ∗ marking an interval that excludes zero. It is therefore not exactly the difference of the two rounded bar labels.

Figure 4: Routing is a faithful, rollout-specific proxy for J64. (a) Median per-axis held-out reconstruction in the eight model–effort settings, each read through its own frame. The dashed line is the strong-reconstruction cut at

r{=}0.7

, which

28

59

of a setting’s

64

axes clear (shuffled routing:

r\approx 0

). Marker shape and color give the model family. (b) The share of J64’s outcome increment over matched token occupancy that the routing-only reconstruction keeps.

Figure 5: Routing indicates which stage the reasoning is in. Two of the eight expert modules and the J64 axes they lock onto, along one High-effort trace: module usage in red, matched axis readout in grey, both curves min–max scaled within the window. Each panel is a

\pm 72

-token window centered where the two curves are jointly highest. Module 1 tracks the drift-and-reconsideration axis and peaks on the paragraph break of “…a complicated process. But”; Module 0 tracks the symbol axis and peaks inside a symbolic expression. All eight modules, their enriched vocabulary and the matched axes are in Appendix A.3 .

single-trace selectors — frozen transfer, averaged over all four training benchmarks
	gpt-oss-20b				gpt-oss-120b				Qwen3-30B-A3B
	Low	Med	High	avg	Low	Med	High	avg	Think	Inst
Avg@64 (random pick)	35.1	66.5	70.9	57.5	48.8	73.3	83.1	68.4	78.8	60.9
J64	41.9 (+6.9)	68.1 (+1.6)	77.5 (+6.6)	62.5 (+5.0)	54.4 (+5.6)	76.9 (+3.7)	88.6 (+5.5)	73.3 (+4.9)	85.3 (+6.5)	65.6 (+4.7)
R64 (routing)	35.8 (+0.8)	68.3 (+1.9)	76.1 (+5.2)	60.1 (+2.6)	51.1 (+2.3)	75.6 (+2.3)	86.1 (+3.0)	70.9 (+2.5)	87.2 (+8.4)	64.7 (+3.8)
token occupancy	35.6 (+0.5)	64.2 ( $-$ 2.3)	74.7 (+3.8)	58.2 (+0.7)	53.9 (+5.1)	75.6 (+2.3)	86.9 (+3.8)	72.1 (+3.7)	87.2 (+8.4)	65.6 (+4.7)

Table 1: Cross-benchmark frozen transfer, best-of-64 (%), across eight deployment settings. Every learned method, including R64’s reconstruction ridge and the token-occupancy vocabulary, is trained on one benchmark and applied unchanged to the other three. Each cell averages that setting’s

12

off-diagonal transfer pairs (deterministic single pick, all

30

questions per test set). Branches are scored by extracted answer (Appendix B.2 ). Parenthesized values are the gain over the Avg@64 reference of the same column, i.e. over a random pick. The avg columns average a model’s three effort settings. Bold

=

best per column within each block.

	oss-20b	oss-120b	Qwen	pooled
failed questions	102	75	46	223
random pick	8.5	10.6	5.3	8.7
J64	14.3 (+5.8)	23.2 (+12.6)	19.2 (+13.9)	17.0 (+8.3) ∗
R64 (routing)	12.3 (+3.8)	20.0 (+9.3)	16.1 (+10.8)	14.1 (+5.3) ∗
token occupancy	8.1 ( $-$ 0.4)	14.6 (+3.9)	15.1 (+9.8)	12.3 (+3.5) ∗
length	3.3 ( $-$ 5.2)	14.6 (+4.0)	11.7 (+6.4)	5.4 ( $-$ 3.4) ∗

Table 2: Rescue on the questions where plain majority voting scores zero (%), by model. Protocol identical to Table 1 , restricted to the failed questions. Model columns average that model’s effort settings, and pooled is the rate over all

223

. Question-paired clustered CIs: J64

+8.3

∗

[+5.3,+11.3]

, R64

+5.3

∗

[+2.3,+8.5]

, length

-3.4

∗

[-6.4,-0.1]

. Per-setting detail is in Appendix A.4 .

gpt-oss-20b High — one uninterrupted rollout: $56.8$
Target cost	J64	R64	DeepConf	permSib
$1.2\times$	$58.3$ / $1.21$	$\mathbf{59.2}$ / $1.21$	$58.7$ / $1.22$	$57.0$ / $1.20$
$1.4\times$	$\mathbf{62.2}$ / $1.42$	$60.8$ / $1.41$	$60.0$ / $1.44$	$57.8$ / $1.45$
$1.6\times$	$\mathbf{62.9}$ / $1.59$	$60.8$ / $1.50$	$59.8$ / $1.54$	$57.7$ / $1.57$
$2.0\times$	$\mathbf{63.5}$ / $1.65$	$60.8$ / $1.50$	$59.4$ / $1.76$	$57.6$ / $1.67$
Qwen3-30B-A3B Thinking — one uninterrupted rollout: $78.1$

Table 3: Cumulative stop-and-resample accuracy under the nested protocol. Each cell reports accuracy (%) / realized test cost, the latter as a multiple of one uninterrupted rollout. Outcomes count delivered answers only, hence the reference level sits below Table 1 ’s random pick (Appendix B.2 ). Question-paired differences against permSib are in the text. Bold

=

best per row.

Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 6: Cross-effort transfer of the instantaneous routing $\rightarrow$ J64 map (the token-level version): training the map at one effort setting and testing at another loses at most $0.04$ reconstruction correlation.

Figure 7: Per-axis held-out reconstruction for the two settings whose

64

-axis vectors are archived: the main figure reports medians and counts for all eight model–effort cells, this one shows the underlying distributions (dashed line

r{=}0.7

; the shuffled-routing control sits at zero).

Mod.	experts	most enriched tokens ( $\times$ ratio)	what it fires on	matched axis	$r$
0	8,23,5,29,16	+b $\times$ 7.9, $\theta$ $\times$ 7.8, (a $\times$ 7.2, R $\times$ 7.2, d $\times$ 7.1	algebraic symbols: variables and expression pieces	#15 symbol	$+0.48$
1	11,17,30,13,7	But $\times$ 10.0, Now $\times$ 9.7, ). $\times$ 9.6, Wait $\times$ 9.6, Alternatively $\times$ 9.6	turn markers and paragraph ends	#44 drift	$+0.59$
2	16,31,25,0,8	ings $\times$ 9.8, letters $\times$ 9.8, path $\times$ 9.8, solutions $\times$ 9.8, blocks $\times$ 9.8	combinatorial objects: discrete structure nouns	#7 alternation	$-0.45$
3	7,29,8,12,18	$>$ = $\times$ 9.5, .. $\times$ 8.9, $>$ $\times$ 8.9, $<$ $\times$ 8.8, )/( $\times$ 8.4	inequalities, fractions and exponents	#54 numeral	$+0.33$
4	13,17,6,20,3	The $\times$ 10.0, find $\times$ 10.0, using $\times$ 10.0, use $\times$ 9.8, on $\times$ 9.6	method statements (“use … to find …”)	#29 named-object	$+0.45$
5	2,17,15,5,1	bis $\times$ 10.0, equ $\times$ 10.0, quadr $\times$ 9.8, .e $\times$ 9.8, non $\times$ 9.6	geometric word roots (bisect, quadr-, hex-)	#30 difficulty	$-0.21$

Appendix

Table 4: The eight expert modules of the token-level factorization (gpt-oss-20b, layer

20

168

k token positions from

420

rollouts across the twelve tier

\times

benchmark banks; NMF with eight components, run means removed). Enrichment ratios are capped by the top-decile construction at

10

. Subword fragments are shown as tokenized. The correlations are from this

168

k-position run; the main text quotes the independent

347

k-position run, which agrees with it to within

0.01

on every module.

	gpt-oss-20b			Qwen3-30B-A3B		pooled
	Low	Med	High	Think	Inst
questions	198	198	198	198	198	990
random pick	56.3	64.4	59.3	71.1	66.3	63.5
J64	59.6 (+3.3)	71.7 (+7.4) ∗	67.2 (+7.9) ∗	72.7 (+1.7)	69.2 (+2.9)	68.1 (+4.6) ∗
R64 (routing)	59.6 (+3.3)	69.7 (+5.3) ∗	65.2 (+5.8) ∗	70.7 ( $-$ 0.4)	73.7 (+7.4) ∗	67.8 (+4.3) ∗
token occupancy	55.1 ( $-$ 1.2)	66.7 (+2.3)	67.7 (+8.4) ∗	72.2 (+1.2)	64.6 ( $-$ 1.7)	65.3 (+1.8)

Appendix

Table 5: Single-branch selection on GPQA (%), best-of-

64

: a second task domain,

198

graduate-level science questions

\times

64

rollouts per setting, question-held-out folds and question-residualized features (Appendix A.4 ). Parenthesized values are the gain over the random-pick reference of the same column; pooled is the single rate over all

990

question–setting units. Bold

=

best per column.

	gpt-oss-20b				gpt-oss-120b				Qwen3-30B-A3B		all
	Low	Med	High	avg	Low	Med	High	avg	Think	Inst	pooled
failed qs	64	20	18	102	50	19	6	75	11	35	223
random pick	9.0	12.3	4.2	8.5	8.4	16.8	6.8	10.6	4.1	6.5	8.7
J64	14.1 (+5.1) ∗	23.3 (+11.0)	5.6 (+1.4)	14.3 (+5.8)	17.3 (+9.0) ∗	24.6 (+7.8)	27.8 (+21.0)	23.2 (+12.6)	21.2 (+17.1) ∗	17.1 (+10.6) ∗	17.0 (+8.3) ∗
R64	9.9 (+0.9)	23.3 (+11.0)	3.7 ( $-$ 0.5)	12.3 (+3.8)	11.3 (+3.0)	26.3 (+9.5)	22.2 (+15.5)	20.0 (+9.3)	15.2 (+11.0)	17.1 (+10.6) ∗	14.1 (+5.3) ∗
token occupancy	8.9 ( $-$ 0.2)	10.0 ( $-$ 2.3)	5.6 (+1.4)	8.1 ( $-$ 0.4)	13.3 (+5.0)	19.3 (+2.5)	11.1 (+4.3)	14.6 (+3.9)	12.1 (+8.0)	18.1 (+11.6) ∗	12.3 (+3.5) ∗

Appendix

Table 6: Rescue on the questions where plain majority voting scores zero, by setting (%). The protocol is identical to Table 1 — frozen cross-benchmark transfer, one uniform regularizer, deterministic single pick from the full pool of

64

, averaged over the same

12

transfer cells per setting — restricted to the failed questions of each test set. Parenthesized values are question-paired gains over a random pick in the same column, a star marks a clustered CI excluding zero, and avg columns are unweighted means over a model’s three effort settings. Individual settings hold only

6

–

64

failed questions, so single cells indicate direction; pooled over all

223

failed questions J64 rescues

17.0\%

against

8.7\%

for a random pick (

+8.3

∗

[+5.3,+11.3]

) and R64 reaches

14.1\%

(

+5.3

∗

[+2.3,+8.5]

), while length falls to

5.4\%

(

-3.4

∗

[-6.4,-0.1]

Diagnosed process	J64 axes	Routing component	Intervention experts	Predicted change	Observed change
Excessive case splitting, failure to commit	case-splitting bundle #20/#7/#16	experts ranked by write-vector advantage on the bundle direction, layers $8$ – $19$	top two per layer, state-dependent	more splitting, less commitment	“Case $n$ ” enumeration ( $100$ headers in one run; $0$ of $160$ sham runs), non-termination (cap $0.98$ ), accuracy $\rightarrow 0.000$
Hovering and probing instead of operating	hover #22, requirement #1, probe #8 vs operation #47	expert group whose gated usage separates incorrect from correct trajectories (within-question, length-controlled)	L12-E23, L12-E12, L12-E24, L16-E11, L20-E0; suppressed; $1.2\%$ of gated mass	less probing, faster exact execution	shorter generations ( $-45.9$ ∗ ; sham $+47.6$ ∗ ), symbolic derivation replaces numerical guessing, answer becomes correct

Appendix

Table 7: The two mechanism-matched interventions of RQ5, from the diagnosed process to the observed change.

Arm	twelve	three
	problems	case studies
no-op	$0.380$	$0.542$
promote pos. ( $0.3$ )	0.411 (+3.1)	—
suppress neg. ( $0.8$ )	$0.406$ (+2.6)	0.625 (+8.3)
suppress neg. ( $0.3$ )	$0.396$ (+1.6)	—
promote pos. ( $0.8$ )	$0.396$ (+1.6)	$0.604$ (+6.2)

Appendix

Table 8: Accuracy of every intervention arm on the two samples (gpt-oss-20b, Low effort,

16

rollouts per problem per arm;

1{,}344

and

240

trajectories). Parenthesized values are points over no-op, green above it and red below. Every treatment arm evaluated on a given sample is at or above no-op; the sham is the only arm below it. A constant is subtracted from, or added to, the router logits of the selected experts at every token; on the three case studies accuracies are multiples of

1/48

, so the arm-level intervals there are too coarse to separate the arms and only the direction is read. A trigger-gated variant that suppresses only at tokens where a targeted expert is naturally selected reproduces the twelve-problem result (

0.401

against

0.406

Mixture-of-Experts Interpretability

See all 18 papers

Jun 22, 2026

Does the Same Token Mean the Same State? MoE Routing as Signal for Reasoning Control

In sparse Mixture-of-Experts language models, does the same token id imply the same router state and the same experts producing it? Holding the emitted token id fixed at repeated anchors, we find it does not: the experts that produce it still separate task context, trajectory history, and reasoning-effort mode. This residual structure supports test-time control: near boundary anchors (the final-response transition) and delimiter anchors (which open the answer, e.g.\ \texttt{\textbackslash boxed{} or code fences), routing neighborhoods already align with final-answer basins at a marker-only readout and strongest when the routing is read at the answer opening. We operationalize this as RAD (Routing Agreement Decoding), an answer-string-free multi-rollout selector: it locates a fixed anchor, represents each rollout by its anchor-window MoE routing states, and returns the densest Weighted-Jaccard

K

-NN route-basin center, without parsing, normalizing, executing, or voting over answer strings. Across 10 sparse-MoE configurations (gpt-oss, Qwen3-MoE) and 6 datasets spanning math, GPQA, and code, RAD is on par with Majority where string voting is well-posed, with small positive paired deltas (RAD

73.9

/ RAD+DC

74.2

vs.\ Majority

73.6

). Like majority voting, RAD is not a verifier: a dense wrong basin can still win. Its value is the interface: the same selector gives direct pass@1 on code, where exact-string voting is ill-defined, and the same routing-density principle, re-anchored to the agentic boundary, improves best-of-16 patch selection on SWE-bench Verified over random, where patches have no answer string to vote on.

Kang Chen, Minshen Yu, Junjie Nian +3

Fudan University · Shanghai Innovation Institute

Mixture-of-Experts Language Models LLM Routing Mixture-of-Experts Expert Specialization

Ask PDF

Oct 14, 2026

Selective Reasoning for Typed Decision Models: When to Start, Read, and Stop

Typed decision models need an interface that can use additional reasoning while retaining candidate outputs. We train a shared candidate-answer reader on a frozen Qwen3.5-4B backbone to unify direct decisions with on-demand, interruptible reasoning. The reader returns distributions from empty and partial reasoning prefixes. A benefit gate starts reasoning, a natural-state trigger requests explicit reads, and an entropy rule accepts acquired answers and cancels generation. Measurements distinguish answer-query work, generated output, and pipeline time. Across 768 questions from ProofWriter, LogiQA 2.0, and ARC-Challenge, accuracy rises from 70.96% for direct reading to 84.38%. Relative to 512-token terminal reading, accuracy improves by 7.29 percentage points with 2.25% more mean output work. Relative to 1,024-token terminal reading, pipeline time falls by 46.30% with an accuracy difference of -1.17 points. Exact-prefix controls isolate the saving from interruption, while read-scheduling controls distinguish reduced observation work from shorter response time.

Bin Zheng, Zheng Chen

University of Electronic Science and Technology of China

Efficient Language Model Reasoning LLM Decision-Making LLM Inference Logical Reasoning

Ask PDF

Jul 24, 2026

J-CoT: Chain-of-Thought in J-Space

Chain-of-thought prompting improves language-model reasoning by carrying intermediate states across successive computation steps. However, relying on natural language as the only recurrent interface is overly restrictive, since many transient computations do not need to be fully verbalized. Existing latent-reasoning methods remove this constraint by recurrently propagating continuous hidden states. However, these methods pass a dense hidden vector as a whole, without an explicit mechanism for selecting and organizing the information needed by the next reasoning step. This motivates an intermediate interface that remains linguistically grounded without requiring a decoded sentence. We introduce J-CoT, a recurrent reasoning framework built on J-space, a vocabulary-indexed coordinate system within the model's hidden representations. Within each cycle, the model computes in its full hidden space. At the cycle boundary, J-CoT expresses the intermediate state as vocabulary-indexed coefficients, carries these coefficients forward as a J-thought, and maps them back into the model's hidden representation for the next cycle. J-CoT therefore requires neither a fluent intermediate rationale nor recurrence over the complete hidden state. Under matched backbone and inference settings, J-CoT-Zero matches or exceeds the strongest evaluated latent-reasoning baseline on every benchmark, while J-CoT-Train obtains the highest score across the evaluated mathematical, scientific, coding, and structured path-reasoning tasks.

Junde Wu, Jiayuan Zhu, Fengling Liu +2

University of Oxford · Stanford University

LLM Reasoning Latent Reasoning

Ask PDF

Abstract

Mixture-of-Experts Interpretability

Explore similar work

Does the Same Token Mean the Same State? MoE Routing as Signal for Reasoning Control

Selective Reasoning for Typed Decision Models: When to Start, Read, and Stop

J-CoT: Chain-of-Thought in J-Space