NeuReasoner: Towards Explainable, Controllable, and Unified Reasoning via Mixture-of-Neurons

Haonan Dong; Kehan Jiang; Haoran Ye; Wenhao Zhu; Zhaolu Kang; Guojie Song

cs.CLApr 3, 2026

NeuReasoner: Towards Explainable, Controllable, and Unified Reasoning via Mixture-of-Neurons

Authors: Haonan Dong, Kehan Jiang, Haoran Ye, Wenhao Zhu, Zhaolu Kang, Guojie Song

Organizations: State Key Laboratory of General Artificial Intelligence, School of Intelligence Science and Technology, Peking University · School of Software and Microelectronics, Peking University

Abstract

Large Reasoning Models (LRMs) have recently achieved remarkable success in complex reasoning tasks. However, closer scrutiny reveals persistent failure modes compromising performance and cost: I) Intra-step level, marked by calculation or derivation errors; II) Inter-step level, involving oscillation and stagnation; and III) Instance level, causing maladaptive over-thinking. Existing endeavors target isolated levels without unification, while their black-box nature and reliance on RL hinder explainability and controllability. To bridge these gaps, we conduct an in-depth white-box analysis, identifying key neurons (Mixture of Neurons, MoN) and their fluctuation patterns associated with distinct failures. Building upon these insights, we propose NeuReasoner, an explainable, controllable, and unified reasoning framework driven by MoN. Technically, NeuReasoner integrates lightweight MLPs for failure detection with a special token-triggered self-correction mechanism learned via SFT. During inference, special tokens are inserted upon failure detection to actuate controllable remedial behaviors. Extensive evaluations across six benchmarks, six backbone models (8B~70B) against nine competitive baselines, demonstrate that NeuReasoner achieves performance gains of up to 27.0% while reducing token consumption by 19.6% ~ 63.3%.

Figures & tables

Figure 1: Distribution and illustration of failure modes across different levels using DeepSeek-R1-Distill-Qwen-7B on MATH ( Hendrycks et al., 2021 ) .

Figure 2: ( Upper ) Distribution of key neurons across distinct failure modes using DeepSeek-R1-Distill-Qwen-7B on MATH. ( Lower ) Time- and frequency-domain analysis of MoN for positive and negative sample pairs.

Figure 3: The overview of our proposed NeuReasoner .

Model	Method	AIME25		MATH500		GSM8K		GPQA-Diamond		LiveCodeBench
		Pass@1 $\uparrow$	Token $\downarrow$	Pass@1 $\uparrow$	Token $\downarrow$	Pass@1 $\uparrow$	Token $\downarrow$	Pass@1 $\uparrow$	Token $\downarrow$	Pass@1 $\uparrow$	Token $\downarrow$
DeepSeek-R1-Distill-Qwen-7B	Vanilla	43.3	11454	91.8	2887	92.4	442	49.2	8016	36.9	9072
	DAST	41.1	7904	91.6	1330	91.8	301	48.8	3635	36.8	4280
	Think or Not	38.9	4760	92.0	1103	92.9	264	47.0	3390	36.4	7397
	AlphaOne	44.4	8921	92.5	3791	93.4	459	50.5	8591	37.5	8177
	RL + LP	43.3	6255	93.4	1322	92.5	291	50.3	3209	36.2	6293

Table 1: Main results. Best results are highlighted in bold , with runners-up underlined .

Figure 5: Test-time scalability under self-consistency.

Model	Method	MATH500		GPQA
		Pass@1 $\uparrow$	Token $\downarrow$	Pass@1 $\uparrow$	Token $\downarrow$
DS-R1-Distill Qwen-32B	w/o $\text{MLP}_{\text{all}}$	93.5	2469	63.9	6010
	w/o $\text{MLP}_{\text{intra}}$	94.5	1412	64.4	3441
	w/o $\text{MLP}_{\text{inter}}$	94.7	1434	64.9	3219
	w/o $\text{MLP}_{\text{inst}}$	95.8	2389	65.4	6118
	NeuReasoner	96.3	1318	66.2	3274

Table 2: Ablation study of NeuReasoner .

Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

Notation	Description
$L$	The total number of layers in the Large Language Model (LLM).
$l^{*}$	The index of the target intermediate layer for analysis, set to $l^{*}=L/2$ .
$\mathcal{C}$	The set of all neurons, including both FFN neurons and attention heads, within the target layer $l^{*}$ .
$y_{t}$	The target token generated at time step $t$ .
$\mathbf{w}_{y_{t}}$	The vector representation of the Language Model (LM) head corresponding to token $y_{t}$ .
$\mathbf{h}_{dec}^{(l^{*})}(t,c)$	The decomposed hidden state associated with neuron $c$ at time step $t$ in layer $l^{*}$ .

Appendix

Table 3: Comprehensive summary of notations and definitions used in this paper.

Failure Mode	Accuracy	Recall
$\text{MLP}_{\mathrm{intra}}$	$0.871\pm 0.010$	$0.848\pm 0.012$
$\text{MLP}_{\mathrm{inter}}$	$0.862\pm 0.011$	$0.936\pm 0.008$
$\text{MLP}_{\mathrm{inst}}$	$0.944\pm 0.006$	$0.951\pm 0.007$

Appendix

Table 4: Experimental results across different MLPs.

	R1-Qwen-7B		R1-Qwen-32B		Qwen3-8B-thinking		Qwen3-32B-thinking		R1-Llama-8B		R1-Llama-70B
Method	Pass@1	Token	Pass@1	Token	Pass@1	Token	Pass@1	Token	Pass@1	Token	Pass@1	Token
Vanilla	54.4	10438	70.0	7873	70.0	11125	77.8	10677	45.6	10798.9	68.9	7766
DAST	55.6	7258	70.0	5802	67.8	5964	77.8	5981	45.6	8246	67.8	5115
Think or Not	52.2	4341	67.8	3993	68.9	5387	76.7	6173	44.4	6761	71.1	4005
AlphaOne	55.6	8224	72.2	8210	73.3	8343	78.9	8007	47.8	8339	72.2	7873
RL + LP	52.2	5693	71.1	5492	71.1	6986	76.7	6238	45.6	5333	66.7	5304

Appendix

Table 5: Additional results on the AIME24 dataset. Best results are bold , runners-up underlined .

Model	Method	AIME24	AIME25	MATH500	GSM8K	GPQA-Diamond	LiveCodeBench
DeepSeek-R1-Qwen-7B	Vanilla	02:13	02:12	04:11	02:26	04:52	24:20
DeepSeek-R1-Qwen-7B	NeuReasoner	02:24	02:23	04:29	02:38	05:13	25:51
DeepSeek-R1-Qwen-32B	Vanilla	04:00	04:10	07:06	01:45	07:29	36:36
DeepSeek-R1-Qwen-32B	NeuReasoner	04:17	04:28	07:35	01:54	07:59	38:51
Qwen3-8B-thinking	Vanilla	02:26	02:28	06:24	05:47	04:12	24:56
Qwen3-8B-thinking	NeuReasoner	02:38	02:40	06:50	06:11	04:30	26:29

Appendix

Table 6: Inference efficiency comparison (Time in MM:SS).

Hyper	Value	Pass@1	Token
$k_{\text{intra}}$	2	96.0	1360
	4	96.3	1318
	8	96.2	1335
$k_{\text{inter}}$	2	96.1	1345
	4	96.3	1318
	8	96.4	1328

Appendix

Table 7: Hyperparameter sensitivity on MATH500 with DeepSeek-R1-Distill-Qwen-32B.

Large Reasoning Models

Large reasoning models are language models trained or prompted to solve difficult tasks through extended, structured computation.

See all 338 papers

Dissecting Failure Dynamics in Large Language Model Reasoning
Apr 16, 2026Wei Zhu, Jian Zhang, Lixing Yu +2Inference-Time Guidance LLM Reasoning
School of Information Science and Engineering, Yunnan University, Kunming, China · Yunnan Key Laboratory of Intelligent Systems and Computing, Kunming, China
Stop Before You Fail: Operational Capability Boundaries for Mitigating Unproductive Reasoning in Large Reasoning Models
Sep 29, 2025Qingjie Zhang, Yujia Fu, Yang Wang +5Efficient Language Model Reasoning Efficient Language Model Inference
Tsinghua University · 2Ant Group
TRUE: A Trustworthy Unified Explanation Framework for Large Language Model Reasoning
Feb 21, 2026Yujiao YangLLM Interpretability Faithfulness of Language Model Explanations
Dalian University of Technology, Dalian, China

Apr 16, 2026J/K move · Enter open · S save

Dissecting Failure Dynamics in Large Language Model Reasoning

Wei Zhu, Jian Zhang, Lixing Yu, Kun Yue+1

School of Information Science and Engineering, Yunnan University, Kunming, China · Yunnan Key Laboratory of Intelligent Systems and Computing, Kunming, China

Inference-Time Guidance LLM Reasoning

Open paper AskPDF arXiv

Large Language Models (LLMs) achieve strong performance through extended inference-time deliberation, yet how their reasoning failures arise remains poorly understood. By analyzing model-generated reasoning trajectories, we find that errors are not uniformly distributed but often originate from a small number of early transition points, after which reasoning remains locally coherent but globally incorrect. These transitions coincide with localized spikes in token-level entropy, and alternative continuations from the same intermediate state can still lead to correct solutions. Based on these observations, we introduce GUARD, a targeted inference-time framework that probes and redirects critical transitions using uncertainty signals. Empirical evaluations across multiple benchmarks confirm that interventions guided by these failure dynamics lead to more reliable reasoning outcomes. Our findings highlight the importance of understanding when and how reasoning first deviates, complementing existing approaches that focus on scaling inference-time computation.

Abstract

Large Reasoning Models

Explore similar work

Dissecting Failure Dynamics in Large Language Model Reasoning

Stop Before You Fail: Operational Capability Boundaries for Mitigating Unproductive Reasoning in Large Reasoning Models

TRUE: A Trustworthy Unified Explanation Framework for Large Language Model Reasoning