Organizations: Institute of Information Engineering, Chinese Academy of Sciences, Beijing, China · School of Cyber Security, University of Chinese Academy of Sciences, Beijing, China · Baidu Inc., Beijing, China
Even successful long CoTs may contain harmful supervision, motivating the need for methods that can automatically identify and mask such signals during SFT. Inspired by the MicroMouse maze-solving process, we view a long CoT as a search trajectory from the input problem toward the target answer. This perspective motivates maintaining a goal-relative measure, analogous to a MicroMouse's spatial coordinates, to quantify how close the current reasoning state is to the target answer. Building on this perspective, we propose MicroMouse, an automated selective-SFT framework for masking harmful supervision in long CoTs. In MicroMouse we introduce a Relative Potential (REP) to measure how closely each reasoning state supports the gold answer. Unlike the perplexity of the gold answer alone, REP measures relative support by contrasting the gold answer with a matched foil. This contrast cancels likelihood variations shared by the pair, yielding a more stable signal of progress toward the target answer. By tracking the REP trajectory, MicroMouse identifies consecutive potential decreases as erroneous exploration and post-peak reasoning as redundant wandering, and masks these harmful supervision during SFT. Extensive experiments across multiple student model scales and reasoning benchmarks show that MicroMouse consistently outperforms Full-CoT SFT and existing selective-SFT baselines, yielding average relative gains of 5.5% in greedy accuracy and 2.6% in Pass@1 relative to Full-CoT SFT. Meanwhile, MicroMouse improves reasoning efficiency by reducing average output length by 8.8%, while identifying harmful supervision in only 1.7 seconds per long CoT on average.
Figures & tables
Figure 1: Overview of MicroMouse , a framework consisting of three stages.
Setting
Metric
Easy
Hard
Total Avg.
Level 1
Level 2
Level 3
Avg.
Level 4
Level 5
Avg.
Full CoT SFT
Pass@1
89.1
93.9
91.0
91.8
82.6
73.1
77.7
84.4
Pass@6
95.3
98.9
99.0
98.3
95.3
90.3
92.7
95.4
MicroMouse
Pass@1
92.6
92.0
92.1
92.2 ↑ 0.4%
85.8
77.2
81.4 ↑ 4.8%
86.5 ↑ 2.5%
Pass@6
100.0
98.9
98.1
98.7 ↑ 0.4%
96.1
91.0
93.5 ↑ 0.9%
96.0 ↑ 0.6%
Table 2: Qwen3-8B temperature sampling performance across MATH500 difficulty levels.
Figure 2: Comparison with other selective-SFT baselines on Qwen3-8B. The top row reports reasoning performance under greedy decoding and temperature sampling, while the bottom row compares output length and the computational cost of masking harmful supervision.
Greedy
Sampling
Variant
Keep (%)
Acc.
Len. (k)
P@1
P@6
Len. (k)
Full CoT SFT
100.0
41.0
18.94
48.6
68.9
14.06
Random Mask
80.9
41.7
17.89
49.1
68.2
14.01
MicroMouse
80.9
45.1
17.22
50.9
70.3
12.88
w/o Mask Serror
83.5
44.4
16.67
50.7
70.2
12.84
w/o Mask Sredundant
97.1
38.9
19.34
49.0
69.0
14.11
Table 3: Ablation studies on Qwen3-8B.
Figure 3: Case study of step-wise supervision analysis on LIMO #539. The figure visualizes the Relative Potential trajectory and compares the supervision decisions of MicroMouse with selective-SFT baselines across all reasoning steps.
Long chain-of-thought (CoT) traces are widely used as supervision for reasoning-oriented LLM SFT, yet answer-correct traces can still lead to markedly different fine-tuning outcomes. We study post-conclusion continuation in answer-correct long-CoT data: a continuation where the answer appears sufficiently supported, but the trace continues with additional reasoning that remains in the supervised target. To test its training effect, we use a delete-only editor to construct answer-preserving suffix removal and compare CoT-based SFT on the original and processed traces. We observe improved SFT outcomes after removing the editor-identified post-conclusion continuation, suggesting that this continuation is harmful to training in our setting. We therefore refer to this empirically supported phenomenon as harmful continuation. Beyond this intervention, we further characterize the removed post-conclusion continuation through uncertainty and hidden-state progress. We observe persistent local uncertainty together with weakened terminal-directional progress, forming an uncertainty--geometry mismatch. Finally, we instantiate Harmful Continuation Cut (HCC), a lightweight boundary proxy that approximates the editor-identified post-conclusion continuation boundary.
Chen He, Yuhao Wu, Lei Wang +2
University of Electronic Science and Technology of China · Singapore University of Technology and Design · Singapore Management University
Chain-of-thought (CoT) reasoning allows humans to inspect how large language models reach their answers, and oversee model behaviour. This reasoning comes at an increased inference cost, motivating efficient methods that train models to solve tasks using fewer tokens. However, a common concern is that such training may cause models to skip important reasoning steps, so the CoT no longer faithfully reflects the model's decision. It is unclear whether or when this occurs in practice, since different efficiency methods apply length pressure to models' CoT in distinct ways, and faithfully explaining a model's decision takes more tokens on some tasks than others. To understand these dynamics, we fine-tune a variety of models with three methods that apply length pressure differently, namely a fixed generation budget, a per-example length target, and a group-relative length reward. We evaluate how efficient reasoning affects CoT faithfulness (i.e., how well the CoT reflects model decisions on related inputs) and monitorability (i.e., whether the CoT reveals when input interventions alter the output). We find that it affects faithfulness and monitorability differently. Faithfulness falls in most settings, primarily because the trained models are less consistent. Monitorability is more robust, as models keep acknowledging the influence on their answer even when the CoT is much shorter.
Samuel Lewis-Lim, Xingwei Tan, Mario Sanger +2
School of Computer Science, University of Sheffield · AstraZeneca
Despite remarkable performance on complex tasks, Large Reasoning Models (LRMs) often generate excessively long Chain-of-Thoughts (CoT), inflating computational costs even for simple queries. Existing efforts to mitigate this inefficiency typically rely on discrete reasoning modes or fixed budget tiers, lacking a principled criterion of when reasoning is sufficient. In this work, we introduce Minimal Sufficient CoT (MSC), defined as the shortest prefix of a CoT trajectory which is adequate for producing the correct answer. We empirically show that MSC not only reduces reasoning tokens, but also improves accuracy across difficulty levels. Building on MSC, we propose Sufficiency-guided Continuous Adaptive Reasoning (SuCo), a two-stage training framework for autonomous reasoning control along a continuous spectrum. In stage 1, MSC-Aligned Fine-Tuning (MFT) constructs MSC data using problem-adaptive sufficiency thresholds that naturally scale with question difficulty, then fine-tunes the model to internalize concise yet sufficient reasoning patterns. In stage 2, Sufficiency-Aware Policy Optimization (SAPO) further optimizes the model through reinforcement learning with dynamic complexity tracking and sufficiency-aware rewards that penalize both over- and under-thinking. Extensive experiments across mathematics, code, and science benchmarks show that SuCo consistently achieves improvements in both accuracy and reasoning efficiency.
Jiahao Wang, Bingyu Liang, Chenhao Hu +5
Harbin Institute of Technology (Shenzhen), Guangdong, China. · TeleAI of China Telecom, China.