Organizations: School of Computing and Data Science, The University of Hong Kong, Hong Kong SAR, China · Department of Computing, The Hong Kong Polytechnic University, Hong Kong SAR, China · School of Data Science, Lingnan University, Hong Kong SAR, China · Centre for Learning, Teaching and Technology, The Education University of Hong Kong, Hong Kong SAR, China
Large reasoning models often produce reasoning traces with verification, revision, and backtracking. When reflection merely re-checks established results, it wastes reasoning tokens and increases latency. Most existing reflection steering methods add a label-derived mean-difference direction across preset layers, but its entanglement with reasoning and length signals destabilizes the accuracy-efficiency trade-off. In this paper, we propose Reflection Steering, a training-free framework for controlling reflection-associated computation within LLMs by disentangling reflection-related activations from general reasoning. Specifically, we contrast reflective and non-reflective hidden states at each LLM layer, denoise the resulting reflection directions with PCA, and orthogonalize them against general-reasoning directions. To limit downstream amplification from early-layer interventions, we calibrate each layer across multiple intervention strengths on a small set, retain only stable layers, and apply bounded projection removal to their residual-stream activations. We conduct extensive experiments across two public benchmarks and three open-weight LLMs against state-of-the-art activation-steering baselines. Results show that Reflection Steering reduces reasoning tokens by 16.9% on average across six matched settings. Besides, our method further introduces a bounded reflection intervention-strength parameter α, enabling deployment-time adjustment to balance token savings, accuracy, and generation stability.
Figures & tables
Figure 2: Reflection Steering overview. Stage 1 estimates a raw reflection direction from reflective and non-reflective residual states. Stage 2 purifies the direction with PCA denoising and orthogonalization against a general-reasoning direction. Stage 3 tests candidate layers over several intervention strengths and keeps only stable responders. Stage 4 applies bounded projection removal at the selected layers during decoding.
Figure 3: Downstream amplification of residual-stream interventions. Median final-layer gain over five teacher-forced MATH traces; ticks below G=1 mark calibrated layers.
Benchmark
Method
Tok. ↓
Δ Tok. (%) ↑
Acc. (%) ↑
Δ Acc. (pp) ↑
ρ↑
Coll. (%) ↓
MATH-500
Raw model
4679 ± 67
—
95.1 ± 0.1
—
—
0.0
CREST
5052 ± 127
−8.0
94.8 ± 0.3
−0.3
−26.7
0.2
ReflCtrl ( λ=−0.48 )
4031 ± 23
13.9
94.1 ± 0.8
−1.0
13.9
0.3
Reflection Steering
3657 ± 11
21.8
94.9 ± 0.8
−0.1
218.0
0.0
MATH-350 (disjoint)
Raw model
4796 ± 75
—
95.3 ± 0.2
—
—
0.0
CREST
5192 ± 148
−8.3
95.3 ± 0.3
0.0
—
0.0
Table 1: Performance comparison on Qwen3-30B-A3B. Thinking-token and accuracy values are three-seed means ± standard deviations. Reflection Steering uses a reflection intervention strength of α=0.7 ; CREST ( Zhang et al. 2025b ) and ReflCtrl ( Yan et al. 2025 ) use reproduced defaults. Gray raw-model rows are references; bold marks the best controlled value per column, with only Reflection Steering bold on ties. “—” denotes an unavailable or undefined entry.
Benchmark
α
Tok. ↓
Δ Tok. (%) ↑
Acc. (%) ↑
Coll. (%) ↓
MATH-500
1.0
4679
—
95.1
0.0
0.7
3657
21.8
94.9
0.0
0.3
3292
29.6
94.6
0.2
0.0
3439
26.5
93.5
1.1
MATH-350 (disjoint)
1.0
4796
—
95.3
0.0
0.7
3674
23.4
95.4
0.0
Table 2: Performance across intervention strengths. Gray α=1.0 rows are raw-model references; the remaining rows report intervened settings within each benchmark under matched accounting. The deployed setting is α=0.7 .
Variant
Tok. ↓
Δ Tok. ↑
Acc. ↑
Coll. ↓
Panel A: direction purification
Raw model
7250
—
70.7
0.0
PCA + orthogonalization
5737
20.9
70.7
0.0
No orthogonalization
4104
43.4
67.2
0.0
No PCA
6105
15.8
71.2
0.0
Panel B: layer calibration and projection removal
Table 3: Ablations on GPQA-Diamond. Full 198-question split, α=0.7 .
Figure 4: Reasoning-subspace overlap of steering directions. Medians over 14 layers; bars span the 5th–95th percentiles. RS is Reflection Steering; RS (Raw) is its unpurified mean-difference direction. Random is an isotropic negligible-overlap reference. PCA retains substantial reasoning-proxy overlap, whereas purification cuts it by 98.8% to the isotropic-random level. CREST has no comparable single direction.
Model
Benchmark
Method
Tok. ↓
Δ Tok. (%) ↑
Acc. ↑
Qwen3-8B
MATH-500
Raw model
4345
—
93.6
ReflCtrl
4449
−2.4
93.6
RS
4060
6.6
93.2
Qwen3-8B
GPQA-D
Raw model
8485
—
57.6
ReflCtrl
8292
2.3
56.1
RS
7670
9.6
56.1
Table 4: Cross-model results. RS denotes Reflection Steering. All rows use matched thinking-token accounting.
Figure 5: Direction stability versus construction budget. Median 1−cos(dN,dN+10) over ten random orderings; the band spans the 5th–95th percentile.
Extending CoT through RL has been widely used to enhance the reasoning capabilities of LLMs. However, due to the sparsity of reward signals, it can also induce undesirable thinking patterns such as overthinking, i.e., generating redundant intermediate reasoning content. In this work, we argue that a major source of such redundancy is inefficient reflection, which often manifests in two problematic patterns: Indiscriminate Reflection, where the model performs broad, low-impact checks throughout reasoning, and Repetitive Reflection, where it repeatedly re-verifies an already established conclusion. To address this, we introduce a graph-based CoT optimization framework. Specifically, we convert each linear CoT into a directed acyclic graph (DAG) with explicit dependency edges, and design a dual pruning strategy: branch-level pruning removes weakly contributing reflection branches, while depth-level pruning eliminates late-stage re-verification. We distill this behavior via a three-stage pipeline: (1) SFT to initialize the policy on pruned concise traces, (2) DPO to prefer correct but less redundant trajectories, and (3) GRPO with length penalty to jointly optimize answer correctness and efficiency. Experiments show that our approach reduces the average reasoning tokens by 42% while maintaining or improving accuracy.
Hongyuan Yuan, Xinran He, Run Shao +6
School of Geosciences and Info-Physics, Central South University, Changsha, China · Baidu Inc., Beijing, China
Reasoning LLMs trained with long chain-of-thought often overthink: they spend tokens on redundant reflection and transitions that inflate cost without improving accuracy. Static activation steering (e.g.\ SEAL) suppresses such content with a fixed vector, but applies the same strength regardless of how redundant the current chunk actually is. We describe PID-steering, a training-free, decoding-time method that modulates the steering strength with a PID controller driven by a lightweight chunk-level redundancy classifier. On a subset of GSM8K with DeepSeek-R1-Distill-Qwen-1.5B, the method improves accuracy from 85.7% to 89.6% (+3.9 pp) while cutting average output length from 1026 to 790 tokens (−23%). We report it as a small-scale proof of concept rather than a benchmark result.
Large language models improve final-answer accuracy through extended chain-of-thought reasoning, but often spend tokens inefficiently and offer little inference-time control. Existing efficient reasoning methods control thinking length by shortening, early-stopping, or compressing traces, leaving how the model thinks implicit. In this paper, we propose Agentic Chain-of-Thought Steering (ACTS), which formulates reasoning steering as a Markov decision process where a controller agent adaptively steers a frozen reasoner during inference. At each step, the controller observes the reasoning trace and remaining thinking budget, then issues a steering action consisting of a reasoning strategy and a steering phrase that initiates the next reasoner step. This enables budget-aware strategy control for efficient reasoning while preserving the reasoner's generation continuity. We initialize the controller agent from our constructed synthetic steering trajectories with multi-budget augmentation, and further optimize it via reinforcement learning with budget-conditioned reward shaping. Experiments across multiple benchmarks show that ACTS matches full-thinking performance with substantial token savings, and enables controllable accuracy-efficiency trade-offs across different reasoners and tasks. The code is available at https://github.com/Andree-9/ACTS.
Yu Xia, Zhouhang Xie, Xin Xu +4
University of California San Diego1 · Intuit AI Research2