Reflection Steering: Disentangling Reflection from Reasoning in Activation Space for Token-Efficient Inference

Jiarui Hu; Zhiyuan Wen; Xiaoyun Liu; Jiaxing Shen; Yu Yang

cs.LGAug 26, 2026

Reflection Steering: Disentangling Reflection from Reasoning in Activation Space for Token-Efficient Inference

Authors: Jiarui Hu, Zhiyuan Wen, Xiaoyun Liu, Jiaxing Shen, Yu Yang

Organizations: School of Computing and Data Science, The University of Hong Kong, Hong Kong SAR, China · Department of Computing, The Hong Kong Polytechnic University, Hong Kong SAR, China · School of Data Science, Lingnan University, Hong Kong SAR, China · Centre for Learning, Teaching and Technology, The Education University of Hong Kong, Hong Kong SAR, China

Abstract

Large reasoning models often produce reasoning traces with verification, revision, and backtracking. When reflection merely re-checks established results, it wastes reasoning tokens and increases latency. Most existing reflection steering methods add a label-derived mean-difference direction across preset layers, but its entanglement with reasoning and length signals destabilizes the accuracy-efficiency trade-off. In this paper, we propose Reflection Steering, a training-free framework for controlling reflection-associated computation within LLMs by disentangling reflection-related activations from general reasoning. Specifically, we contrast reflective and non-reflective hidden states at each LLM layer, denoise the resulting reflection directions with PCA, and orthogonalize them against general-reasoning directions. To limit downstream amplification from early-layer interventions, we calibrate each layer across multiple intervention strengths on a small set, retain only stable layers, and apply bounded projection removal to their residual-stream activations. We conduct extensive experiments across two public benchmarks and three open-weight LLMs against state-of-the-art activation-steering baselines. Results show that Reflection Steering reduces reasoning tokens by 16.9% on average across six matched settings. Besides, our method further introduces a bounded reflection intervention-strength parameter $α$ , enabling deployment-time adjustment to balance token savings, accuracy, and generation stability.

Figures & tables

Figure 2: Reflection Steering overview. Stage 1 estimates a raw reflection direction from reflective and non-reflective residual states. Stage 2 purifies the direction with PCA denoising and orthogonalization against a general-reasoning direction. Stage 3 tests candidate layers over several intervention strengths and keeps only stable responders. Stage 4 applies bounded projection removal at the selected layers during decoding.

Figure 3: Downstream amplification of residual-stream interventions. Median final-layer gain over five teacher-forced MATH traces; ticks below

G=1

mark calibrated layers.

Benchmark	Method	Tok. $\downarrow$	$\Delta$ Tok. (%) $\uparrow$	Acc. (%) $\uparrow$	$\Delta$ Acc. (pp) $\uparrow$	$\rho$ $\uparrow$	Coll. (%) $\downarrow$
MATH-500	Raw model	4679 $\pm$ 67	—	95.1 $\pm$ 0.1	—	—	0.0
	CREST	5052 $\pm$ 127	$-8.0$	94.8 $\pm$ 0.3	$-0.3$	$-26.7$	0.2
	ReflCtrl ( $\lambda{=}{-}0.48$ )	4031 $\pm$ 23	13.9	94.1 $\pm$ 0.8	$-1.0$	13.9	0.3
	Reflection Steering	3657 $\pm$ 11	21.8	94.9 $\pm$ 0.8	$-0.1$	218.0	0.0
MATH-350 (disjoint)	Raw model	4796 $\pm$ 75	—	95.3 $\pm$ 0.2	—	—	0.0
	CREST	5192 $\pm$ 148	$-8.3$	95.3 $\pm$ 0.3	$0.0$	—	0.0

Table 1: Performance comparison on Qwen3-30B-A3B. Thinking-token and accuracy values are three-seed means

\pm

standard deviations. Reflection Steering uses a reflection intervention strength of

\alpha=0.7

; CREST ( Zhang et al. 2025b ) and ReflCtrl ( Yan et al. 2025 ) use reproduced defaults. Gray raw-model rows are references; bold marks the best controlled value per column, with only Reflection Steering bold on ties. “—” denotes an unavailable or undefined entry.

Benchmark	$\alpha$	Tok. $\downarrow$	$\Delta$ Tok. (%) $\uparrow$	Acc. (%) $\uparrow$	Coll. (%) $\downarrow$
MATH-500	1.0	4679	—	95.1	0.0
	0.7	3657	21.8	94.9	0.0
	0.3	3292	29.6	94.6	0.2
	0.0	3439	26.5	93.5	1.1
MATH-350 (disjoint)	1.0	4796	—	95.3	0.0
	0.7	3674	23.4	95.4	0.0

Table 2: Performance across intervention strengths. Gray

\alpha=1.0

rows are raw-model references; the remaining rows report intervened settings within each benchmark under matched accounting. The deployed setting is

\alpha=0.7

Variant	Tok. $\downarrow$	$\Delta$ Tok. $\uparrow$	Acc. $\uparrow$	Coll. $\downarrow$
Panel A: direction purification
Raw model	7250	—	70.7	0.0
PCA + orthogonalization	5737	20.9	70.7	0.0
No orthogonalization	4104	43.4	67.2	0.0
No PCA	6105	15.8	71.2	0.0
Panel B: layer calibration and projection removal

Table 3: Ablations on GPQA-Diamond. Full 198-question split,

\alpha=0.7

Figure 4: Reasoning-subspace overlap of steering directions. Medians over 14 layers; bars span the 5th–95th percentiles. RS is Reflection Steering; RS (Raw) is its unpurified mean-difference direction. Random is an isotropic negligible-overlap reference. PCA retains substantial reasoning-proxy overlap, whereas purification cuts it by 98.8% to the isotropic-random level. CREST has no comparable single direction.

Model	Benchmark	Method	Tok. $\downarrow$	$\Delta$ Tok. (%) $\uparrow$	Acc. $\uparrow$
Qwen3-8B	MATH-500	Raw model	4345	—	93.6
		ReflCtrl	4449	$-2.4$	93.6
		RS	4060	6.6	93.2
Qwen3-8B	GPQA-D	Raw model	8485	—	57.6
		ReflCtrl	8292	2.3	56.1
		RS	7670	9.6	56.1

Table 4: Cross-model results. RS denotes Reflection Steering. All rows use matched thinking-token accounting.

Figure 5: Direction stability versus construction budget. Median

1-\cos(d_{N},d_{N+10})

over ten random orderings; the band spans the 5th–95th percentile.

Language Model Steering

Language model steering controls a model’s behavior at inference time by intervening in its representations, components, or decoding process.

See all 281 papers

Graph-Based Chain-of-Thought Pruning for Reducing Redundant Reflections in Reasoning LLMs
Apr 7, 2026Hongyuan Yuan, Xinran He, Run Shao +6Efficient Language Model Reasoning Overthinking in Language Models
School of Geosciences and Info-Physics, Central South University, Changsha, China · Baidu Inc., Beijing, China
Adaptive Activation Steering for Efficient LLM Reasoning via Closed-Loop PID Control
Jun 23, 2025Aryasomayajula Ram BharadwajLanguage Model Steering Efficient Language Model Inference
Independent Researcher
Agentic Chain-of-Thought Steering for Efficient and Controllable LLM Reasoning
Jun 2, 2026Yu Xia, Zhouhang Xie, Xin Xu +4Language Model Steering CoT Reasoning
University of California San Diego1 · Intuit AI Research2

Apr 7, 2026J/K move · Enter open · S save

Graph-Based Chain-of-Thought Pruning for Reducing Redundant Reflections in Reasoning LLMs

Hongyuan Yuan, Xinran He, Run Shao, Bolei He+5

School of Geosciences and Info-Physics, Central South University, Changsha, China · Baidu Inc., Beijing, China

Efficient Language Model Reasoning Overthinking in Language Models RL for Language Model Reasoning

Open paper AskPDF arXiv

Extending CoT through RL has been widely used to enhance the reasoning capabilities of LLMs. However, due to the sparsity of reward signals, it can also induce undesirable thinking patterns such as overthinking, i.e., generating redundant intermediate reasoning content. In this work, we argue that a major source of such redundancy is inefficient reflection, which often manifests in two problematic patterns: Indiscriminate Reflection, where the model performs broad, low-impact checks throughout reasoning, and Repetitive Reflection, where it repeatedly re-verifies an already established conclusion. To address this, we introduce a graph-based CoT optimization framework. Specifically, we convert each linear CoT into a directed acyclic graph (DAG) with explicit dependency edges, and design a dual pruning strategy: branch-level pruning removes weakly contributing reflection branches, while depth-level pruning eliminates late-stage re-verification. We distill this behavior via a three-stage pipeline: (1) SFT to initialize the policy on pruned concise traces, (2) DPO to prefer correct but less redundant trajectories, and (3) GRPO with length penalty to jointly optimize answer correctness and efficiency. Experiments show that our approach reduces the average reasoning tokens by 42% while maintaining or improving accuracy.

Abstract

Language Model Steering

Explore similar work

Graph-Based Chain-of-Thought Pruning for Reducing Redundant Reflections in Reasoning LLMs

Adaptive Activation Steering for Efficient LLM Reasoning via Closed-Loop PID Control

Agentic Chain-of-Thought Steering for Efficient and Controllable LLM Reasoning