Reliable Control-Point Selection for Steering Reasoning in Large Language Models

Haomin Zhuang; Hojun Yoo; Xiaonan Luo; Kehan Guo; Xiangliang Zhang

cs.CLApr 2, 2026

Reliable Control-Point Selection for Steering Reasoning in Large Language Models

Authors: Haomin Zhuang, Hojun Yoo, Xiaonan Luo, Kehan Guo, Xiangliang Zhang

Organizations: University of Notre Dame

Abstract

Steering vectors offer a training-free mechanism for controlling reasoning behaviors in large language models, but constructing effective vectors requires identifying genuine behavioral signals in the model's hidden states. For behaviors that can be toggled via prompts, this is straightforward. However, many reasoning behaviors -- such as self-reflection -- emerge spontaneously and resist prompt-level control. Current methods detect these behaviors through keyword matching in chain-of-thought traces, implicitly assuming that every detected boundary encodes a genuine behavioral signal. We show that this assumption is overwhelmingly wrong: across 541 keyword-detected boundaries, 93.3% are behaviorally unstable, failing to reproduce the detected behavior under re-generation from the same prefix. We develop a probabilistic model that formalizes intrinsic reasoning behaviors as stochastic events with context-dependent trigger probabilities, and show that unstable boundaries dilute the steering signal. Guided by this analysis, we propose stability filtering, which retains only boundaries where the model consistently reproduces the target behavior. Combined with a content-subspace projection that removes residual question-specific noise, our method achieves 0.784 accuracy on MATH-500 (+5.0 over the strongest baseline). The resulting steering vectors transfer across models in the same architecture family without re-extraction, improving Nemotron-Research-Reasoning-1.5B (+5.0) and DeepScaleR-1.5B-Preview (+6.0). Code is available at https://github.com/zhmzm/stability-steering.

Figures & tables

Figure 1: Extrinsic behaviors are steered via prompt contrast (left). Intrinsic behaviors emerge spontaneously and resist prompting (middle-left). Keyword matching detects them from traces (middle-right), but most detections are unstable under re-generation (right; data from § 4.3 ).

Figure 2: Overview of our method. Left : SEAL mixes all keyword-detected boundaries, including unstable ones that dilute the signal. Middle : Our content-aware extraction computes per-question steering directions and projects out question-specific information via SVD. Right : Stability probing re-generates from each boundary’s prefix and retains only those that reliably reproduce the target behavior.

Model	Method	Acc	$\Delta$	Avg Tok	Avg R
Source model (vectors extracted here)
Qwen-1.5B	Baseline	0.608	—	1547	39.1
	Prompt text	0.672	+6.4	1116	24.3
	Prompt vector	0.610	+0.2	1604	39.4
	SEAL	0.734	+12.6	1103	10.3
	Ours	0.784	+17.6	1008	10.7

Table 1: Main results on MATH-500 (500 questions, max_tokens=4096, layer 20). Steering vectors extracted from DeepSeek-R1-Distill-Qwen-1.5B.

\Delta

= accuracy change vs. each model’s own baseline. Avg Tok = average word count; Avg R = average reflection keyword count.

Panel of Figure 3 — Figure 3: (a) Distribution of stability scores across 541 R+T boundaries. 93.3% are unstable ( $s(b)<0.8$ ); only 6.7% pass the threshold. (b) Accuracy peaks at $\tau{=}0.8$ (+7.0 over SEAL), then drops at $\tau{=}0.9$ as sample size shrinks. Shaded band shows random baseline $\pm 1\sigma$ .

Figure 4: Behavior probe confidence by stability bin (balanced R+T vs. E, GroupKFold by question). Stable boundaries (

s\geq 0.8

) yield the highest confidence (0.942).

Table 6

Appendix figures & tables2 assets

Supplementary material from the paper’s appendix.

Appendix

$\alpha$	SEAL	Content Removal ( $K{=}4$ )
$-100$	0.426	0.442
$-50$	0.348	0.342
$-20$	0.258	0.256
$-1$	0.192	0.192
Baseline	0.182	0.182
$1$	0.190	0.190

Appendix

Table 4: Full coefficient sweep on MATH-500 (Qwen-1.5B, layer 20, max_tokens=1024 , 500 questions).

$\alpha$	$\mathbf{v}_{\text{stable}}$ ( $\tau{=}0.8$ )	A+B Combined
$-100$	0.496	0.504
$-50$	0.364	0.366
$-20$	0.254	0.260

Appendix

Table 5: Coefficient sweep for

\mathbf{v}_{\text{stable}}

(

\tau{=}0.8

) and A+B combined on Qwen-1.5B (MATH-500, max_tokens=1024 , 500 questions).

Language Model Steering

Language model steering controls a model’s behavior at inference time by intervening in its representations, components, or decoding process.

See all 278 papers

Predicting Future Behaviors in Reasoning Models Enables Better Steering
Jun 9, 2026Evgenii Kortukov, Piotr Komorowski, Florian Klein +5Language Model Steering Representation Steering
1Fraunhofer HHI · 2Northeastern University · 3KAIST
Can We Break LLMs Out of Self-Loops? Fine-Grained Reasoning Control with Activation Steering
Jul 20, 2026Sheldon Yu, Tong Yu, Xunyi Jiang +6Overthinking in Language Models Causal Interventions in Language Models
UC San Diego · Adobe Research · University of New South Wales
Behavior Cue Reasoning: Monitorable Reasoning Improves Efficiency and Safety through Oversight
May 7, 2026Christopher Z. Cui, Taylor W. Killian, Prithviraj AmmanabroluScalable Oversight LLM Safety
University of California, San Diego · 2Brigham Young University

Jun 9, 2026J/K move · Enter open · S save

Predicting Future Behaviors in Reasoning Models Enables Better Steering

Evgenii Kortukov, Piotr Komorowski, Florian Klein, Paula Engl+4

1Fraunhofer HHI · 2Northeastern University · 3KAIST

Language Model Steering Representation Steering Language Model Probing Large Reasoning Models

Open paper AskPDF arXiv

Deployed large reasoning models (LRMs) often behave unexpectedly. Test-time steering controls LRM outputs by intervening on their hidden representations, but it can degrade output quality. We argue that prior steering work implicitly relies on internal features that detect behavior in already generated text. We show that these detection features are poor predictors of future behavioral outcomes, and thus not the natural intervention target. Instead, we train activation probes to predict future behavior likelihoods from intermediate reasoning steps. These probes predict the most likely behavior with 64%-91% accuracy, revealing a separate type of internal prediction features. Building on these prediction features, we introduce a text-level steering method, Future Probe Controlled Generation. FPCG samples multiple candidate sentences and chooses the best one according to a probe predicting the future behavior likelihood. This enables steering with almost no output quality degradation. FPCG also enables steering in several evaluations where activation steering fails. These results show that distinguishing detection and prediction features enables a more nuanced approach to controlling LRM behaviors.

Abstract

Language Model Steering

Explore similar work

Predicting Future Behaviors in Reasoning Models Enables Better Steering

Can We Break LLMs Out of Self-Loops? Fine-Grained Reasoning Control with Activation Steering

Behavior Cue Reasoning: Monitorable Reasoning Improves Efficiency and Safety through Oversight