A direction in activation space that changes safety-relevant behavior when steered is not necessarily one the model uses to produce that behavior on its own. We therefore ask whether steering acts through the computation of the unmodified model or through a different set of components, studying two traits whose directions have been extracted and validated in prior work: refusal and sycophancy in Qwen2.5-7B-Instruct. For each, we use the trait vector to split the computation into a reconstruction circuit before the vector and a transmission circuit after it. We then test whether restoring the coordinate returns behavior removed by ablation. For refusal, both circuits are compact and faithful, restoring the coordinate alone recovers almost all of the refusal signal lost to ablation, and a circuit built around the vector matches the faithfulness of a direct input-to-output circuit at roughly half the edges. For sycophancy, transmission is compact, but reconstruction is broader and only partially faithful. Circuits that transmit a steering intervention are therefore not automatically the circuits that produce the behavior, and we report the counterfactual, target, and behavioral test for every circuit accordingly.
Figures & tables
Figure 1: We divide computation around a selected trait vector. Under a specified corruption, the reconstruction circuit recovers the vector component from an earlier source. The transmission circuit carries a source-local vector intervention to model outputs. A direct input-to-output circuit provides a comparison.
Figure 2: Refusal results on 64 held-out prompts. (a) Prompt-end refusal log-odds after ablating the top-ranked 3,000-edge reconstruction subgraph and restoring the coordinate or complete layer-15 state. (b) Refusal under recurrent steering, learned-subgraph transmission or ablation, and matched random controls for the 8,000-edge transmission subgraph. (c) Static faithfulness of the two-stage construction and equal-budget direct top-ranked subgraph; the open point is the first comparably faithful source-connected direct graph. (d) Free-generation refusal for the 10,699-edge two-stage construction and equal-budget direct top-ranked subgraph; error bars show 95% paired-bootstrap intervals.
Figure 3: Prompt- and response-vector interventions both reduce sycophancy on 300 held-out questions, and their combination has the largest effect. Lower scores indicate less sycophancy.
Figure 4: All evaluated input-connected prompt- and response-coordinate reconstruction graphs, with 64 held-out questions per point. Lines connect evaluated graph sizes only as visual guides. Prompt reconstruction crosses the 85% faithfulness criterion at 16,451 edges; response reconstruction remains below it through 14,656 edges.
Appendix figures & tables1 asset
Supplementary material from the paper’s appendix.
Appendix
Phase
Refusal
Sycophancy
Purpose
Vector extraction
256
256
Estimate the contrastive trait directions
Vector validation
64
64
Select and calibrate the intervention location
Edge scoring
64
64
Compute EAP-IG edge rankings
Circuit selection
64
64
Select graph size and evaluate F/N
Behavioral evaluation
300
300
Held-out vector and generation tests
Circuit evaluation
64 of the 300
Held-out F/N, controls, and circuit behavior
Appendix
Table 1: Question-level data accounting for each trait. Circuit evaluation reuses 64 questions from the behavioral split and is not an additional split.
Reliable refusal of harmful requests is essential to the safe deployment of language models. Because excessive eagerness to please users may undermine existing refusal capabilities, reducing sycophancy offers a potential route to stronger refusal beyond the harmful scenarios covered by safety training. We investigate this possibility using compensatory feature injection (CFI), a training technique designed to limit the acquisition of a target concept by supplying its associated activation during learning. Across three Qwen3.5 base models, we use sparse autoencoders (SAEs) to identify the top-ranked sycophancy feature from paired sycophantic and independent responses, then validate its behavioral influence through inference steering. We subsequently inject the selected feature during supervised fine-tuning on sycophantic targets. Positive injection reduces learned sycophancy after removal (by 62.0% relative to ordinary fine-tuning in 35B-A3B), whereas modest negative injection increases it. Unexpectedly, these reductions in sycophancy do not consistently improve direct refusal of harmful requests, motivating a narrower evaluation of the same harmful intents under user pressure. In this setting, ordinary fine-tuning on sycophantic responses substantially weakens refusal, while selected checkpoints trained with positive injection recover part of the loss, including approximately 95% in 35B-A3B. These findings show that persistent sycophancy reduction does not guarantee stronger direct refusal, while identifying recovery under user pressure as a distinct, conditional benefit of training intervention.
Xu Wang, Difan Zou, Xuansheng Wu
The University of Hong Kong · Shenzhen Loop Area Institute · Shanghai Artificial Intelligence Laboratory
Applying steering vectors to large language models (LLMs) is an efficient and effective model alignment technique, but we lack an interpretable explanation for how it works--specifically, what internal mechanisms steering vectors affect and how this results in different model outputs. To investigate the causal mechanisms underlying the effectiveness of steering vectors, we conduct a comprehensive case study on refusal. We propose a multi-token activation patching framework and discover that different steering methodologies leverage functionally interchangeable circuits when applied at the same layer. These circuits reveal that steering vectors primarily interact with the attention mechanism through the OV circuit while largely ignoring the QK circuit. Freezing all attention scores during steering drops performance by only 8.83% across three model families. A mathematical decomposition of the steered OV circuit further reveals semantically interpretable concepts, even in cases where the steering vector itself does not. Leveraging the activation patching results, we show that steering vectors can be sparsified by up to 85-96% while retaining most performance, and that different steering methodologies agree on a subset of important dimensions.
Existing activation steering methods often assume that a high-level concept can be mediated by a single steering direction. To support this, two complementary interventions should be achieved: additive steering should induce the target behavior, while directional ablation should suppress it. Yet behaviors may occupy richer activation geometries beyond a single direction, and semantically similar behaviors may be represented by distinct directions. In this work, we study how many directions can reliably control two different types of refusal behaviors: refusal triggered by the safety alignment and refusal in general contexts. Given the limited expressive capability of a single steering vector, we study the general setting of steering subspaces and introduce the notion of steering dimensionality as the minimum subspace dimensionality required to reliably control a behavior. We characterize sufficient steering subspaces that cover the full extent of the target behavior through both the (monotonic) improvement before the sufficient dimensionality, and the saturation beyond it. Empirically, we find that refusal triggered by safety alignment is 1-dim steerable, while multiple distinct steering directions can achieve comparable control. In contrast, refusal in general contexts exhibits substantially richer activation geometry where even 5-dim steering subspaces fail to reliably capture its full steerable variation. Our results reveal that the activation geometry underlying refusal is highly context-dependent and can be substantially more complex than a single linear steering direction.
Han Wang, Erik Miehling, Dennis Wei +2
University of Illinois Urbana-Champaign · IBM Research