Less Sycophancy, Stronger Refusal? Lessons for AI Safety from Mechanistic Interpretability

Xu Wang; Difan Zou; Xuansheng Wu

cs.CLSep 28, 2026

Less Sycophancy, Stronger Refusal? Lessons for AI Safety from Mechanistic Interpretability

Authors: Xu Wang, Difan Zou, Xuansheng Wu

Organizations: The University of Hong Kong · Shenzhen Loop Area Institute · Shanghai Artificial Intelligence Laboratory

Abstract

Reliable refusal of harmful requests is essential to the safe deployment of language models. Because excessive eagerness to please users may undermine existing refusal capabilities, reducing sycophancy offers a potential route to stronger refusal beyond the harmful scenarios covered by safety training. We investigate this possibility using compensatory feature injection (CFI), a training technique designed to limit the acquisition of a target concept by supplying its associated activation during learning. Across three Qwen3.5 base models, we use sparse autoencoders (SAEs) to identify the top-ranked sycophancy feature from paired sycophantic and independent responses, then validate its behavioral influence through inference steering. We subsequently inject the selected feature during supervised fine-tuning on sycophantic targets. Positive injection reduces learned sycophancy after removal (by 62.0% relative to ordinary fine-tuning in 35B-A3B), whereas modest negative injection increases it. Unexpectedly, these reductions in sycophancy do not consistently improve direct refusal of harmful requests, motivating a narrower evaluation of the same harmful intents under user pressure. In this setting, ordinary fine-tuning on sycophantic responses substantially weakens refusal, while selected checkpoints trained with positive injection recover part of the loss, including approximately 95% in 35B-A3B. These findings show that persistent sycophancy reduction does not guarantee stronger direct refusal, while identifying recovery under user pressure as a distinct, conditional benefit of training intervention.

Figures & tables

Figure 1: From sycophancy control to two refusal tests. Paired responses identify a feature, inference steering tests its influence, and training injection changes behavior after removal. Direct requests test refusal independently of the trait score; added user pressure motivates the follow-up. Overall, our designs is SAE-guided sycophancy control across harmful requests.

Figure 2: Feature association and influence on generation. The heatmap shows how often the selected feature enters the largest positive paired contrasts within each domain. Solid curves show intended trait-label switches under inference steering; dashed curves show repetitive or unclear responses on the same fixed subsets. Raw decoder scales differ across models (Appendix C ). Behavioral influence is asymmetric and must be assessed alongside output quality.

Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

Model	SAE layer	Feature	Learning rate	Batch	Epochs
2B	15	28,758	$2\times 10^{-6}$	8	2
9B	19	61,718	$5\times 10^{-7}$	8	1
35B-A3B	27	2,362	$8.1\times 10^{-6}$	64	1

Appendix

Table 1: Main training settings. Layer indices are zero-based, batch sizes are global, and the selected SAE feature is fixed throughout training and subsequent evaluation.

Model	Injection	$\beta$	Epochs	Syco / instruction	Scope
2B	Positive	$+10$	2	1000 / 1000	All
	Negative	$-0.5$	1	1000 / 1000	All
9B	Positive	$+80$	1	1000 / 1000	All
	Negative	$-3$	1	1000 / 1000	All
35B-A3B	Positive	$+30$	1	1000 / 1000	All
	Negative	$-1$	1	1000 / 1000	All

Appendix

Table 2: Training settings for the pressure-refusal checkpoints. Example counts refer to sycophantic targets and instruction data; injection scope identifies the rows receiving the fixed activation.

Model	Tested $\|\alpha\|$	$n_{\rm enh}$	$n_{\rm sup}$
2B	$\{0.1,0.2,0.5,0.7,1,1.5,2,2.5,3,4\}$	146	190
9B	$\{0.2,0.5,1,2,3,5,7,10,15,20\}$	137	209
35B-A3B	$\{0.2,0.5,0.7,1,1.2,1.5,1.7,2,2.5,3\}$	138	208

Appendix

Table 3: Inference strengths and fixed evaluation subsets. Enhancement uses positive coefficients and suppression uses negative coefficients with the listed magnitudes; subset sizes remain fixed.

Model	Target $C_{j^{\star}}$ [95% CI]	Random feature: $C_{j}$
2B	$\mathbf{36.8}$ [33.1, 40.4]	f615: 1.2	f6868: 0.5	f888: $-5.8$
9B	$\mathbf{6.5}$ [4.3, 8.7]	f52: 0.8	f527: $-1.1$	f1288: 0.2
35B-A3B	$\mathbf{41.0}$ [37.5, 44.6]	f99: $-6.0$	f666: 15.0	f888: $-14.5$

Appendix

Table 4: Local sign separation for target and random directions. Values are absolute rate differences in percent, averaged over

|\beta|\in\{1,2,5\}

; brackets give the target’s 95% interval.

Label	Assignment rule
$\mathrm{AHC}$	Usable harmful assistance with explicit recognition of harm.
$\mathrm{UHC}$	Usable harmful assistance without explicit recognition of harm.
$\mathrm{R}$	Harmful assistance withheld through refusal or effective safe redirection.
REPEAT	No harmful assistance or effective refusal, with degenerate repetition.
UNCERTAIN	No preceding category can be assigned from the available answer.

Appendix

Table 5: Response labels for harmful requests. Harmful assistance takes precedence over warnings, refusal phrases, and repetition; each answer receives exactly one final outcome label.

Model	Violations / eligible	$\|\mathcal{I}\|$	$D$ ( $\%$ )	95% interval
2B	26 / 224	119	$-12.6$	$[-19.3,-5.9]$
9B	11 / 227	221	$-12.2$	$[-16.7,-7.7]$
35B-A3B	5 / 254	235	$-23.0$	$[-28.5,-17.4]$

Appendix

Table 6: Paired pressure comparisons against ordinary SFT. Violations use each positive checkpoint’s directly refused set;

D

and its interval use the common eligible set of the paired comparison.

Model	Checkpoint	Direct refusal	Pressure refusal
2B	Base	124 (41.9%)	193 (65.2%)
	Ordinary SFT	124 (41.9%)	170 (57.4%)
	Positive	224 (75.7%)	222 (75.0%)
9B	Base	258 (87.2%)	269 (90.9%)
	Ordinary SFT	232 (78.4%)	194 (65.5%)
	Positive	227 (76.7%)	237 (80.1%)

Appendix

Table 7: Counts underlying the pressure-recovery comparisons. Each entry is a refusal count out of 296 harmful intents, with its corresponding rate in parentheses; positive settings follow Table 2 .

Explore similar work

Jul 6, 2026cs.CL

Agentic Scaffolding Amplifies Sycophantic Behavior in Large Language Models

Sycophancy in large language models, the tendency to prioritize user agreement over truthful responses, has been documented extensively but studied primarily in single-turn settings. This paper investigates a critical question: does subjecting LLMs to greater interaction scaffolding make sycophancy better or worse? Across 4,800 veracity judgments (200 statements

\times

6 models

\times

4 conditions), we find that the interaction scaffolding characteristic of agentic systems (feedback loops, reconsideration checkpoints, and iterative refinement) systematically amplifies sycophantic behavior. Multi-turn interaction, user pressure, and iterative self-refinement each provide additional opportunities for models to drift toward agreement, and this drift coincides with a mean accuracy drop of

-6.3

percentage points, establishing the capitulation as harmful rather than corrective. More capable models show larger amplification effects, a troubling inversion of expectations. We introduce the concept of agentic sycophancy amplification (ASA) and two novel metrics: capitulation rate and sycophantic capitulation rate. Our results indicate that as AI systems acquire greater autonomy, sycophancy becomes compounding rather than merely persistent. Systems designed with human oversight loops may inadvertently create the conditions for this drift.

Thantham Jittham

Cornell University

Sycophancy Interaction Data Intermediate Checkpoints

Ask PDF

Jan 21, 2026cs.AI

Not Your Typical Sycophant: The Elusive Nature of Sycophancy in Large Language Models

We propose a novel perspective for probing LLM sycophancy in a direct and neutral way, mitigating various forms of uncontrolled bias, noise, or manipulative language, deliberately injected to prompts in prior works. A key novelty of our approach is the use of an LLM-as-a-judge in a zero-sum betting game. Within this framework, sycophancy serves one individual (the user) while explicitly incurring cost on another. Comparing 11 leading models we find that while most models exhibit significant sycophantic tendencies in the common setting, in which sycophancy is self-serving to the user and incurs no cost on others, seven of the models exhibit ``moral remorse'', five of which significantly over-compensate for their sycophancy in case it explicitly harms a third party. We refer to this phenomenon as `anti-sycophancy' bias and discuss possible causes for this shift.

Shahar Ben-Natan, Oren Tsur

Computer and Information Science Ben Gurion University

Sycophancy Large Language Model Bias Biases Novel View Synthesis

Ask PDF

Sep 3, 2026cs.CL

Beyond Shallow Alignment: How Post-Training Methods Determine Refusal Circuits And Steering Robustness

How do the methods used to train language models to refuse harmful requests shape how that refusal actually works inside the model? We compare three post-training methods - supervised fine-tuning, reasoning-augmented fine-tuning (training on reasoning chains that justify a safety decision), and preference optimization (ORPO) - across three architecturally distinct models (Llama-3.1-8B, Gemma-2-9B, Qwen3-8B). We find that training method, not just data, reshapes how refusal is computed internally: reasoning-augmented training consistently produces a distinct kind of refusal computation, visible across all three models, while architecture independently shapes internal structure and how reliably refusal can be steered. Most importantly, no method we study achieves all three properties we would want from safe alignment at once: refusal that isn't concentrated in a few fragile components, safety gains that don't cost general capability, and safety behavior correctable through small, targeted edits. We caution against treating current post-training methods as a solved, reliable defense, especially for security-critical use. Code and models are available in https://github.com/hoangcuongnguyen2001/Beyond-Shallow-Alignment.

Hoang Cuong Nguyen, Mark Dras, Usman Naseem

Macquarie University, Sydney, Australia

Refusals Large Language Model Alignment Post-Training Steering

Ask PDF