Less Sycophancy, Stronger Refusal? Lessons for AI Safety from Mechanistic Interpretability

Xu Wang; Difan Zou; Xuansheng Wu

cs.CLSep 28, 2026

Less Sycophancy, Stronger Refusal? Lessons for AI Safety from Mechanistic Interpretability

Authors: Xu Wang, Difan Zou, Xuansheng Wu

Organizations: The University of Hong Kong · Shenzhen Loop Area Institute · Shanghai Artificial Intelligence Laboratory

Abstract

Reliable refusal of harmful requests is essential to the safe deployment of language models. Because excessive eagerness to please users may undermine existing refusal capabilities, reducing sycophancy offers a potential route to stronger refusal beyond the harmful scenarios covered by safety training. We investigate this possibility using compensatory feature injection (CFI), a training technique designed to limit the acquisition of a target concept by supplying its associated activation during learning. Across three Qwen3.5 base models, we use sparse autoencoders (SAEs) to identify the top-ranked sycophancy feature from paired sycophantic and independent responses, then validate its behavioral influence through inference steering. We subsequently inject the selected feature during supervised fine-tuning on sycophantic targets. Positive injection reduces learned sycophancy after removal (by 62.0% relative to ordinary fine-tuning in 35B-A3B), whereas modest negative injection increases it. Unexpectedly, these reductions in sycophancy do not consistently improve direct refusal of harmful requests, motivating a narrower evaluation of the same harmful intents under user pressure. In this setting, ordinary fine-tuning on sycophantic responses substantially weakens refusal, while selected checkpoints trained with positive injection recover part of the loss, including approximately 95% in 35B-A3B. These findings show that persistent sycophancy reduction does not guarantee stronger direct refusal, while identifying recovery under user pressure as a distinct, conditional benefit of training intervention.

Figures & tables

Figure 1: From sycophancy control to two refusal tests. Paired responses identify a feature, inference steering tests its influence, and training injection changes behavior after removal. Direct requests test refusal independently of the trait score; added user pressure motivates the follow-up. Overall, our designs is SAE-guided sycophancy control across harmful requests.

Figure 2: Feature association and influence on generation. The heatmap shows how often the selected feature enters the largest positive paired contrasts within each domain. Solid curves show intended trait-label switches under inference steering; dashed curves show repetitive or unclear responses on the same fixed subsets. Raw decoder scales differ across models (Appendix C ). Behavioral influence is asymmetric and must be assessed alongside output quality.

Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

Model	SAE layer	Feature	Learning rate	Batch	Epochs
2B	15	28,758	$2\times 10^{-6}$	8	2
9B	19	61,718	$5\times 10^{-7}$	8	1
35B-A3B	27	2,362	$8.1\times 10^{-6}$	64	1

Appendix

Table 1: Main training settings. Layer indices are zero-based, batch sizes are global, and the selected SAE feature is fixed throughout training and subsequent evaluation.

Model	Injection	$\beta$	Epochs	Syco / instruction	Scope
2B	Positive	$+10$	2	1000 / 1000	All
	Negative	$-0.5$	1	1000 / 1000	All
9B	Positive	$+80$	1	1000 / 1000	All
	Negative	$-3$	1	1000 / 1000	All
35B-A3B	Positive	$+30$	1	1000 / 1000	All
	Negative	$-1$	1	1000 / 1000	All

Appendix

Table 2: Training settings for the pressure-refusal checkpoints. Example counts refer to sycophantic targets and instruction data; injection scope identifies the rows receiving the fixed activation.

Model	Tested $\|\alpha\|$	$n_{\rm enh}$	$n_{\rm sup}$
2B	$\{0.1,0.2,0.5,0.7,1,1.5,2,2.5,3,4\}$	146	190
9B	$\{0.2,0.5,1,2,3,5,7,10,15,20\}$	137	209
35B-A3B	$\{0.2,0.5,0.7,1,1.2,1.5,1.7,2,2.5,3\}$	138	208

Appendix

Table 3: Inference strengths and fixed evaluation subsets. Enhancement uses positive coefficients and suppression uses negative coefficients with the listed magnitudes; subset sizes remain fixed.

Model	Target $C_{j^{\star}}$ [95% CI]	Random feature: $C_{j}$
2B	$\mathbf{36.8}$ [33.1, 40.4]	f615: 1.2	f6868: 0.5	f888: $-5.8$
9B	$\mathbf{6.5}$ [4.3, 8.7]	f52: 0.8	f527: $-1.1$	f1288: 0.2
35B-A3B	$\mathbf{41.0}$ [37.5, 44.6]	f99: $-6.0$	f666: 15.0	f888: $-14.5$

Appendix

Table 4: Local sign separation for target and random directions. Values are absolute rate differences in percent, averaged over

|\beta|\in\{1,2,5\}

; brackets give the target’s 95% interval.

Label	Assignment rule
$\mathrm{AHC}$	Usable harmful assistance with explicit recognition of harm.
$\mathrm{UHC}$	Usable harmful assistance without explicit recognition of harm.
$\mathrm{R}$	Harmful assistance withheld through refusal or effective safe redirection.
REPEAT	No harmful assistance or effective refusal, with degenerate repetition.
UNCERTAIN	No preceding category can be assigned from the available answer.

Appendix

Table 5: Response labels for harmful requests. Harmful assistance takes precedence over warnings, refusal phrases, and repetition; each answer receives exactly one final outcome label.

Model	Violations / eligible	$\|\mathcal{I}\|$	$D$ ( $\%$ )	95% interval
2B	26 / 224	119	$-12.6$	$[-19.3,-5.9]$
9B	11 / 227	221	$-12.2$	$[-16.7,-7.7]$
35B-A3B	5 / 254	235	$-23.0$	$[-28.5,-17.4]$

Appendix

Table 6: Paired pressure comparisons against ordinary SFT. Violations use each positive checkpoint’s directly refused set;

D

and its interval use the common eligible set of the paired comparison.

Model	Checkpoint	Direct refusal	Pressure refusal
2B	Base	124 (41.9%)	193 (65.2%)
	Ordinary SFT	124 (41.9%)	170 (57.4%)
	Positive	224 (75.7%)	222 (75.0%)
9B	Base	258 (87.2%)	269 (90.9%)
	Ordinary SFT	232 (78.4%)	194 (65.5%)
	Positive	227 (76.7%)	237 (80.1%)

Appendix

Table 7: Counts underlying the pressure-recovery comparisons. Each entry is a refusal count out of 296 harmful intents, with its corresponding rate in parentheses; positive settings follow Table 2 .

Explore similar work

Agentic Scaffolding Amplifies Sycophantic Behavior in Large Language Models
Jul 6, 2026Thantham JitthamSycophancy Interaction Data
Cornell University
Not Your Typical Sycophant: The Elusive Nature of Sycophancy in Large Language Models
Jan 21, 2026Shahar Ben-Natan, Oren TsurSycophancy Large Language Model Bias
Computer and Information Science Ben Gurion University
Beyond Shallow Alignment: How Post-Training Methods Determine Refusal Circuits And Steering Robustness
Sep 3, 2026Hoang Cuong Nguyen, Mark Dras, Usman NaseemRefusals Large Language Model Alignment
Macquarie University, Sydney, Australia

Jul 6, 2026, cs.CLJ/K move · Enter open · S save

Agentic Scaffolding Amplifies Sycophantic Behavior in Large Language Models

Thantham Jittham

Cornell University

Sycophancy Interaction Data Intermediate Checkpoints

Open paper AskPDF arXiv

Sycophancy in large language models, the tendency to prioritize user agreement over truthful responses, has been documented extensively but studied primarily in single-turn settings. This paper investigates a critical question: does subjecting LLMs to greater interaction scaffolding make sycophancy better or worse? Across 4,800 veracity judgments (200 statements

\times

6 models

\times

4 conditions), we find that the interaction scaffolding characteristic of agentic systems (feedback loops, reconsideration checkpoints, and iterative refinement) systematically amplifies sycophantic behavior. Multi-turn interaction, user pressure, and iterative self-refinement each provide additional opportunities for models to drift toward agreement, and this drift coincides with a mean accuracy drop of

-6.3

percentage points, establishing the capitulation as harmful rather than corrective. More capable models show larger amplification effects, a troubling inversion of expectations. We introduce the concept of agentic sycophancy amplification (ASA) and two novel metrics: capitulation rate and sycophantic capitulation rate. Our results indicate that as AI systems acquire greater autonomy, sycophancy becomes compounding rather than merely persistent. Systems designed with human oversight loops may inadvertently create the conditions for this drift.

Abstract

Explore similar work

Agentic Scaffolding Amplifies Sycophantic Behavior in Large Language Models

Not Your Typical Sycophant: The Elusive Nature of Sycophancy in Large Language Models

Beyond Shallow Alignment: How Post-Training Methods Determine Refusal Circuits And Steering Robustness