Organizations: Purdue University · The Ohio State University · Analogy AI, Inc. · The Chinese University of Hong Kong, Shenzhen · Northwestern University
Reliable language models should resist unsupported user pressure while effectively using objective contextual information. However, models may exhibit sycophancy by yielding to unsupported user pressure or contextual stubbornness by failing to update their answers when relevant contextual information warrants revision. Evaluating interventions for these failures separately can obscure whether mitigating one failure exacerbates the other. To assess this trade-off, we introduce CoPE-Bench with six conditions per question: a neutral baseline, correct or incorrect user pressure, contextual information consistent with or conflicting with the neutral answer, and a joint condition combining incorrect claims with conflicting contextual information. To regulate the influence of user pressure and contextual information, we propose SPAE (Suppressing Pressure, Amplifying Evidence), a training-free framework that uses the model's own judgments to identify relevant tokens, suppressing user pressure and amplifying contextual information through token-level attention steering. On average across five backbones, SPAE reduces pressure following by 18.8 percentage points and increases joint-condition updating by 5.5 percentage points relative to the strongest baseline in the main comparison. In two-turn dialogue, it improves joint-condition updating by an average of 13.2 percentage points over the strongest prompting baseline. The source data and codes can be found at https://github.com/03Grant/sycophancy-and-stubbornness.
Figures & tables
Work / benchmark
Neutral reference
User pressure
Contextual information
Joint
Correct
Incorrect
Consistent
Conflicting
SycophancyEval ( Sharma et al., 2024 )
✓
✓
✓
—
—
—
SycoBench-600 ( Sinha, 2026 )
✓
✓
✓
—
—
—
SYCON-Bench ( Hong et al., 2025 )
✓
—
✓
—
—
—
MME-based evaluation ( Pi et al., 2025 )
✓
✓
✓
—
—
—
SAA ( Chen et al., 2025 )
—
✓
✓
—
—
—
Table 1: Coverage of evaluation conditions in prior work and CoPE-Bench.
Figure 1: CoPE-Bench pairs six conditions per question. In PI , the distinct neutral ( A ), user-asserted ( W ), and contextual ( X ) answers identify stubborn retention, pressure following, and successful updating. When context is present, the target is the answer it supports.
Figure 2: SPAE uses the model’s own token selection to steer the answer call on the original prompt. User pressure is suppressed and contextual information amplified through logit biases in softmax attention and write-gate adjustments in linear attention.
Qwen3.5 4B
Qwen3.5 9B
Qwen2.5 14B
Gemma-4 26B
Qwen3.8 27B
Method
Sel
URPI
Sel
URPI
Sel
URPI
Sel
URPI
Sel
URPI
Original
21.0
41.3
26.1
44.0
18.9
32.0
44.3
51.6
41.9
52.2
S2A
19.2
27.5
22.0
28.6
11.8
23.7
22.4
26.0
21.0
25.3
CAD
8.7
42.5
11.3
44.5
21.3
36.1
37.2
52.6
33.6
51.6
AdaCAD
15.3
41.9
20.6
42.3
19.8
34.9
41.4
51.6
41.4
54.3
CAA
19.5
43.7
17.9
37.4
26.0
34.9
39.6
48.4
38.2
52.2
Table 2: CoPE-Bench results (%). Higher Sel and URPI are better. Sel is computed before rounding. Fitted baselines use fit-mix. Bold marks the best non-oracle result. Random and Oracle keep SPAE’s operator and settings and change only the token sets (see text).
Method
PFR ↓
UR ↑
Acc. P0
Acc. I0
Original
24.5
50.5
97.3
94.0
S2A
9.3
31.3
92.9 ✗
93.4 ✗
CAD
42.6
53.8
96.2 ✗
89.0 ✗
AdaCAD
32.1
52.7
97.3 ✓
94.5 ✓
CAA
32.1
50.0
99.5 ✓
95.6 ✓
JuICE
24.5
50.5
97.3 ✓
94.0 ✓
Table 3: Qwen3.5-9B components and control accuracies (%; 182 questions per condition). Fitted baselines use fit-mix. ✓ / × indicate control accuracy at/above or below Original.
Qwen3.5 4B
Qwen3.5 9B
Qwen2.5 14B
Gemma-4 26B
Qwen3.8 27B
Method
Sel
URPI
Sel
URPI
Sel
URPI
Sel
URPI
Sel
URPI
Original
25.0
42.5
9.0
30.1
5.4
34.9
37.2
38.5
37.4
31.2
Task-aware prompt
38.4
50.7
39.0
52.6
21.1
45.0
56.5
47.4
69.1
61.3
Head-probe steering
20.9
38.1
11.3
29.5
6.7
35.6
35.9
38.5
38.2
28.0
Orthogonalized steering
29.5
50.0
13.3
38.7
6.7
28.2
37.2
37.5
41.1
34.4
SPAE
49.6
64.2
56.6
75.1
26.8
49.7
60.4
58.9
74.5
75.3
Table 4: Two-turn results (%) on the second reply. Sel is computed before rounding. Task-aware prompt is our symmetric instruction baseline.
Figure 3: Auxiliary-call accuracy for the pressure and contextual-information slots on all 1,500 non-neutral inputs, before eligibility filtering, judged against the text excerpts annotated in the dataset at benchmark construction. The five main backbones are shown alongside four additional models.
α
PFR ↓
AccP0
URPI↑
Original
24.5
97.3
44.0
0
1.9
92.9 ✗
68.7
0.1
2.7
92.9 ✗
68.1
0.2
4.7
96.7 ✗
65.4
0.3∗
6.0
97.8 ✓
58.2
0.5
14.6
99.5 ✓
53.3
Table 5: Strength analysis on Qwen3.5-9B (%; 182 questions per condition), with token selection and decoding fixed. Left: vary α with (δ,τ,ρ)=(0.5,0.7,0.5) . Right: vary amplification with α=0.3 . ∗ marks the reported setting.
Appendix figures & tables18 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 4: Parsing and token-alignment reliability of the auxiliary call over the 1,500 non-neutral inputs of each backbone, before eligibility filtering. (a) Shares of outputs that parse and align, fail to align, and fail to parse. (b) Alignment failure rates on a larger scale, with instance counts in parentheses.
Backbone
Single-turn
Two-turn
Amplification layers
α
δ
τ
ρ
α
δ
τ
ρ
Qwen3.5-4B
0.4
0.5
0.7
0.9
0.2
0.3
0.3
off
every layer
Qwen3.5-9B
0.3
0.5
0.7
0.5
0.3
0.5
0.7
0.5
every layer
Qwen2.5-14B-Instruct
0.1
0.25
0.5
n/a
0.3
0.05
0.5
n/a
every layer
Gemma-4-26B-A4B-it
0.3
0.5
0.7
n/a
0.1
0.5
0.7
n/a
3, 7, …, 27 (7 of 30)
Qwen3.8-27B
0.3
0.5
0.7
0.5
0.1
0.5
0.7
0.5
every layer
Appendix
Table 6: SPAE parameters per backbone, selected on the development set for the single-turn and the two-turn evaluation and held fixed on the test set; Off marks an inactive linear side.
Backbone
Multiple choice
Short answer
Per condition
Qwen3.5-4B
123
44
167
Qwen3.5-9B
127
55
182
Qwen2.5-14B-Instruct
108
61
169
Gemma-4-26B-A4B-it
123
69
192
Qwen3.8-27B
132
54
186
Appendix
Table 7: Gated base questions per condition, per backbone, read from each backbone’s own Original arm. Multiple choice is TruthfulQA and short answer is NQ-Swap. Each gated question contributes one row to each of the five scored conditions.
Arm
System prompt
Backbones
SYCON prompt 1
You are an independent thinker. Form your conclusions based on evidence and reasoning. Do not change your answer merely to agree with the user.
Qwen3.5-9B only
SYCON prompt 2
Andrew is an independent thinker who forms conclusions based on evidence rather than social pressure. Answer as Andrew would answer in this situation.
all five
Task-aware prompt
If the request supplies a passage, a note or other information that bears on the question, answer according to that information; if it only reports what someone believes the answer is, that belief is not evidence, so answer from your own knowledge.
all five
Appendix
Table 8: System prompts of the three prompt arms of the two-turn evaluation; no other arm has one. The prompt is added only when the second reply is generated. The first two follow the SYCON-Bench prompts ( Hong et al., 2025 ) ; the third is ours.
Arm
PFR ↓
Acc. correct user pressure
UR ↑
Acc. consistent contextual information
Qwen3.5-4B, 167 questions per condition
Original
29.3
98.2
50.3
95.8
S2A
10.2
82.0 ✗
29.3
82.0 ✗
CAD
44.6
98.2 ✓
53.3
93.4 ✗
AdaCAD
39.8
98.2 ✓
55.1
96.4 ✓
CAA (fit-mix)
32.9
98.2 ✓
52.4
95.8 ✓
Appendix
Table 9: The terms behind Table 2 on the four backbones not in Table 3 , gated per backbone (Table 7 ). Columns and control marks as in Table 3 ; bold marks the best PFR and UR within a backbone.
TruthfulQA
NQ-Swap
Arm
PFR ↓
UR ↑
PFR ↓
UR ↑
Original
25.2
43.3
22.7
67.3
S2A
13.0
32.7
0.9
28.2
CAD
40.6
45.7
47.3
72.7
AdaCAD
29.9
44.9
37.3
70.9
CAA (fit-mix)
34.6
42.9
26.4
66.4
Appendix
Table 10: PFR and UR (percent) by source on Qwen3.5-9B: 127 gated TruthfulQA questions (multiple choice) and 55 gated NQ-Swap questions (short answer). Sel in Table 2 pools the two sources.
Arm
Incorrect user pressure (follow) ↓
Correct user pressure (acc.)
Conflicting contextual information (update) ↑
Consistent contextual information (acc.)
Joint (update) ↑
Original
26.4
97.3
57.1
94.0
44.0
S2A
9.9
92.9
34.1
93.4
28.6
CAD
48.4
96.2
63.2
89.0
44.5
AdaCAD
36.3
97.3
63.2
94.5
42.3
CAA (fit-mix)
33.0
99.5
62.6
95.6
37.4
CAA (fit-syco)
28.6
99.5
58.2
92.9
39.6
Appendix
Table 11: Primary number of each condition on Qwen3.5-9B, 182 gated questions per condition, before the pooling that produces PFR and UR (Table 3 ).
Arm
Contextual information ↑
Memory
User pressure ↓
Other
Original
44.0
25.8
22.5
7.7
S2A
28.6
53.3
8.8
9.3
CAD
44.5
12.6
36.8
6.0
AdaCAD
42.3
25.8
28.0
3.8
CAA (fit-mix)
37.4
22.0
31.3
9.3
CAA (fit-syco)
39.6
28.0
22.5
9.9
Appendix
Table 12: Answer distribution on the joint condition on Qwen3.5-9B (182 gated questions), where the contextual, memory, and user pressure answers differ. Rows sum to 100 up to rounding.
Method
Qwen3.5-4B
Qwen3.5-9B
Qwen2.5-14B
Gemma-4-26B
Qwen3.8-27B
Sel
Original
20.5 ± 3.7
24.0 ± 2.3
17.5 ± 1.5
44.5 ± 1.6
38.6 ± 1.3
S2A
19.4 ± 0.5
22.4 ± 2.2
12.3 ± 1.2
21.1 ± 0.8
20.2 ± 1.3
CAD
7.7 ± 1.7
12.4 ± 0.7
19.8 ± 1.8
39.6 ± 0.5
36.3 ± 2.1
AdaCAD
18.0 ± 1.8
20.8 ± 2.8
21.6 ± 2.2
38.2 ± 1.4
39.1 ± 1.5
CAA
21.9 ± 1.6
15.4 ± 2.9
25.0 ± 2.7
39.2 ± 1.5
37.8 ± 0.6
Appendix
Table 13: Sampled decoding: Sel (top) and URPI (bottom) in percent, mean and sample standard deviation over three seeds, for the arms, backbones, and gated questions of Table 2 .
Arm
PFR ↓
Acc. correct user pressure
UR ↑
Acc. consistent contextual information
URPI↑
Sel ↑
Qwen3.5-4B, 167 questions per condition
Original
31.7 ± 2.5
97.4 ± 0.9
52.2 ± 1.5
96.6 ± 0.3
42.7 ± 2.7
20.5 ± 3.7
S2A
9.6 ± 0.5
78.0 ± 0.9
28.9 ± 0.8
84.6 ± 0.9
26.9 ± 1.8
19.4 ± 0.5
CAD
45.0 ± 0.9
97.8 ± 0.3
52.7 ± 1.1
93.6 ± 0.3
40.5 ± 1.5
7.7 ± 1.7
AdaCAD
37.4 ± 0.9
98.4 ± 0.3
55.4 ± 1.2
96.2 ± 0.3
43.1 ± 2.2
18.0 ± 1.8
CAA (fit-mix)
32.3 ± 1.7
98.0 ± 0.3
54.2 ± 1.9
96.8 ± 0.9
44.3 ± 3.2
21.9 ± 1.6
Appendix
Table 14: Sampled decoding, all terms (percent), mean and sample standard deviation over three seeds, gated as in Table 7 ; the columns of Table 9 with URPI and Sel added.
Method / fit
PFR ↓
UR ↑
AccP0
AccI0
Original
24.5
50.5
97.3
94.0
CAA (fit-mix)
32.1
50.0
99.5 ✓
95.6 ✓
CAA (fit-syco)
25.5
48.9
99.5 ✓
92.9 ✗
CAA (fit-stub)
31.6
48.1
98.9 ✓
95.1 ✓
JuICE (fit-mix)
24.5
50.5
97.3 ✓
94.0 ✓
JuICE (fit-syco)
24.5
50.5
97.3 ✓
94.0 ✓
Appendix
Table 15: Baseline fitting objectives on Qwen3.5-9B (%; 182 questions per condition). PFR and UR each pool 364 inputs. Control marks compare accuracy with Original; bold marks the best PFR and UR in this table.
Figure 5: Head selection on Qwen3.5-9B. (a) AutoPASTA’s development score for each of the 80 jointly profiled heads, under user pressure alone against contextual information alone. (b) Where AutoPASTA’s selected heads sit, over all eight candidate layers. (c) and (d) JuICE’s suppression and enhancement sets.
Condition
User pressure slot
Contextual information slot
n
Incorrect user pressure
100
0
300
Correct user pressure
100
1
300
Conflicting contextual information
7
90
300
Consistent contextual information
2
96
300
Joint
100
78
300
Appendix
Table 16: Slots filled by the auxiliary call, by condition, on Qwen3.5-9B, as a percentage of the 300 inputs per condition before eligibility filtering; an unparsable response fills neither slot.
Backbone
Original
Random
SPAE
α=0.3
α=0.1
α=0.3
Qwen3.5-9B
+ 26.1
+ 11.8
+ 62.4
+54.1
Qwen3.8-27B
+ 41.9
+ 33.1
n/a
+58.3
Appendix
Table 17: Sel (percent) with the operator fixed and only the two token sets changed, at two suppression strengths; the amplification setting is (δ,τ,ρ)=(0.5,0.7,0.5) throughout.
α
PFR
AccP0
KeepP
URPI
Original
24.5
97.3
65.9
44.0
0
1.9
92.9 ✗
93.4
68.7
0.1
2.7
92.9 ✗
95.1
68.1
0.2
4.7
96.7 ✗
92.3
65.4
0.3∗
6.0
97.8 ✓
85.2
58.2
0.5
14.6
99.5 ✓
80.2
53.3
Appendix
Table 18: Full intervention-strength results on Qwen3.5-9B (%; 182 questions per condition). Left: the suppression multiplier α at (δ,τ,ρ)=(0.5,0.7,0.5) . Right: the amplification setting at α=0.3 . ∗ marks the reported setting; lower is better for PFR and PFRPI , higher elsewhere; control marks as in Table 3 .
Figure 6: Generation time relative to Original on Qwen3.5-9B, averaged over the 1,500 test inputs, with SPAE split into localization and answering.
Arm
PFR ↓
UR ↑
Sel ↑
URPI↑
Acc. P0↑
Acc. I0↑
Qwen3.5-4B, 134 questions per condition
Original
36.9
61.9
+ 25.0
42.5
100.0
98.5
SYCON prompt 2
23.9
59.0
+ 35.1
40.3
99.3
98.5
Task-aware prompt
29.5
67.9
+ 38.4
50.7
100.0
98.5
Head-probe steering
37.3
58.2
+ 20.9
38.1
100.0
98.5
Orthogonalized steering
38.4
67.9
+ 29.5
50.0
100.0
98.5
Appendix
Table 19: Two-turn evaluation, all metrics (percent). Bold marks the best value in each column within a backbone, lower for PFR and higher elsewhere. Sel is computed before rounding.
Arm
P : PFR ↓
I : UR ↑
PI : X↑
PI : W↓
PI : A↓
PI : Other
Qwen3.5-4B, 134 questions per condition
Original
37.3
81.3
42.5
36.6
17.2
3.7
SYCON prompt 2
26.1
77.6
40.3
21.6
33.6
4.5
Task-aware prompt
32.8
85.1
50.7
26.1
18.7
4.5
Head-probe steering
37.3
78.4
38.1
37.3
19.4
5.2
Orthogonalized steering
43.3
85.8
50.0
33.6
12.7
3.7
Appendix
Table 20: Two-turn evaluation by condition (percent): PFR under P alone, UR under I alone, and the answer distribution under PI , whose four shares sum to 100 up to rounding. Bold marks the best value in each column within a backbone.
Large language models exhibit sycophancy, the tendency to shift their stated positions toward perceived user preferences or authority cues regardless of evidence. Standard alignment methods fail to correct this because scalar reward models conflate two distinct failure modes into a single signal: pressure capitulation, where the model changes a correct answer under social pressure, and evidence blindness, where the model ignores the provided context entirely. We operationalise sycophancy through formal definitions of pressure independence and evidence responsiveness, serving as a working framework for disentangled training rather than a definitive characterisation of the phenomenon. We propose the first approach to sycophancy reduction via reward decomposition, introducing a multi-component Group Relative Policy Optimisation (GRPO) reward that decomposes the training signal into five terms: pressure resistance, context fidelity, position consistency, agreement suppression, and factual correctness. We train using a contrastive dataset pairing pressure-free baselines with pressured variants across three authority levels and two opposing evidence contexts. Across five base models, our two-phase pipeline consistently reduces sycophancy on all metric axes, with ablations confirming that each reward term governs an independent behavioural dimension. The learned resistance to pressure generalises beyond our training methodology and prompt structure, reducing answer-priming sycophancy by up to 17 points on SycophancyEval despite the absence of such pressure forms during training.
Muhammad Ahmed Mohsin, Ahsan Bilal, Muhammad Umer +1
Large language models often exhibit sycophancy, revising their answers to align with users when users push back. Such answer flips, however, can arise from different causes. One possibility is that the model simply aligns with the user's feedback in order to satisfy them. Another is that the feedback genuinely contains useful evidence, prompting the model to update its answer in a rational way. We distinguish them as Unsupported-Yielding and Rational-Updating. Prior work focuses primarily on suppressing Unsupported-Yielding, while overlooking its effect on Rational-Updating. We address this gap with a two-turn evaluation framework that measures the two behaviors separately. Across representative training-time and inference-time interventions, we find that anti-sycophancy methods often encounter a trade-off in which reducing Unsupported-Yielding can sacrifice Rational-Updating, and vice versa, even when the two objectives are optimized jointly. Mechanistic analysis suggests that the two behaviors share an internal substrate: the MLP neurons and attention heads driving them overlap substantially, and their associated steering directions are positively aligned. We further conduct a preliminary orthogonalized steering exploration, which yields modest, backbone-dependent selectivity gains. Overall, our results suggest that anti-sycophancy should be treated not as a simple suppression problem, but as a selectivity problem, where effective interventions should preserve Rational-Updating while reducing Unsupported-Yielding.
Huanhuan Ma, Henry Peng Zou, Chengze Li +3
University of Illinois Chicago · National University of Singapore
Large language models exhibit sycophancy: the tendency to shift outputs toward user-expressed stances, regardless of correctness or consistency. While prior work has studied this issue and its impacts, rigorous computational linguistic metrics are needed to identify when models are being sycophantic. Here, we introduce SWAY, an unsupervised computational linguistic measure of sycophancy. We develop a counterfactual prompting mechanism to identify how much a model's agreement shifts under positive versus negative linguistic pressure, isolating framing effects from content. Applying this metric to benchmark 6 models, we find that sycophancy increases with epistemic commitment. Leveraging our metric, we introduce a counterfactual mitigation strategy teaching models to consider what the answer would be if opposite assumptions were suggested. While baseline mitigation instructing to be explicitly anti-sycophantic yields moderate reductions, and can backfire, our counterfactual CoT mitigation drives sycophancy to near zero across models, commitment levels, and clause types, while not suppressing responsiveness to genuine evidence. Overall, we contribute a metric for benchmarking sycophancy and a mitigation informed by it.