cs.CLSep 21, 2026

Tracing mechanisms of sycophantic agreement in language models

Authors: Sixing Chen, Zhuofan Josh Ying, Logan Riggs Smith, Jeremy Wertheimer, Natalie Shapira

Organizations: New York University · Cambridge Boston Alignment Initiative · Columbia University · Independent · 

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Sycophantic agreement in language models refers to the tendency to overly affirm a user's stated beliefs or preferences, often at the expense of factual accuracy. Although it is widely recognized as an alignment failure, its underlying mechanisms remain poorly understood. In this work, we use causal mediation analysis to identify the mechanisms behind sycophantic agreement. We show that a stated opinion is incorporated into the residual stream of the final prompt token early, where it biases subsequent answer retrieval. A sparse set of early attention heads carries this opinion signal. Ablating these heads substantially reduces sycophancy while leaving factual accuracy largely intact. The same heads carry the opinion when it is explicitly stated, regardless of how it is phrased. When an opinion is not stated explicitly but instead conveyed through content-free pushback (e.g., ``Are you sure?"), we find a distinct set of heads that suppresses the model's original correct answer to promote a revised answer. By providing a mechanistic account of how opinions induce sycophantic agreement, this work takes a step toward developing more targeted and reliable alignment interventions.

Figures & tables

Figure 1: Schematic of the mechanism. Without a stated opinion (left), the model retrieves the answer by moving information from the residual stream of the correct-answer token to that of the last token. With a stated opinion (right), information from the opinion token is written into the last-token residual stream earlier (opinion registration). This signal then biases the subsequent answer retrieval, which instead moves information from the residual stream of the wrong-answer token to that of the last token (biased retrieval). Gray bars denote the residual stream at each token position across layers; colored arrows trace information about the correct answer (blue), the wrong answer (red), and the stated opinion (yellow) as it moves through the network.
Figure 2: Prompt formats. Blue marks the correct answer and red marks the wrong answer used as the stated opinion in the deceptive run. In the plain run, the model correctly answers (D), Helium. In the deceptive run, the model sycophantically agrees with the stated opinion and answers (B), Hydrogen. In the label-swapped run, the labels attached to these two answers are swapped while their content stays in place. The model’s answer also shifts to (B), but this reflects correct tracking of the relabeled content, since (B) now refers to the correct answer, Helium.
Figure 3: Opinion registration happens before answer retrieval. (A) Restricted softmax probability of the correct answer (blue), the sycophantic answer (red), and the mean of the remaining two answers (gray), obtained via the logit lens at each layer, in the plain (left) and deceptive (right) runs. Both start near chance (0.25, dashed horizontal line) and diverge only after a critical layer (layer 16, dashed vertical line). (B) Normalized logit difference for residual-stream patching, layer by layer, from the plain run into the label-swapped run (cyan) and into the deceptive run (orange). Shaded bands denote the 25th–75th percentile range across examples. The plain-to-label-swap curve rises sharply only after layer 16, while the plain-to-deceptive curve begins rising around layer 12.
Figure 4: Two distinct populations of attention heads carry opinion and retrieve answers. (A) Normalized logit difference from patching individual heads’ output to the last token’s residual stream, from the plain run into the label-swapped run, split into layers 17–31 (top) and layers 0–16 (bottom). Outlined cells mark the ten heads with the highest normalized logit difference in each band, which we subsequently identify as answer retrieval heads (top) and opinion heads (bottom). (B) As in (A), but patching from the plain run into the deceptive run. (C) Normalized logit difference for each head in the plain-to-deceptive patching against the plain-to-label-swap patching, for heads before the critical layer (early heads) and after it (late heads). (D) Classification accuracy of linear probes trained to decode the correct answer and the sycophantic answer from each opinion head’s output to the last token’s residual stream, in the plain and deceptive runs. Black lines mark the baseline accuracy obtained by shuffling the corresponding labels. (E) As in (D), for the answer retrieval heads. (F) Sycophancy rate, deceptive accuracy, and plain accuracy as opinion heads are ablated cumulatively.
Figure 5: Opinion heads generalize across prompt formats and datasets. Each panel shows an example prompt (top) and the correlation between early heads’ normalized logit difference in that format and the original single-round, label-based MMLU setting (bottom). Each point in the scatter plots is one head, with purple points in the top-10 by effect size in both formats and red/orange points in the top-10 in only one format. (A) Free-form question answering using TriviaQA. (B) Multi-round sycophantic agreement where the opinion is expressed in a second turn after the model’s initial correct answer. (C) Description-based opinion where the opinion describes the target answer without containing any of its tokens.
Figure 6: Pushback recruits a distinct set of heads that suppress the correct answer. (A) Example prompt for the pushback setting, in which the user questions the model’s answer without proposing an alternative. (B) Correlation between early heads’ normalized logit difference in the pushback and the original affirmation settings (right). Each point is a head. Purple points are in the top-10 by effect size in both settings, red points are in the top-10 in the affirmation setting only, and green points are in the top-10 in the pushback setting only. (C) Classification accuracy of linear probes trained to decode the correct answer and the sycophantic answer from each pushback-only head’s output, in the first and second rounds. Black lines mark the baseline accuracy obtained by shuffling the corresponding labels.
Appendix figures & tables19 assets

Supplementary material from the paper’s appendix.

Appendix

SubjectQuestions
Anatomy135
Astronomy152
Clinical knowledge265
College biology144
College chemistry100
College medicine168
Appendix
Table 1: MMLU subjects used, with the number of candidate questions in each after subject and length filtering.
Figure S1: Usable examples and sycophancy rates across models and conditions. Left: number of examples passing all screens, shown for the four MMLU conditions (base single-round opinion, multi-round opinion, description-based opinion, and content-free pushback) and for free-form TriviaQA. Right: sycophancy rate on MMLU, defined as the proportion of questions passing the first three screens that at least one stated wrong opinion flips. Sycophancy rates are not shown for TriviaQA, where the sycophantic answer is generated by beam search rather than fixed in advance.
Figure S2: Cumulative patching of opinion heads in Llama-3.1-8B-Instruct. Sycophancy rate in the deceptive run as the outputs of opinion heads are patched from the plain run at the last token position, with heads added one at a time in order of their individual causal effect. The dashed line marks the unpatched deceptive run, in which every example is sycophantic by construction. Patching the four highest-ranked heads reduces the sycophancy rate from 100% to 13.5%, after which additional heads produce little further change.
Figure S3: Effect of ablating each opinion head individually in Llama-3.1-8B-Instruct. Change in deceptive-run and plain-run accuracy when each opinion head is zero-ablated on its own, relative to the unablated model. Heads are ordered by their individual causal effect in the patching analysis. Ablating a single head raises deceptive accuracy by up to 44% while reducing plain accuracy by at most 6%.
Figure S4: Opinion heads are not induction heads in Llama-3.1-8B-Instruct. (A) Prefix-matching (induction) score for every head, by layer and head. (B) Distribution of induction scores over all 1,024 heads. (C) Copying score, measured by normalized direct logit attribution, by layer and head. (D) Distribution of copying scores. In (A) and (C), green outlines mark the ten opinion heads and blue outlines the ten highest-scoring heads on that panel’s score. In (B) and (D), red ticks mark the opinion heads and the dotted line marks the tenth-ranked head. (E) The two criteria plotted jointly, with dotted lines at the 95th percentile of each score. Heads exceeding both thresholds are classified as induction heads (orange, n=7n=7 ). All ten opinion heads (dark red) sit at the null on the copying axis, and none exceeds both thresholds.
Figure S5: Cross-question patching separates reference from content in Llama-3.1-8B-Instruct. Change in log-probability of the first token of the source question’s stated wrong answer when patching the output at the final token position from a source question into an unrelated target question. Answer retrieval heads substantially increase the probability of the source answer, consistent with carrying transferable answer content. Opinion heads produce much smaller effects, supporting the view that they primarily carry a context-dependent representation that must be resolved relative to the original prompt.
Figure S6: Opinion and answer retrieval heads before and after instruction tuning in Llama-3.1-8B. Normalized logit difference for each head in the instruction-tuned model against the base model, computed on the questions usable for both checkpoints. Left: early heads, from plain-to-deceptive patching. Right: late heads, from plain-to-label-swap patching. Each point is a head. Purple points are in the top-10 heads by effect size for both checkpoints, while orange and blue points are in the top-10 heads for only the base or only the instruction-tuned model.
Figure S7: Opinion registration and answer retrieval in Mistral-7B-Instruct-v0.3. (A) Restricted softmax probability of the correct answer (blue), the sycophantic answer (red), and the mean of the remaining two answers (gray), obtained via the logit lens at each layer, in the plain (left) and deceptive (right) runs. The dashed horizontal line marks chance (0.25) and the dashed vertical line the critical layer (layer 17). (B) Normalized logit difference for residual-stream patching, layer by layer, from the plain run into the label-swapped run (cyan) and into the deceptive run (orange). Shaded bands denote the 25th–75th percentile range across examples. (C) Normalized logit difference from patching individual heads’ output to the last token’s residual stream, from the plain run into the label-swapped run, split into layers 18–31 (top) and layers 0–17 (bottom). Outlined cells mark the ten heads with the highest normalized logit difference in each band. (D) As in (C), but patching from the plain run into the deceptive run. (E) Normalized logit difference for each head in the plain-to-deceptive patching against the plain-to-label-swap patching, for heads before the critical layer (early heads) and after it (late heads).
Figure S8: Cumulative patching of opinion heads in Mistral-7B-Instruct-v0.3. Sycophancy rate in the deceptive run as the outputs of opinion heads are patched from the plain run at the last token position, with heads added one at a time in order of their individual causal effect. The dashed line marks the unpatched deceptive run, in which every example is sycophantic by construction. Because the opinion effect is distributed across more heads than in Llama-3.1-8B-Instruct, we patch the twenty highest-ranked heads rather than ten. The first ten heads reduce the sycophancy rate from 100% to 18.6%, after which additional heads produce little further change.
Figure S9: Ablating opinion heads in Mistral-7B-Instruct-v0.3. (A) Change in deceptive-run and plain-run accuracy when each opinion head is zero-ablated on its own. Heads are ordered by their individual causal effect in the patching analysis. Ablating a single head raises deceptive accuracy by up to 16.0% while reducing plain accuracy by at most 4.9%. (B) Sycophancy rate, deceptive accuracy, and plain accuracy as opinion heads are zero-ablated cumulatively.
Figure S10: Generalization across prompt formats in Mistral-7B-Instruct-v0.3. Each panel shows the correlation between early heads’ normalized logit difference in one format and in the original single-round, label-based MMLU setting. Each point is one head. (A) Free-form question answering using TriviaQA. (B) Multi-round sycophantic agreement, in which the opinion is expressed in a second turn after the model’s initial correct answer. (C) Description-based opinion, in which the opinion describes the target answer without containing any of its tokens. In (A) to (C), purple points are in the top-10 by effect size in both formats and red and orange points in the top-10 in only one format. (D) Content-free pushback, compared against the affirmation setting in which the user proposes a specific wrong answer. Purple points are in the top-10 in both settings, red points in the affirmation setting only, and green points in the pushback setting only. As in Llama-3.1-8B-Instruct, the same early heads carry the opinion across all three opinion formats, whereas pushback recruits a largely distinct set.
Figure S11: Opinion heads are not induction heads in Mistral-7B-Instruct-v0.3. (A) Prefix-matching (induction) score for every head, by layer and head. (B) Distribution of induction scores over all 1,024 heads. (C) Copying score, measured by normalized direct logit attribution, by layer and head. (D) Distribution of copying scores. In (A) and (C), green outlines mark the twenty opinion heads and blue outlines the ten highest-scoring heads on that panel’s score. In (B) and (D), red ticks mark the opinion heads and the dotted line marks the tenth-ranked head. (E) The two criteria plotted jointly, with dotted lines at the 95th percentile of each score. Heads exceeding both thresholds are classified as induction heads (orange, n=15n=15 ). Five of the twenty opinion heads (dark red) exceed the induction threshold, but none exceeds the copying threshold, so none is classified as an induction head.
Figure S12: Cross-question patching separates reference from content in Mistral-7B-Instruct-v0.3. Change in log-probability of the first token of the source question’s stated wrong answer when patching the output at the final token position from a source question into an unrelated target question. Answer retrieval heads substantially increase the probability of the source answer, consistent with carrying transferable answer content. Opinion heads produce much smaller effects, supporting the view that they primarily carry a context-dependent representation that must be resolved relative to the original prompt.
Figure S13: Opinion registration and answer retrieval in Gemma-2-9B-it. (A) Restricted softmax probability of the correct answer (blue), the sycophantic answer (red), and the mean of the remaining two answers (gray), obtained via the logit lens at each layer, in the plain (left) and deceptive (right) runs. The dashed horizontal line marks chance (0.25) and the dashed vertical line the critical layer (layer 27). (B) Normalized logit difference for residual-stream patching, layer by layer, from the plain run into the label-swapped run (cyan) and into the deceptive run (orange). Shaded bands denote the 25th–75th percentile range across examples. (C) Normalized logit difference from patching individual heads’ output to the last token’s residual stream, from the plain run into the label-swapped run, split into layers 28–41 (top) and layers 0–27 (bottom). Outlined cells mark the ten heads with the highest normalized logit difference in each band. (D) As in (C), but patching from the plain run into the deceptive run. (E) Normalized logit difference for each head in the plain-to-deceptive patching against the plain-to-label-swap patching, for heads before the critical layer (early heads) and after it (late heads).
Figure S14: Cumulative patching of opinion heads in Gemma-2-9B-it. Sycophancy rate in the deceptive run as the outputs of opinion heads are patched from the plain run at the last token position, with heads added one at a time in order of their individual causal effect. The dashed line marks the unpatched deceptive run, in which every example is sycophantic by construction. As in Mistral-7B-Instruct-v0.3, the opinion effect is distributed across more heads than in Llama-3.1-8B-Instruct, so we patch the twenty highest-ranked heads. The first ten heads reduce the sycophancy rate from 100% to 23.0%, and the remaining ten bring it to 13.1%.
Figure S15: Ablating opinion heads in Gemma-2-9B-it. (A) Change in deceptive-run and plain-run accuracy when each opinion head is zero-ablated on its own. Heads are ordered by their individual causal effect in the patching analysis. Ablating a single head raises deceptive accuracy by up to 17.8% while reducing plain accuracy by at most 5.3%. (B) Sycophancy rate, deceptive accuracy, and plain accuracy as opinion heads are zero-ablated cumulatively.
Figure S16: Generalization across prompt formats in Gemma-2-9B-it. Each panel shows the correlation between early heads’ normalized logit difference in one format and in the original single-round, label-based MMLU setting. Each point is one head. Purple points are in the top-10 by effect size in both formats and red and orange points in the top-10 in only one format. (A) Free-form question answering using TriviaQA. (B) Multi-round sycophantic agreement, in which the opinion is expressed in a second turn after the model’s initial correct answer. (C) Description-based opinion, in which the opinion describes the target answer without containing any of its tokens. As in Llama-3.1-8B-Instruct and Mistral-7B-Instruct-v0.3, the same early heads carry the opinion across all three opinion formats. Content-free pushback is not shown because Gemma-2-9B-it yields only 12 usable examples in that condition ( Figure S1 ), too few to support head-level patching.
Figure S17: Opinion heads are not induction heads in Gemma-2-9B-it. (A) Prefix-matching (induction) score for every head, by layer and head. (B) Distribution of induction scores over all 672 heads. (C) Copying score, measured by normalized direct logit attribution, by layer and head. (D) Distribution of copying scores. In (A) and (C), green outlines mark the twenty opinion heads and blue outlines the ten highest-scoring heads on that panel’s score. In (B) and (D), red ticks mark the opinion heads and the dotted line marks the tenth-ranked head. (E) The two criteria plotted jointly, with dotted lines at the 95th percentile of each score. Heads exceeding both thresholds are classified as induction heads (orange, n=5n=5 ). Three of the twenty opinion heads (dark red) exceed the induction threshold, but none exceeds the copying threshold, so none is classified as an induction head.
Figure S18: Cross-question patching separates reference from content in Gemma-2-9B-it. Change in log-probability of the first token of the source question’s stated wrong answer when patching the output at the final token position from a source question into an unrelated target question. Answer retrieval heads substantially increase the probability of the source answer, consistent with carrying transferable answer content. Opinion heads produce much smaller effects, supporting the view that they primarily carry a context-dependent representation that must be resolved relative to the original prompt.
More in this topic

LLM Sycophancy

Large language model sycophancy is the tendency to mirror users’ beliefs, preferences, or authority cues at the expense of independent judgment.

See all 161 papers

Explore similar work

CardsList
Aug 31, 2026

Sycophantic Agreement Transfers with Neutral Data via Contrastive Preference Optimization

Sycophantic agreement refers to a behavior in which language models excessively affirm the user, often at the cost of factual accuracy. Although sycophantic agreement is a well-known failure of model alignment, there is limited understanding of how it emerges from model training. In this work, we demonstrate that sycophantic agreement can emerge as an unintended consequence of widely used contrastive preference optimization objectives. Using the OLMo 3 post-training pipeline, we show that, for various pairs of teacher models across three families, there is a strong correlation between the log-ratio of the teacher model sycophantic agreement rates and the resulting student model sycophantic agreement rate. We further demonstrate that this unintended transfer is not limited to DPO but also occurs across 6 other preference optimization objectives. To understand whether this effect can be attributed to particular training examples, we analyze the preference data and find that the sycophancy signal is diffused across the entire dataset rather than concentrated in a sparse set of examples: each example appears neutral, i.e., there are no explicit instances of sycophantic agreement, and filtering based on probe-based data attribution or logit-linear selection fails to mitigate sycophancy without removing a large portion of the dataset. Overall, our findings suggest that the teacher models used to generate preference data can interact with alignment training objectives in unexpected ways, generalizing to undesirable and potentially harmful behaviors like sycophantic agreement.

Camila Blank, Zhuofan Ying, Christopher Potts +2

1Stanford University · 2Columbia University

LLM SycophancyEmergent Misalignment in Language ModelsPreference OptimizationPreference Alignment
AskPDF
Apr 21, 2026

LLMs Know They're Wrong and Agree Anyway: The Shared Sycophancy-Lying Circuit

When a language model agrees with a user's false belief, is it failing to detect the error, or noticing and agreeing anyway? We show the latter. Across twelve open-weight models from five labs, spanning small to frontier scale, the same small set of attention heads carries a "this statement is wrong" signal, whether the model is evaluating a claim on its own or being pressured to agree with a user. Silencing these heads flips sycophantic behavior sharply while leaving factual accuracy intact, so the circuit controls deference rather than knowledge. Edge-level path patching confirms that the same head-to-head connections drive sycophancy, factual lying, and instructed lying. Opinion-agreement, where no factual ground truth exists, reuses these head positions but writes into an orthogonal direction, ruling out a simple "truth-direction" reading of the substrate. Alignment training leaves this circuit in place: an RLHF refresh cuts sycophantic behavior roughly tenfold while the shared heads persist or grow, a pattern that replicates on an independent model family and under targeted anti-sycophancy DPO. When these models sycophant, they register that the user is wrong and agree anyway.

Manav Pandey

Georgia Institute of Technology

LLM SycophancyLLM AlignmentDeception in Language ModelsMechanistic InterpretabilityAttention MechanismsCircuits in Language Models+1
AskPDF
Feb 27, 2026

Ask don't tell: Reducing sycophancy in large language models

Sycophancy, the tendency of large language models to favour user-affirming responses over critical engagement, has been identified as an alignment failure, particularly in high-stakes advisory and social contexts. While prior work has documented conversational features correlated with sycophancy, we lack a systematic understanding of what provokes or prevents AI sycophancy. Here, we present a set of controlled experimental studies where we first isolate how input framing influences sycophancy, and second, leverage these findings to develop mitigation strategies. In a nested factorial design, we compare questions to various non-questions where we vary three orthogonal factors: epistemic certainty (statement, belief, conviction), perspective (I- vs user-perspective), and affirmation vs negation. Measuring expressed sycophancy, how sycophantically a model phrases its free-text response, we show that (1) sycophancy is substantially higher in response to non-questions compared to questions. Additionally, we find that (2) sycophancy increases monotonically with epistemic certainty conveyed by the user, and (3) is amplified by I-perspective framing. Building on this, we show that asking a model to convert non-questions into questions before answering significantly reduces sycophancy. Importantly, this effect is stronger than a simple baseline prompt asking models "not to be sycophantic". In a follow-up experiment, we show that these framing effects generalise to choice sycophancy, which answer a model commits to, in a more context-rich, personalised setting: when a model is given extensive knowledge of a user and must select between forced binary choices, the same question vs. statement framing shapes how often it picks the answer aligned with the user's stance. Our work offers a practical and effective input-level mitigation that both developers and users can easily adopt.

Magda Dubois, Cozmin Ududec, Christopher Summerfield +1

UK AI Security Institute, London, UK

LLM SycophancyFraming Effects in Language Models
AskPDF