cs.CLSep 21, 2026
SaveTracing mechanisms of sycophantic agreement in language models
Organizations: New York University · Cambridge Boston Alignment Initiative · Columbia University · Independent ·
Abstract
Sycophantic agreement in language models refers to the tendency to overly affirm a user's stated beliefs or preferences, often at the expense of factual accuracy. Although it is widely recognized as an alignment failure, its underlying mechanisms remain poorly understood. In this work, we use causal mediation analysis to identify the mechanisms behind sycophantic agreement. We show that a stated opinion is incorporated into the residual stream of the final prompt token early, where it biases subsequent answer retrieval. A sparse set of early attention heads carries this opinion signal. Ablating these heads substantially reduces sycophancy while leaving factual accuracy largely intact. The same heads carry the opinion when it is explicitly stated, regardless of how it is phrased. When an opinion is not stated explicitly but instead conveyed through content-free pushback (e.g., ``Are you sure?"), we find a distinct set of heads that suppresses the model's original correct answer to promote a revised answer. By providing a mechanistic account of how opinions induce sycophantic agreement, this work takes a step toward developing more targeted and reliable alignment interventions.
Figures & tables
Figure 1: Schematic of the mechanism. Without a stated opinion (left), the model retrieves the answer by moving information from the residual stream of the correct-answer token to that of the last token. With a stated opinion (right), information from the opinion token is written into the last-token residual stream earlier (opinion registration). This signal then biases the subsequent answer retrieval, which instead moves information from the residual stream of the wrong-answer token to that of the last token (biased retrieval). Gray bars denote the residual stream at each token position across layers; colored arrows trace information about the correct answer (blue), the wrong answer (red), and the stated opinion (yellow) as it moves through the network.
Figure 2: Prompt formats. Blue marks the correct answer and red marks the wrong answer used as the stated opinion in the deceptive run. In the plain run, the model correctly answers (D), Helium. In the deceptive run, the model sycophantically agrees with the stated opinion and answers (B), Hydrogen. In the label-swapped run, the labels attached to these two answers are swapped while their content stays in place. The model’s answer also shifts to (B), but this reflects correct tracking of the relabeled content, since (B) now refers to the correct answer, Helium.
Figure 3: Opinion registration happens before answer retrieval. (A) Restricted softmax probability of the correct answer (blue), the sycophantic answer (red), and the mean of the remaining two answers (gray), obtained via the logit lens at each layer, in the plain (left) and deceptive (right) runs. Both start near chance (0.25, dashed horizontal line) and diverge only after a critical layer (layer 16, dashed vertical line). (B) Normalized logit difference for residual-stream patching, layer by layer, from the plain run into the label-swapped run (cyan) and into the deceptive run (orange). Shaded bands denote the 25th–75th percentile range across examples. The plain-to-label-swap curve rises sharply only after layer 16, while the plain-to-deceptive curve begins rising around layer 12.
Figure 4: Two distinct populations of attention heads carry opinion and retrieve answers. (A) Normalized logit difference from patching individual heads’ output to the last token’s residual stream, from the plain run into the label-swapped run, split into layers 17–31 (top) and layers 0–16 (bottom). Outlined cells mark the ten heads with the highest normalized logit difference in each band, which we subsequently identify as answer retrieval heads (top) and opinion heads (bottom). (B) As in (A), but patching from the plain run into the deceptive run. (C) Normalized logit difference for each head in the plain-to-deceptive patching against the plain-to-label-swap patching, for heads before the critical layer (early heads) and after it (late heads). (D) Classification accuracy of linear probes trained to decode the correct answer and the sycophantic answer from each opinion head’s output to the last token’s residual stream, in the plain and deceptive runs. Black lines mark the baseline accuracy obtained by shuffling the corresponding labels. (E) As in (D), for the answer retrieval heads. (F) Sycophancy rate, deceptive accuracy, and plain accuracy as opinion heads are ablated cumulatively.
Figure 5: Opinion heads generalize across prompt formats and datasets. Each panel shows an example prompt (top) and the correlation between early heads’ normalized logit difference in that format and the original single-round, label-based MMLU setting (bottom). Each point in the scatter plots is one head, with purple points in the top-10 by effect size in both formats and red/orange points in the top-10 in only one format. (A) Free-form question answering using TriviaQA. (B) Multi-round sycophantic agreement where the opinion is expressed in a second turn after the model’s initial correct answer. (C) Description-based opinion where the opinion describes the target answer without containing any of its tokens.
Figure 6: Pushback recruits a distinct set of heads that suppress the correct answer. (A) Example prompt for the pushback setting, in which the user questions the model’s answer without proposing an alternative. (B) Correlation between early heads’ normalized logit difference in the pushback and the original affirmation settings (right). Each point is a head. Purple points are in the top-10 by effect size in both settings, red points are in the top-10 in the affirmation setting only, and green points are in the top-10 in the pushback setting only. (C) Classification accuracy of linear probes trained to decode the correct answer and the sycophantic answer from each pushback-only head’s output, in the first and second rounds. Black lines mark the baseline accuracy obtained by shuffling the corresponding labels.
Appendix figures & tables19 assets
Supplementary material from the paper’s appendix.
Appendix
| Subject | Questions |
|---|---|
| Anatomy | 135 |
| Astronomy | 152 |
| Clinical knowledge | 265 |
| College biology | 144 |
| College chemistry | 100 |
| College medicine | 168 |
Table 1: MMLU subjects used, with the number of candidate questions in each after subject and length filtering.
Figure S1: Usable examples and sycophancy rates across models and conditions. Left: number of examples passing all screens, shown for the four MMLU conditions (base single-round opinion, multi-round opinion, description-based opinion, and content-free pushback) and for free-form TriviaQA. Right: sycophancy rate on MMLU, defined as the proportion of questions passing the first three screens that at least one stated wrong opinion flips. Sycophancy rates are not shown for TriviaQA, where the sycophantic answer is generated by beam search rather than fixed in advance.
Figure S2: Cumulative patching of opinion heads in Llama-3.1-8B-Instruct. Sycophancy rate in the deceptive run as the outputs of opinion heads are patched from the plain run at the last token position, with heads added one at a time in order of their individual causal effect. The dashed line marks the unpatched deceptive run, in which every example is sycophantic by construction. Patching the four highest-ranked heads reduces the sycophancy rate from 100% to 13.5%, after which additional heads produce little further change.
Figure S3: Effect of ablating each opinion head individually in Llama-3.1-8B-Instruct. Change in deceptive-run and plain-run accuracy when each opinion head is zero-ablated on its own, relative to the unablated model. Heads are ordered by their individual causal effect in the patching analysis. Ablating a single head raises deceptive accuracy by up to 44% while reducing plain accuracy by at most 6%.
Figure S4: Opinion heads are not induction heads in Llama-3.1-8B-Instruct. (A) Prefix-matching (induction) score for every head, by layer and head. (B) Distribution of induction scores over all 1,024 heads. (C) Copying score, measured by normalized direct logit attribution, by layer and head. (D) Distribution of copying scores. In (A) and (C), green outlines mark the ten opinion heads and blue outlines the ten highest-scoring heads on that panel’s score. In (B) and (D), red ticks mark the opinion heads and the dotted line marks the tenth-ranked head. (E) The two criteria plotted jointly, with dotted lines at the 95th percentile of each score. Heads exceeding both thresholds are classified as induction heads (orange, ). All ten opinion heads (dark red) sit at the null on the copying axis, and none exceeds both thresholds.
Figure S5: Cross-question patching separates reference from content in Llama-3.1-8B-Instruct. Change in log-probability of the first token of the source question’s stated wrong answer when patching the output at the final token position from a source question into an unrelated target question. Answer retrieval heads substantially increase the probability of the source answer, consistent with carrying transferable answer content. Opinion heads produce much smaller effects, supporting the view that they primarily carry a context-dependent representation that must be resolved relative to the original prompt.
Figure S6: Opinion and answer retrieval heads before and after instruction tuning in Llama-3.1-8B. Normalized logit difference for each head in the instruction-tuned model against the base model, computed on the questions usable for both checkpoints. Left: early heads, from plain-to-deceptive patching. Right: late heads, from plain-to-label-swap patching. Each point is a head. Purple points are in the top-10 heads by effect size for both checkpoints, while orange and blue points are in the top-10 heads for only the base or only the instruction-tuned model.
Figure S7: Opinion registration and answer retrieval in Mistral-7B-Instruct-v0.3. (A) Restricted softmax probability of the correct answer (blue), the sycophantic answer (red), and the mean of the remaining two answers (gray), obtained via the logit lens at each layer, in the plain (left) and deceptive (right) runs. The dashed horizontal line marks chance (0.25) and the dashed vertical line the critical layer (layer 17). (B) Normalized logit difference for residual-stream patching, layer by layer, from the plain run into the label-swapped run (cyan) and into the deceptive run (orange). Shaded bands denote the 25th–75th percentile range across examples. (C) Normalized logit difference from patching individual heads’ output to the last token’s residual stream, from the plain run into the label-swapped run, split into layers 18–31 (top) and layers 0–17 (bottom). Outlined cells mark the ten heads with the highest normalized logit difference in each band. (D) As in (C), but patching from the plain run into the deceptive run. (E) Normalized logit difference for each head in the plain-to-deceptive patching against the plain-to-label-swap patching, for heads before the critical layer (early heads) and after it (late heads).
Figure S8: Cumulative patching of opinion heads in Mistral-7B-Instruct-v0.3. Sycophancy rate in the deceptive run as the outputs of opinion heads are patched from the plain run at the last token position, with heads added one at a time in order of their individual causal effect. The dashed line marks the unpatched deceptive run, in which every example is sycophantic by construction. Because the opinion effect is distributed across more heads than in Llama-3.1-8B-Instruct, we patch the twenty highest-ranked heads rather than ten. The first ten heads reduce the sycophancy rate from 100% to 18.6%, after which additional heads produce little further change.
Figure S9: Ablating opinion heads in Mistral-7B-Instruct-v0.3. (A) Change in deceptive-run and plain-run accuracy when each opinion head is zero-ablated on its own. Heads are ordered by their individual causal effect in the patching analysis. Ablating a single head raises deceptive accuracy by up to 16.0% while reducing plain accuracy by at most 4.9%. (B) Sycophancy rate, deceptive accuracy, and plain accuracy as opinion heads are zero-ablated cumulatively.
Figure S10: Generalization across prompt formats in Mistral-7B-Instruct-v0.3. Each panel shows the correlation between early heads’ normalized logit difference in one format and in the original single-round, label-based MMLU setting. Each point is one head. (A) Free-form question answering using TriviaQA. (B) Multi-round sycophantic agreement, in which the opinion is expressed in a second turn after the model’s initial correct answer. (C) Description-based opinion, in which the opinion describes the target answer without containing any of its tokens. In (A) to (C), purple points are in the top-10 by effect size in both formats and red and orange points in the top-10 in only one format. (D) Content-free pushback, compared against the affirmation setting in which the user proposes a specific wrong answer. Purple points are in the top-10 in both settings, red points in the affirmation setting only, and green points in the pushback setting only. As in Llama-3.1-8B-Instruct, the same early heads carry the opinion across all three opinion formats, whereas pushback recruits a largely distinct set.
Figure S11: Opinion heads are not induction heads in Mistral-7B-Instruct-v0.3. (A) Prefix-matching (induction) score for every head, by layer and head. (B) Distribution of induction scores over all 1,024 heads. (C) Copying score, measured by normalized direct logit attribution, by layer and head. (D) Distribution of copying scores. In (A) and (C), green outlines mark the twenty opinion heads and blue outlines the ten highest-scoring heads on that panel’s score. In (B) and (D), red ticks mark the opinion heads and the dotted line marks the tenth-ranked head. (E) The two criteria plotted jointly, with dotted lines at the 95th percentile of each score. Heads exceeding both thresholds are classified as induction heads (orange, ). Five of the twenty opinion heads (dark red) exceed the induction threshold, but none exceeds the copying threshold, so none is classified as an induction head.
Figure S12: Cross-question patching separates reference from content in Mistral-7B-Instruct-v0.3. Change in log-probability of the first token of the source question’s stated wrong answer when patching the output at the final token position from a source question into an unrelated target question. Answer retrieval heads substantially increase the probability of the source answer, consistent with carrying transferable answer content. Opinion heads produce much smaller effects, supporting the view that they primarily carry a context-dependent representation that must be resolved relative to the original prompt.
Figure S13: Opinion registration and answer retrieval in Gemma-2-9B-it. (A) Restricted softmax probability of the correct answer (blue), the sycophantic answer (red), and the mean of the remaining two answers (gray), obtained via the logit lens at each layer, in the plain (left) and deceptive (right) runs. The dashed horizontal line marks chance (0.25) and the dashed vertical line the critical layer (layer 27). (B) Normalized logit difference for residual-stream patching, layer by layer, from the plain run into the label-swapped run (cyan) and into the deceptive run (orange). Shaded bands denote the 25th–75th percentile range across examples. (C) Normalized logit difference from patching individual heads’ output to the last token’s residual stream, from the plain run into the label-swapped run, split into layers 28–41 (top) and layers 0–27 (bottom). Outlined cells mark the ten heads with the highest normalized logit difference in each band. (D) As in (C), but patching from the plain run into the deceptive run. (E) Normalized logit difference for each head in the plain-to-deceptive patching against the plain-to-label-swap patching, for heads before the critical layer (early heads) and after it (late heads).
Figure S14: Cumulative patching of opinion heads in Gemma-2-9B-it. Sycophancy rate in the deceptive run as the outputs of opinion heads are patched from the plain run at the last token position, with heads added one at a time in order of their individual causal effect. The dashed line marks the unpatched deceptive run, in which every example is sycophantic by construction. As in Mistral-7B-Instruct-v0.3, the opinion effect is distributed across more heads than in Llama-3.1-8B-Instruct, so we patch the twenty highest-ranked heads. The first ten heads reduce the sycophancy rate from 100% to 23.0%, and the remaining ten bring it to 13.1%.
Figure S15: Ablating opinion heads in Gemma-2-9B-it. (A) Change in deceptive-run and plain-run accuracy when each opinion head is zero-ablated on its own. Heads are ordered by their individual causal effect in the patching analysis. Ablating a single head raises deceptive accuracy by up to 17.8% while reducing plain accuracy by at most 5.3%. (B) Sycophancy rate, deceptive accuracy, and plain accuracy as opinion heads are zero-ablated cumulatively.
Figure S16: Generalization across prompt formats in Gemma-2-9B-it. Each panel shows the correlation between early heads’ normalized logit difference in one format and in the original single-round, label-based MMLU setting. Each point is one head. Purple points are in the top-10 by effect size in both formats and red and orange points in the top-10 in only one format. (A) Free-form question answering using TriviaQA. (B) Multi-round sycophantic agreement, in which the opinion is expressed in a second turn after the model’s initial correct answer. (C) Description-based opinion, in which the opinion describes the target answer without containing any of its tokens. As in Llama-3.1-8B-Instruct and Mistral-7B-Instruct-v0.3, the same early heads carry the opinion across all three opinion formats. Content-free pushback is not shown because Gemma-2-9B-it yields only 12 usable examples in that condition ( Figure S1 ), too few to support head-level patching.
Figure S17: Opinion heads are not induction heads in Gemma-2-9B-it. (A) Prefix-matching (induction) score for every head, by layer and head. (B) Distribution of induction scores over all 672 heads. (C) Copying score, measured by normalized direct logit attribution, by layer and head. (D) Distribution of copying scores. In (A) and (C), green outlines mark the twenty opinion heads and blue outlines the ten highest-scoring heads on that panel’s score. In (B) and (D), red ticks mark the opinion heads and the dotted line marks the tenth-ranked head. (E) The two criteria plotted jointly, with dotted lines at the 95th percentile of each score. Heads exceeding both thresholds are classified as induction heads (orange, ). Three of the twenty opinion heads (dark red) exceed the induction threshold, but none exceeds the copying threshold, so none is classified as an induction head.
Figure S18: Cross-question patching separates reference from content in Gemma-2-9B-it. Change in log-probability of the first token of the source question’s stated wrong answer when patching the output at the final token position from a source question into an unrelated target question. Answer retrieval heads substantially increase the probability of the source answer, consistent with carrying transferable answer content. Opinion heads produce much smaller effects, supporting the view that they primarily carry a context-dependent representation that must be resolved relative to the original prompt.
More in this topic
LLM Sycophancy
Large language model sycophancy is the tendency to mirror users’ beliefs, preferences, or authority cues at the expense of independent judgment.
See all 161 papers