cs.AIAug 16, 2026
SaveTHESIS-MoE: Trainable Hierarchical Extraction and SteerIng of Sycophancy in Mixture-of-Experts
Organizations: Department of Electrical and Computer Engineering, American University of Beirut, Riad El-Solh, Beirut 1107 2020, Lebanon
Abstract
Sycophancy, the tendency of a language model to change its answer to match a user's stated belief, is a common alignment failure. Existing activation steering methods typically apply a single contrastive direction uniformly throughout the model, which is an unconditional intervention that alters activations even when no sycophantic behavior is present, trading knowledge retention for behavioral correction. In Mixture-of-Experts (MoE) models, prior work further suggests that behavior is encoded within expert computations rather than routing decisions alone, making precise behavioral steering particularly challenging. In this work, we introduce a shared contrastive signal, built from matched prompts with and without a stated belief, that identifies where sycophancy lives across the MoE hierarchy and drives interventions that act only where the behavior is present. We formulate localization as a causal search over a granularity ladder of MoE blocks, experts, attention blocks, and heads, and compare unconditional subtraction against two conditional alternatives: an analytic projection-based subtraction and a learned per-token gate that steers the model away from sycophancy while keeping its weights frozen. We evaluate on three MoE models measuring sycophancy alongside general knowledge and reasoning benchmarks. Our conditional interventions removed up to 90% of the belief-induced sycophancy. Our results demonstrate that sycophancy resides in identifiable computational subcircuits and can be selectively steered while maintaining a favorable removal-retention trade-off.
Figures & tables
Figure 1: The unit of intervention. Every attention head and expert carries a gate ; the gated contributions and also include gated mechanisms at the output stream.
| Nudge | No-nudge | |
|---|---|---|
| Conforming | belief, agrees | neutral, agrees |
| Nonconforming | belief, opposes | neutral, opposes |
Table 1: Each item appears in four matched forms. Nudge versus no-nudge defines the belief direction and belief push, whereas contrastive activation addition compares conforming and nonconforming answers only under the nudge condition.
| Sycophancy data | Retention data | ||||
|---|---|---|---|---|---|
| Split | Items | Balance | Set | Items | Per seed |
| Matched | 8234 | 4117/4117 | MMLU | 1482 | 741 |
| Train | 4940 | 2470/2470 | GSM8K | 1319 | 659 |
| Dev. | 1646 | 823/823 | TruthfulQA | 817 | 408 |
| Test | 1648 | 824/824 | Perplexity | 1723 | 861 |
Table 2: Dataset splits and retention sets. Splits are balanced, identifier-disjoint, and verified to have zero leakage.
| Method | Description |
|---|---|
| Raw | No intervention; fixes the baseline |
| Prompted | Instructed to be objective |
| FARE | Reweights the router away from belief experts |
| CAA subtraction | Condition direction, unconditional; swept over nine sets |
| Cond. subtraction | Condition direction, conditional; swept over nine sets |
| Learned gate | Conditional; all sets jointly |
Table 3: The methods compared. The subtraction methods are swept over nine sets of the identified components, from a single component type up to all together.
| Metric | Target after edit |
|---|---|
| Belief push, Eq. ( 3 ) | to zero |
| Nudge shift | to zero |
| No-nudge rate (anchor) | unchanged |
| MMLU | baseline |
| GSM8K | baseline |
| TruthfulQA | baseline |
Table 4: The evaluation panel. The upper block measures the behavior we remove and the lower block the knowledge we keep.
Figure 4: The central result. For five methods across three models, each cell reports belief push removed together with GSM8K, MMLU, TruthfulQA, and perplexity, each shown relative to baseline (%); for accuracy this is the ratio of edited to baseline score, and for perplexity the inverted ratio, so that above is better on every metric. The two conditional methods remove most of the push while holding GSM8K at or above baseline; routing and constant subtraction trade retention for removal.
| Axis | Values |
|---|---|
| Gate structure | Flat, conditional, shared |
| Edit class | Frozen direction, learned direction, component scaling |
| Schedule | Simultaneous, coarse-to-fine |
| Initialization | Warm localized, cold global |
Table 5: Design axes used in the conditional-gating sweep.
Figure 5: Every operating point for all methods and models, plotting removed belief push against GSM8K relative to baseline (%). The subtractions and FARE give many points because they are swept over component sets and coefficients, unlike the single-point gate and prompting.
Appendix figures & tables15 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 6: The frontier, one model at a time. Each point is one operating point, with belief push removed (%) on the horizontal axis and GSM8K retained (%) on the vertical axis; marker shape and colour denote the method (see legend), and the dashed line marks baseline retention. Points toward the upper right remove more push while keeping more reasoning. The learned gate and conditional subtraction lead in every model; the shaded band marks the high-removal region where retention stays near baseline.
Figure 7: A single ranking. For each model, the panel Z-score combines behavioral removal with retention into one all-round score (higher is better); bars are ordered within each panel and labelled with their value. The two conditional methods, conditional subtraction and the learned gate, rank first in every model, while the routing baseline (FARE) ranks last.
Figure 8: The carrying heads. Per-head causal effect on the belief push, shown as a heatmap over layer (horizontal) and head index (vertical) with colour giving the magnitude of the effect; the selected heads are boxed in magenta, eight in OLMoE, ten in Qwen, and thirteen in Granite. A few heads stand out against a near-silent background, all in the mid-to-late layers.
Figure 9: The carrying experts. Each selected expert is placed by layer (horizontal) and expert index (vertical) and labelled by its location, five in OLMoE and six in Granite. Qwen has none, because its always-on shared experts cannot appear in a routed contrast, the first sign that the shared-expert regime behaves differently.
Figure 10: Belief accumulates in the mid-to-late layers. For each model, the left panel tracks the model’s lean toward the user’s stated option (logits, vertical axis) as the residual stream passes through successive layers (horizontal axis), comparing a belief run against a matched no-belief run; the shaded region between the two curves is the preference caused by the stated opinion, and the dotted curve is a random-direction control. The right panel shows how much preference each individual layer contributes, with the dashed marker at the layer by which half of the total preference has formed. In every model the lean is flat through the early layers and rises sharply in the upper layers.
Figure 11: Where the belief signal is read and carried, per head. For each model, the left heatmap gives each head’s estimated causal effect on the belief push and the middle heatmap gives how much attention each head places on the stated-opinion text, both over layer (vertical) and head index (horizontal). The right panel plots attention on the opinion (horizontal) against causal effect (vertical), with colour showing the extra attention a head pays when an opinion is present; the labelled heads in the upper region both attend to the opinion and carry its effect, which we call transporter heads.
Figure 12: GSM8K holds until the coefficient grows large. Each panel plots GSM8K accuracy (vertical axis) against the conditional-subtraction strength (horizontal axis). Colour marks the component set the edit is applied to: S1 attention blocks; S2 MoE blocks; S3 attention and MoE blocks; S4 attention heads; S5 experts; S6 heads and experts; S7 attention blocks and heads; S8 MoE blocks and experts; and S9 all located components. Within each set, thin lines are the individual seeds and the thick line is their mean. Accuracy stays roughly flat at low and collapses once becomes large, at a threshold that differs by model.
Figure 13: Main effect of each design axis on belief push removed, per model (one bar per model; higher removes more). The axes and levels are edit class (ComponentMask CM, VectorMaskFrozen VF, VectorMaskLearned VL), gate structure (FlatGate FG, CondGate CG, SharedGate SG), schedule (FullTrain FT, FineToCoarseTrain FC), and initialisation (WarmStart W, ColdStart C). The conditional gate (CG) and full schedule (FT) remove the most push across all three models, motivating those two choices in the selected configuration.
Figure 14: Choosing the gate. Each row is a candidate configuration named in the edit-gate-schedule-init convention of Figure 13 (for example VF-CG-FT-C), restricted to configurations that qualify on all three models. Rows are scored by their distance to complete removal under reasoning and fluency penalties, with over-removal runs discarded; lower is better, the starred row is the winner, and the final column notes why each runner-up was not chosen. The winner, VF-CG-FT-C, a frozen conditional gate (VectorMaskFrozen with CondGate) trained on a full schedule from a cold start, is the pick carried into the seed study.
Figure 15: Choosing the gate architecture, one territory per model. Each point is a configuration labelled edit-gate-schedule-init; the horizontal axis is nudge removed (%) and the vertical axis is the GSM8K change versus baseline (negative means reasoning lost). Each shaded region encloses one model’s top-ten configurations, and the star, pentagon, and cross mark that model’s first, second, and third best picks. The acronym key is given in the side panel.
Figure 16: Architecture sweep on the removal-against-perplexity plane, used for configuration selection. The horizontal axis is nudge removed (%) and the vertical axis is the perplexity change versus baseline (positive means fluency lost), with the dashed line marking the honest ceiling and the background shading giving the distance-to-ideal ranking score (brighter is better). Perplexity here is the decision-token-off control of Section 4.3 , so this plane screens configurations for gross perplexity regressions rather than measuring open-ended fluency; the selected configurations sit at high removal with perplexity near baseline.
Figure 17: The edit is sparse. Per-site edit amount ( ) of the selected gate at its checkpoint, one bar per component with the dashed line marking the inclusion threshold. A handful of attention and mixture-of-experts sites carry most of the edit, matching the concentrated placement of Figure 2 .
Figure 18: What the gate edits, and what it leaves. The most and least edited components of the selected gate, ranked by edit amount and coloured by component type. The heavily edited sites are mid-to-late-layer attention and expert components, while the least edited are individual attention heads.
Figure 19: The localization is stable. Gate activity across the three seeds (s1, s2, s4), with colour giving each component’s edit amount. The set of components the gate opens on is consistent from seed to seed, so the located support is a property of the model rather than of a single run.
Figure 20: Method comparison across the three MoE models. Each axis is a metric (Removal, Invariance, MMLU, GSM8K, TruthfulQA, PPL), normalised so that farther out is better, and each closed curve is one method. Conditional subtraction (Cond sub) and the learned gate achieve high removal while retaining capability, whereas prompting and FARE trade little removal for capability or fail to remove sycophancy.
More in this topic
LLM Sycophancy
Large language model sycophancy is the tendency to mirror users’ beliefs, preferences, or authority cues at the expense of independent judgment.
See all 161 papers