Mixture-of-Experts (MoE) models appear to encode moral content as robustly as dense models, yet prove far more fragile in their encoding. In OLMoE-1B-7B, linear probes recover moral valence from nearly every expert-layer combination, with mean peak-layer accuracy above 90%. But these representations collapse under levels of activation noise that a dense model of matched size easily tolerates, with a 4.2-fold difference in robustness. We trace this to output dilution. Because the MoE block averages across active experts before contributing to the residual stream, the feedforward signal reaching downstream layers is nearly two orders of magnitude smaller than in a dense MLP. Moral information, our interest, survives aggregation intact but at a scale trivially overwhelmed by perturbation. Routing itself remains stable under noise while the vulnerability originates entirely in the diluted aggregate. Checkpoint trajectories confirm this is architectural, not learned. Experts never specialize and accuracy saturates within the first few thousand steps. In sparse architectures, redundant encoding does not imply robust encoding.
Figures & tables
Figure 1: Dense and MoE encode moral content with near-identical accuracy but very different robustness. (a) Per-layer moral probing accuracy for OLMoE-1B-7B and dense OLMo-2 1B; both peak at 99.0%, differing only at the early layers. (b) Per-layer critical noise σ∗ (smallest σ at which probe accuracy falls below 0.6, on the log grid {0.1,0.3,1.0,3.0,10.0} ): OLMoE is 5.1 × more fragile (mean σ∗ 0.84 vs. 4.25) and concentrates robustness in the final two layers.
Figure 2: No expert moral specialization. (a) Distribution of the 64 per-expert probe accuracies at each layer (box plot) with the per-layer mean overlaid; every expert encodes moral content, with no sparse high-accuracy subset. (b) The per-layer Gini coefficient of expert accuracy stays in [0.016,0.023] , far below any concentration threshold, and is lowest at the late layers where encoding peaks.
Layer
OLMoE MoE std
OLMo MLP std
Ratio
0
0.003
0.448
167 ×
5
0.003
0.291
108 ×
8
0.008
0.459
60 ×
12
0.018
1.070
61 ×
15
0.096
8.779
91 ×
Table 3
Figure 3: Output dilution explains MoE fragility. (a) Per-layer feedforward output scale (standard deviation of the mean-pooled output) for the OLMoE MoE block vs. the dense OLMo-2 MLP; the dense MLP output is 74 × larger on average. (b) Per-layer critical noise for the three MoE perturbation targets: the router is most robust (mean σ∗ 9.1), the aggregated output most fragile (mean σ∗ 0.56), because the output operates on the diluted scale from panel (a).
Figure 4: Specialization never emerges during training. Across 11 OLMoE checkpoints (step 5K–1.2M, 20B–5,117B tokens), peak-layer and overall mean per-expert accuracy (left axis) stay in a 92–94% band from the earliest checkpoint, while the Gini coefficient of expert accuracy (right axis) stays flat near zero. Moral encoding is present from the start and never concentrates into specific experts.
Appendix figures & tables5 assets
Supplementary material from the paper’s appendix.
Appendix
Foundation
Moral
Neutral
care/harm
“Offering shelter to someone stranded in a storm, everyone helped.”
“Offering directions to someone lost in a new city, everyone helped.”
fairness
“The manager promoted the most qualified candidate despite personal ties.”
“The manager promoted the candidate who had applied first.”
loyalty
“She reported her company’s illegal dumping to protect the community.”
“She reported her company’s quarterly earnings to the board.”
Appendix
Table 6
Layer
Mean
Std
Min
Max
Gini
>90%
<60%
0
0.825
0.031
0.760
0.896
0.021
0
0
1
0.825
0.032
0.719
0.896
0.021
0
0
2
0.857
0.034
0.750
0.948
0.022
7
0
3
0.844
0.035
0.740
0.917
0.023
2
0
4
0.880
0.035
0.792
0.938
0.022
20
0
5
0.905
0.030
0.854
0.969
0.019
39
0
Appendix
Table 7
Layer
OLMoE FFN std
OLMo-2 MLP std
Ratio (OLMo/OLMoE)
0
0.003
0.45
167 ×
1
0.014
0.36
25 ×
2
0.051
0.27
5 ×
3
0.008
0.28
34 ×
4
0.008
0.33
41 ×
5
0.003
0.29
108 ×
Appendix
Table 8
Experiment
Seed(s)
Where set
Probing dataset split
42
deepsteer/datasets/pipeline.py
Per-expert probes (Exp 1)
torch default
exp1_2_expert_probing.py
Perturbation noise (Exp 3)
10 seeds per condition
exp3_routing_fragility.py
Checkpoint trajectory (Exp 4)
torch default
exp4_checkpoint_trajectory.py
Appendix
Table 9
Model
Repo
Revision
Used for
OLMoE-1B-7B
allenai/OLMoE-1B-7B-0924
main
Exp 1–3, Exp 5
OLMoE-1B-7B (trajectory)
allenai/OLMoE-1B-7B-0924
step5000-tokens20B through step1200000-tokens5033B
Mixture-of-experts models use input-dependent routing to activate specialized subnetworks, increasing model capacity without running every parameter for each input.
Mixture of Experts (MoE) achieve parameter-efficient scaling through sparse expert routing, yet their internal representations remain poorly understood compared to dense models. We present a systematic comparison of MoE and dense model internals using crosscoders, a variant of sparse autoencoders, that jointly models multiple activation spaces. We train 5-layer dense and MoEs (equal active parameters) on 1B tokens across code, scientific text, and english stories. Using BatchTopK crosscoders with explicitly designated shared features, we achieve ∼87% fractional variance explained and uncover concrete differences in feature organization. The MoE learns significantly fewer unique features compared to the dense model. MoE-specific features also exhibit higher activation density than shared features, whereas dense-specific features show lower density. Our analysis reveals that MoEs develop more specialized, focused representations while dense models distribute information across broader, more general-purpose features.
Marmik Chaudhari, Nishkal Hundia, Idhant Gulati
University of California, Berkeley · University of Maryland
As model parameters scale up, sparsely activated Mixture-of-Experts (MoE) approaches have become essential for model scaling. However, mainstream models currently adopt hierarchical architectures, where experts in different layers, although topologically independent, exhibit substantial similarity and redundancy in their stored content. Compared to the hierarchically arranged Local Expert in existing mainstream models, we hypothesize that inter-layer shared Meta Expert hold greater potential. To this end, we conduct a multi-dimensional empirical comparison between Local and Meta Experts, and find that models incorporating Meta Experts exhibit lower parameter redundancy, together with lower cross-entropy loss and higher downstream task accuracy. With the activated quantity and the expert granularity held fixed, our model matches the performance of Vanilla MoE using only 70--80% of the experts, substantially improving the model's scaling potential. Beyond removing inter-layer redundancy among experts and improving information storage efficiency, our theoretical analysis further shows that, compared with Local Experts, Meta Experts provide hidden states with more flexible residual pathways, thereby improving information retrieval efficiency.
Mixture of Experts (MoE) models enable parameter-efficient scaling through sparse expert activations, yet optimizing their inference and memory costs remains challenging due to limited understanding of their specialization behavior. We present a systematic analysis of expert specialization in MoEs through two complementary approaches: domain-specific routing patterns and an early decoding framework that tracks expert contributions to output representations. Our analysis of the DeepSeekMoE model reveals that despite having 64 routed experts with 6 active for each layer's computation, the model predominantly relies on a few specialized experts, with the top-weighted expert's output closely approximating the full ensemble prediction. We quantitatively validate these findings through a systematic analysis of the token routing distribution, demonstrating that very few experts handle over 50% of routing decisions across different specialized domains. Hidden state similarity between single and ensemble experts for every layer is extremely high, with some layers having cosine similarity as high as 0.95 and perplexity increasing by only 5% when using a single expert across all three domains. Our results indicate that Mixture of Experts models exhibit concentrated expertise highlighting potential opportunities for inference optimization through targeted expert pruning while maintaining model performance and opening avenues towards studying localization of learned knowledge in these models.
The Pennsylvania State University, University Park, PA 16802 USA · University of Maryland, College Park, MD 20742 USA · Harvard University, Cambridge, MA 02138 USA