Rethinking SAE Evaluation: An Atomic Interpretable Unit Framework Reveals Hidden Polysemanticity and Redundancy

Riteng Zhang; Xiaoqian Wang

Figure 5: Empirical validation of decoder cosine similarity as a proxy for semantic alignment. ( 5(a) ) Pooled across all 95,206 feature pairs, mean explanation similarity rises monotonically with decoder cosine similarity. ( 5(b) ) Per-SAE Spearman

r_{s}

distribution shows the effect is consistent across SAEs, not driven by outliers.

Figure 6: Pairwise scatter of MSE against

R^{2}

(plotted as

\text{VIF}=1/(1-R^{2})

for log-scale clarity) and cosine similarity, across 610 SAEs from 7 families. MSE ranks SAEs near-identically to both alternatives (

|r_{s}|\geq 0.97

), confirming MSE as a valid simplifying proxy for reporting.

Figure 7: Synthetic validation of DAME on ground-truth concepts. For each architecture, horizontal bars show the Q1 monosemanticity deficit (mean Q1

-

mean Q2–Q4) of each individual SAE; stars mark statistical significance (Mann–Whitney U,

p<0.05

). Subplot titles show the per-architecture count of significant SAEs out of 15: JumpReLU 15/15 (strongest); Gated 3/15 (weakest). 71 of 105 SAEs (68%) overall show significant Q1 deficit.

	TPP		RAVEL
Width	C2 alone	C1 $+$ C3	C2 alone	C1 $+$ C3
4k	$+0.832^{***}$	$-0.178$ n.s.	$+0.983^{***}$	$-0.453^{**}$
16k	$+0.755^{***}$	$+0.152$ n.s.	$+0.975^{***}$	$-0.332^{*}$
65k	$+0.711^{***}$	$+0.231^{*}$	$+0.955^{***}$	$-0.187$ n.s.

Appendix

Table 3: Per-width Spearman correlations of C2 alone vs. the C1

+

C3 joint proxy with TPP and RAVEL. The C2 bottleneck dominates at all widths; the intended C1

+

C3 signal only begins to emerge at 65k, after the C2

\leftrightarrow

C3 anti-correlation weakens.

Method	Prompt Tokens	Output Tokens	Runtime (5 ex.)
Explanation	397 + 566	29.90	3.14s
Detection/Fuzzing	725 + 53	11.99	4.29s
Simulation	24,074.85	1,598.1	73.9s

Appendix

Table 4: Token and runtime comparison between scoring methods. Detection and fuzzing achieve comparable accuracy with approximately 5× fewer tokens and 17× less runtime than simulation.

Stage	Prior Work	DAME
Feature Sampling	Uniform random (Bills et al.) or probe-based (SAEBench)	Dominance-stratified feature sampling by mean dominance $\bar{\rho}_{i}$
Explanation Generation	Show activation values; unbounded explanations	Hide magnitudes; length-constrained explanations
Scoring	Binary classification (Delphi) or continuous correlation (Bills et al.)	Activation-balanced contrastive validation
Aggregation	Uniform mean: $\frac{1}{N}\sum_{i}s_{i}$	Per-quantile + dominance-weighted: $\sum_{i}w_{i}^{\text{dom}}s_{i}$

Appendix

Table 5: Comparison of DAME to prior automated interpretability approaches.

Family	Avg Active	Dead Prop (mean)	MSE (mean)
GPT2-Small	605.0	0.214	1.33e-01
OPT-125M	279.9	0.460	3.40e-04
Tinystories	376.4	0.187	3.71e-02
Canrager	392.2	0.500	7.15e+00
Saebench Gemma	2013.2	0.294	1.46e+02
Saebench Pythia	1106.2	0.392	8.28e+00

Appendix

Table 6: Summary statistics by SAE family for sparsity (average active features), dead feature proportion, and MSE.

Figure 8: Average active features by SAE family. Lower values indicate sparser (more efficient) activations. SAEBench Gemma shows the highest number of active features (less sparse), while OPT-125M achieves the lowest (most sparse).

Figure 9: Mean squared error (MSE) by SAE family. Lower values indicate better reconstruction fidelity. OPT-125M achieves the lowest MSE, while SAEBench Gemma shows higher reconstruction error.

Figure 10: Average active features by architecture across all families. TopK architecture achieves the lowest sparsity, while other architectures show more variable performance.

Figure 11: Mean squared error by architecture across all families. Gated architecture shows the best average reconstruction fidelity, while TopK and BatchTopK show higher variance.

Architecture	Median $L_{0}$	$L_{0}$ /dict (%)	$L_{0}$ / $d_{\text{in}}$ (%)
BatchTopK	49	0.30	5.21
TopK	100	0.61	6.94
Matryoshka	50	0.49	5.21
PAnneal	292	1.31	20.56
Standard ReLU	354	2.17	32.81
Gated	1383	17.34	126.89

Appendix

Table 7: Per-architecture median

L_{0}

in absolute count, as a percentage of dictionary size, and as a percentage of model dimensionality

d_{\text{in}}

. Computed per-SAE then aggregated across SAEs of each architecture (each SAE weighted equally). Last column is the most informative: a sparse decomposition should have

L_{0}\ll d_{\text{in}}

. Gated’s

\sim

127% is in the regime where the SAE is no longer sparse; JumpReLU’s

\sim

53% is borderline.

Figure 12: Mean explanation length by architecture and quantile. Q1 explanations are about 0.6 words longer than Q2–Q4 on average (23.6 vs. 23.0 words; paired

p<10^{-21}

).

Figure 13: Mean perfect ranking accuracy by SAE family and quantile (token-based). The Q1 underperformance pattern varies by family, with some families showing Q1 performing better than Q2–Q4.

Family	Q1	Q2	Q3	Q4	Diff
GPT2-Small	65.3	62.5	62.5	62.7	- 2.7
OPT125M	44.0	43.3	42.3	44.5	- 0.6
Tinystories	64.9	66.1	69.5	68.3	-3.1
Canrager	63.1	68.4	67.4	67.7	-4.7
Jbloom	66.5	67.0	67.1	69.6	-1.4
Saebench Gemma	60.7	62.8	63.2	64.6	-2.8

Appendix

Table 8: Per-family average DAME perfect ranking accuracy by quantile.

Architecture	Q1	Q2	Q3	Q4	Diff
BatchTopK	57.3	68.5	65.6	67.2	-9.8
Gated	62.4	61.1	63.2	66.3	-1.1
JumpReLU	60.5	63.1	64.6	68.2	-4.8
Matryoshka	57.4	64.1	64.5	68.2	-8.2
PAnneal	69.2	71.1	74.1	77.3	-5.0
Standard ReLU	63.9	65.3	72.1	75.8	-7.2

Appendix

Table 9: Per-architecture quantile accuracy for SAEBench Pythia family (token-based evaluation).

Family	Q1	Q2	Q3	Q4	Diff
GPT2-Small	25.3	23.0	22.8	22.6	- 2.5
OPT125M	23.3	23.8	23.8	23.7	-0.5
Tinystories	21.9	22.7	22.4	22.8	-0.7
Canrager	23.7	23.1	23.2	23.2	- 0.5
Jbloom	24.2	22.3	23.0	22.9	- 1.5
Saebench Gemma	24.0	23.0	23.2	23.2	- 0.9

Appendix

Table 10: Per-family average explanation length (number of words) by quantile.

Metric	Positive	Negative
Sparsity $\rightarrow$ Perfect Ranking Acc	35 (39.8%)	53 (60.2%)
Sparsity $\rightarrow$ Q1 Perfect Ranking Acc	50 (56.8%)	38 (43.2%)
Sparsity $\rightarrow$ Pairwise Acc	34 (38.6%)	54 (61.4%)
Sparsity $\rightarrow$ Q1 Pairwise Acc	52 (59.8%)	35 (40.2%)

Appendix

Table 11: Aggregated sparsity correlation results across 88 SAE configurations (controlling for width and architecture). Positive correlations indicate that less sparse SAEs (more active features) have higher interpretability scores.

Figure 14: Q1 performance gap (Q1 accuracy minus Q2–Q4 average, in percentage points) by family and architecture. Negative values indicate Q1 underperformance. The pattern varies substantially across combinations, with SAEBench Pythia showing larger gaps than GPT2-Small.

Figure 15: Overall perfect ranking accuracy (averaged across all quantiles) by family and architecture. Higher values indicate better overall monosemanticity. PAnneal architecture consistently achieves high scores across families.

Figure 16: Overall pairwise accuracy (averaged across all quantiles) by family and architecture. Higher values indicate better overall monosemanticity. Patterns are similar to perfect ranking accuracy, with PAnneal and Gated architectures performing well.

Figure 17: SAEBench Pythia: Architecture comparison showing overall vs Q1 monosemanticity (left) and Q1 performance gap (right). Negative gaps indicate Q1 underperformance relative to Q2–Q4 average.

Figure 18: SAEBench Gemma: Architecture comparison showing overall vs Q1 monosemanticity (left) and Q1 performance gap (right). Negative gaps indicate Q1 underperformance relative to Q2–Q4 average.

Family	Arch	Width	Overall	Sparsity	Dominance (%)
Saebench Pythia	PAnneal	16k	78.8%	27.6	0.30%
Saebench Pythia	PAnneal	4k	78.8%	22.9	0.34%
Saebench Pythia	PAnneal	65k	76.5%	38.2	0.19%
JBloom OAI v5	Standard ReLU	128k	75.5%	893.9	0.03%
Saebench Pythia	PAnneal	16k	74.8%	74.1	0.11%
JBloom OAI v5	Standard ReLU	128k	74.8%	1492.3	0.03%

Appendix

Table 12: Top 10 SAE configurations by overall perfect ranking accuracy. Sparsity is the mean number of active features per input (lower = more sparse). Dominance (%) is

100\times

the median dominance score among the SAE’s 10 most dominant features (Top-10 median).

Family	Arch	Width	Gap	Q1	Sparsity	Dominance (%)
Saebench Pythia	BatchTopK	16k	-19.6%	50.5%	160.0	0.11%
Saebench Pythia	TopK	65k	-19.2%	54.0%	40.0	0.19%
Tinystories	TopK	16k	-18.4%	55.5%	97.4	0.05%
Tinystories	Matryoshka	65k	-17.2%	56.8%	0.1	1.14%
Saebench Pythia	Standard ReLU	16k	-16.4%	54.2%	455.0	0.06%
Saebench Pythia	TopK	16k	-16.2%	46.2%	639.9	0.21%

Appendix

Table 13: Top 10 SAE configurations with the largest Q1–Q2–4 gap (more negative = Q1 performs worse relative to Q2–4). Gap is Q1 accuracy minus the average accuracy over Q2–4, in percentage points. Dominance (%) is

100\times

the median dominance score among the SAE’s 10 most dominant features (Top-10 median).

Family	Arch	Width	Q1	Overall	Sparsity	Dominance (%)
JBloom OAI v5	Standard ReLU	32k	78.2%	72.0%	199.9	0.03%
Saebench Pythia	PAnneal	16k	76.8%	74.8%	74.1	0.11%
Saebench Pythia	PAnneal	4k	75.8%	78.8%	22.9	0.34%
Saebench Pythia	PAnneal	65k	75.7%	76.5%	38.2	0.19%
JBloom OAI v5	Standard ReLU	32k	75.3%	70.7%	224.4	0.05%
Tinystories	Standard ReLU	65k	75.2%	69.6%	673.5	0.01%

Appendix

Table 14: Top 10 SAE configurations by Q1 perfect ranking accuracy. Dominance (%) is

100\times

the median dominance score among the SAE’s 10 most dominant features (Top-10 median).

Figure 19: Relationship between dominance, interpretability, and sparsity. The inverse dominance-interpretability relationship holds across different sparsity levels.

Figure 20: Interpretability patterns for top 10% most dominant features. Even among the highest-dominance features, the inverse relationship with interpretability persists.

Architecture	Q1	Q2	Q3	Q4
Standard ReLU	3.81	2.58	2.59	2.81
TopK	1.79	0.29	0.12	0.15
Gated	1.62	0.88	0.69	0.61
JumpReLU	1.10	0.17	0.18	0.44
BatchTopK	0.94	0.10	0.07	0.05
PAnneal	0.38	0.33	0.23	0.09

Appendix

Table 15: Percentage of features with at least one redundant pair by architecture and quantile (decoder cosine similarity

\geq 0.7

).

Architecture	Q1 (%)	ANY (%)	Pairs Within
BatchTopK	40.33	65.67	2.6
TopK	35.44	68.78	5.0
JumpReLU	28.61	81.94	2.5
Standard ReLU	26.11	93.33	4.7
Matryoshka	25.00	33.61	2.1
PAnneal	21.94	63.61	3.1

Appendix

Table 16: Top 10 features involvement in redundancy by architecture. Q1 (%): percentage of top 10 features involved in Q1 redundant pairs. ANY (%): percentage involved in any redundant pair. Pairs Within: average redundant pairs formed within top 10.

Architecture	Q1 (%)	Q2 (%)	Q3 (%)	Q4 (%)
Standard ReLU	63.74	8.33	10.50	17.43
PAnneal	86.35	7.71	4.23	1.72
Gated	59.02	16.37	10.90	13.71
TopK	77.22	10.29	3.11	9.38
BatchTopK	66.42	18.56	8.21	6.80
Matryoshka	83.65	14.18	1.82	0.35

Appendix

Table 17: Redundancy results across SAE architectures (per-SAE averaged, Q4 dead-feature pairs filtered). Left: per-architecture share of redundant pairs by dominance quantile, at decoder cosine similarity

\geq 0.7

. Right: per-SAE Wilcoxon signed-rank test of Q1 vs. Q2–Q4 redundant-pair counts; positive

d

indicates Q1 has more redundant pairs.

Architecture	Q1 (%)	Q2 (%)	Q3 (%)	Q4 (%)	Pair filter rate (%)
Standard ReLU	38.48	6.92	16.04	38.56	38.85
PAnneal	53.98	15.88	17.91	12.23	35.89
Gated	58.99	16.35	10.90	13.77	0.08
TopK	61.43	11.40	8.01	19.16	21.54
BatchTopK	41.62	18.71	11.43	28.25	36.94
Matryoshka	49.31	26.29	7.62	16.78	31.11

Appendix

Table 18: Unfiltered version of Table 17 (left): per-architecture share of redundant pairs by dominance quantile, per-SAE averaged, with pairs involving a dead feature not excluded. Last column is the per-SAE-averaged percentage of pairs that would be removed by dead-feature filtering. Without filtering, dead-feature pairs inflate Q4 and mask the Q1 concentration for BatchTopK / Matryoshka / JumpReLU; after filtering (Table 17 ), all 7 architectures are Q1-dominant.

Figure 21: Q1 redundancy concentration across decoder cosine thresholds

\theta\in[0.1,0.9]

, with and without dead-feature pair filtering. Filtered: 72–83% of SAEs Q1-dominant at every threshold. Unfiltered: 45–67%.

Architecture	Q1 (%)	ANY (%)	Pairs Within
BatchTopK	13.87	18.50	10.7
TopK	14.52	20.93	17.7
JumpReLU	14.53	29.22	19.3
Standard ReLU	20.53	71.36	296.3
Matryoshka	2.72	3.58	2.3
PAnneal	5.33	16.89	10.6

Appendix

Table 19: Top 100 features involvement in redundancy by architecture. Q1 (%): percentage of top 100 features involved in Q1 redundant pairs. ANY (%): percentage involved in any redundant pair. Pairs Within: average redundant pairs formed within top 100.

Quantile	Mean	Median
Q1	0.0561	0.0239
Q2	0.1090	0.0313
Q3	0.1249	0.0385
Q4	0.1347	0.0527

Appendix

Table 20: Encoder similarity (all pairs) by quantile across all SAEs. Q1 has the lowest encoder similarity, indicating more encoder space partitioning.

Figure 22: Q1 features involved in redundancy (percentage) by family and architecture. Standard ReLU shows the highest Q1 redundancy, particularly in TinyStories (up to 28.6%) and GPT2-Small (up to 10.3%), while JumpReLU shows minimal redundancy across all families. Smaller models show more severe redundancy than larger models.

Figure 23: Q1 redundancy gap (Q1 features involved % minus Q2–Q4 average %) by family and architecture. Positive values indicate Q1 has more redundancy than Q2–Q4. Standard ReLU shows the largest positive gaps, indicating stronger Q1 redundancy concentration.

Figure 24: Total redundant pairs (sum across all quantiles) by family and architecture. Standard ReLU shows the highest total redundancy, particularly in TinyStories (over 1.3 million pairs at 65k) and GPT2-Small. JumpReLU shows high total redundancy but with different quantile distribution patterns.

Figure 25: Average encoder pairwise similarity (across all quantiles) by family and architecture. Lower values indicate more encoder space partitioning.

Figure 26: Q1 encoder pairwise similarity by family and architecture. Q1 consistently shows lower encoder similarity than Q2–Q4, indicating greater encoder space partitioning for dominant features.

Family	Arch	Width	Total Redundant Pairs	Q1 Redundant Pairs
Tinystories	Standard ReLU	65k	1340724	1328227
GPT2-Small	JumpReLU	16k	1020088	5573
Tinystories	JumpReLU	16k	665199	7438
Tinystories	Standard ReLU	16k	561217	545472
GPT2-Small	JumpReLU	4k	430005	18665
GPT2-Small	JumpReLU	65k	346169	5714

Appendix

Table 21: Top 10 SAE configurations by total redundant pairs (sum across all quantiles). TinyStories Standard ReLU at 65k shows the highest total redundancy (1.3 million pairs), with 99.1% concentrated in Q1. GPT2-Small JumpReLU also appears but with different quantile distribution.

Family	Arch	Width	Q1 Redundant Pairs	Q1 Features Involved (%)
Tinystories	Standard ReLU	65k	1328227	28.63
Tinystories	Standard ReLU	16k	545472	16.72
GPT2-Small	Standard ReLU	65k	188940	10.28
Tinystories	Gated	65k	125887	8.98
Tinystories	Standard ReLU	4k	101376	21.19
GPT2-Small	Standard ReLU	16k	54739	5.03

Appendix

Table 22: Top 10 SAE configurations by Q1 redundant pairs. TinyStories Standard ReLU dominates, with up to 1.3 million Q1 redundant pairs (28.6% of Q1 features) at 65k width. GPT2-Small Standard ReLU also shows high Q1 redundancy (up to 188,940 pairs, 10.3% of Q1 features).

Family	Arch	Width	Q1 Features Involved (%)	Q1 Redundant Pairs
Tinystories	Standard ReLU	65k	28.63	1328227
Tinystories	Standard ReLU	4k	21.19	101376
Tinystories	Standard ReLU	16k	16.72	545472
GPT2-Small	Standard ReLU	65k	10.28	188940
Tinystories	Gated	65k	8.98	125887
GPT2-Small	JumpReLU	4k	6.32	18665

Appendix

Table 23: Top 10 SAE configurations by Q1 features involved percentage. TinyStories Standard ReLU shows the highest percentages (28.6% at 65k, 21.2% at 4k, 16.7% at 16k), followed by GPT2-Small Standard ReLU (up to 10.3%). Smaller models show dramatically higher Q1 redundancy than larger models.

Family	Arch	Width	Q1 Redundancy Rate	Q1 Redundant Pairs
Tinystories	Standard ReLU	65k	147.581	1328227
Tinystories	Standard ReLU	16k	60.608	545472
Tinystories	Standard ReLU	4k	33.000	101376
GPT2-Small	Standard ReLU	65k	20.993	188940
Tinystories	Gated	65k	13.987	125887
GPT2-Small	Standard ReLU	16k	6.082	54739

Appendix

Table 24: Top 10 SAE configurations by Q1 redundancy rate (redundant pairs per Q1 feature). TinyStories Standard ReLU shows the highest rates (up to 147.6 pairs per feature at 65k), far exceeding larger models. GPT2-Small Standard ReLU also shows high rates (up to 21.0 per feature).

Family	Arch	Width	Q1 Gap (pairs)	Q1 Redundant Pairs	Total Redundant Pairs
Tinystories	Standard ReLU	65k	1324061.3	1328227	1340724
Tinystories	Standard ReLU	16k	540223.7	545472	561217
GPT2-Small	Standard ReLU	65k	186668.7	188940	195754
Tinystories	Gated	65k	125653.3	125887	126588
Tinystories	Standard ReLU	4k	100107.0	101376	105183
GPT2-Small	Standard ReLU	16k	51776.0	54739	63628

Appendix

Table 25: Top 10 SAE configurations by Q1 redundancy gap (Q1 redundant pairs minus Q2–Q4 average). TinyStories Standard ReLU shows the largest gaps (up to 1.3 million pairs at 65k), indicating extreme Q1 redundancy concentration. GPT2-Small Standard ReLU also shows large gaps (up to 186,669 pairs).

Figure 27: Distribution of maximum firing frequencies across all SAEs (all families). The median maximum firing frequency across 600+ SAEs is shown, with 75th percentile indicating that many SAEs have features that activate on a substantial fraction of inputs despite nominal sparsity constraints.

Figure 28: Firing frequency distributions by architecture for SAEBench Pythia (top) and Gemma (bottom) SAEs, shown as examples of patterns observed across all families. Violin plots show the distribution of maximum firing frequencies across all SAE configurations within each architecture, with the red line indicating the median. ReLU, PAnneal, Gated, and JumpReLU architectures consistently exhibit higher median maximum firing frequencies with lower variability compared to TopK and BatchTopK across all model families, which show lower medians but wider distributions.

Figure 29: Extreme case: Firing frequency distribution for a JumpRelu Pythia SAE where 10.18% of features fire more than 90% of the time (threshold

\epsilon=0.01

). This represents the most extreme case observed across all evaluated SAEs, demonstrating that high-frequency features can dominate a substantial fraction of the feature dictionary in pathological cases.

Figure 30: Distribution of the percentage of features that fire more than 90% of the time across all SAEs (shown here for SAEBench Pythia and Gemma families as examples). While median values are low (0.02–0.04%) across all families, maximum values reveal extreme cases where up to 10.18% of features violate sparsity constraints by firing on nearly all inputs.

Figure 31: Distribution of feature fraction needed to account for 50%, 75%, and 90% of total activation magnitude across all SAEs. The median SAE requires 3.3% of features to account for 50% of total magnitude, 11.8% to account for 75%, and 26.6% to account for 90%, demonstrating extreme concentration where the vast majority of features contribute negligibly.

Figure 32: Magnitude concentration by architecture for SAEBench Pythia (top) and Gemma (bottom) SAEs, shown as examples of patterns observed across all families. Violin plots show the distribution of the fraction of features needed to account for 90% of total magnitude across all SAE configurations within each architecture, with the red line indicating the median. Lower values indicate higher concentration (fewer features account for most magnitude).

Figure 33: Extreme case: ReLU SAE from the CanRager family (layer 19, trainer 5, l1 penalty 0.07) cumulative magnitude distribution. Only

\sim

2.5% of features account for nearly 100% of cumulative activation magnitude—an extreme example where the vast majority of the feature dictionary contributes negligibly to actual computation.

Figure 34: Distribution of top-30 feature dominance across all evaluated SAEs.

Figure 35: Dominance distributions by architecture for SAEBench Pythia (top) and Gemma (bottom) SAEs, shown as examples of patterns observed across all families. Violin plots show the distribution of top 10 dominance across all SAE configurations within each architecture, with the red line indicating the median. Matryoshka, BatchTopK, and PAnneal architectures exhibit higher median dominance compared to TopK and ReLU across all families.

Figure 36: Extreme case: Mean dominance by the most frequent feature. For some SAEs, the single most frequently firing feature contributes up to 60% of total activation magnitude on average across all inputs, demonstrating persistent structural dominance where individual features encode multiple concepts.

Figure 37: Dominance vs. effective sparsity (90%) for all architectures in SAEBench Pythia (top) and Gemma (bottom), shown as examples of patterns observed across all families. The x-axis shows effective active features (90%), where lower values indicate higher sparsity. All architectures are shown together, revealing that ReLU and PAnneal exhibit strong positive correlations (higher sparsity

\to

higher dominance) across all families, while TopK and BatchTopK show weaker or mixed patterns. This counterintuitive relationship indicates that sparsity constraints alone are insufficient to promote distributed feature usage.

Arch	Width	N	r
BatchTopK	16k	6	0.877
BatchTopK	4k	6	0.863
BatchTopK	65k	6	0.867
Gated	16k	6	0.934
Gated	4k	6	0.629
Gated	65k	6	0.694

Appendix

Table 26: Correlation between sparsity level and top 10 dominance for Saebench Pythia SAEs, controlling for width and architecture. For each architecture and dictionary size (width), we compute Pearson correlation between sparsity level (higher values = more sparse, fewer active features) and dominance across different sparsity parameter settings. A positive correlation indicates that as sparsity increases, dominance increases—more sparse SAEs concentrate computation in fewer dominant features. Correlations with

|r|\geq 0.5

are shown in bold .

Arch	Width	N	r
BatchTopK	4k	6	-0.174
BatchTopK	65k	6	-0.021
Gated	16k	6	-0.375
Gated	4k	6	0.038
Gated	65k	6	0.954
JumpReLU	16k	6	-0.462

Appendix

Table 27: Correlation between sparsity level and top 10 dominance for Saebench Gemma SAEs, controlling for width and architecture. For each architecture and dictionary size (width), we compute Pearson correlation between sparsity level (higher values = more sparse, fewer active features) and dominance across different sparsity parameter settings. A positive correlation indicates that as sparsity increases, dominance increases—more sparse SAEs concentrate computation in fewer dominant features. Correlations with

|r|\geq 0.5

are shown in bold .

Arch	Width	N	r
BatchTopK	16k	3	0.860
BatchTopK	4k	3	0.853
BatchTopK	65k	3	0.834
Gated	16k	3	0.831
Gated	4k	3	0.878
Gated	65k	3	0.792

Appendix

Table 28: Correlation between sparsity level and top 10 dominance for GPT2-Small SAEs, controlling for width and architecture. For each architecture and dictionary size (width), we compute Pearson correlation between sparsity level (higher values = more sparse, fewer active features) and dominance across different sparsity parameter settings. A positive correlation indicates that as sparsity increases, dominance increases—more sparse SAEs concentrate computation in fewer dominant features. Correlations with

|r|\geq 0.5

are shown in bold .

Arch	Width	N	r
Gated	16k	3	0.958
Gated	4k	3	0.956
Gated	65k	3	0.965
JumpReLU	16k	3	0.982
JumpReLU	4k	3	0.922
JumpReLU	65k	3	0.996

Appendix

Table 29: Correlation between sparsity level and top 10 dominance for OPT125M SAEs, controlling for width and architecture. For each architecture and dictionary size (width), we compute Pearson correlation between sparsity level (higher values = more sparse, fewer active features) and dominance across different sparsity parameter settings. A positive correlation indicates that as sparsity increases, dominance increases—more sparse SAEs concentrate computation in fewer dominant features. Correlations with

|r|\geq 0.5

are shown in bold .

Arch	Width	N	r
BatchTopK	16k	3	0.858
BatchTopK	4k	3	0.844
BatchTopK	65k	3	0.876
Gated	16k	3	0.714
Gated	4k	3	0.802
Gated	65k	3	0.714

Appendix

Table 30: Correlation between sparsity level and top 10 dominance for Tinystories SAEs, controlling for width and architecture. For each architecture and dictionary size (width), we compute Pearson correlation between sparsity level (higher values = more sparse, fewer active features) and dominance across different sparsity parameter settings. A positive correlation indicates that as sparsity increases, dominance increases—more sparse SAEs concentrate computation in fewer dominant features. Correlations with

|r|\geq 0.5

are shown in bold .

Arch	Width	N	r
TopK	16k	18	-0.140
TopK	4k	18	-0.043
TopK	65k	18	-0.132

Appendix

Table 31: Correlation between sparsity level and top 10 dominance for Canrager SAEs, controlling for width and architecture. For each architecture and dictionary size (width), we compute Pearson correlation between sparsity level (higher values = more sparse, fewer active features) and dominance across different sparsity parameter settings. A positive correlation indicates that as sparsity increases, dominance increases—more sparse SAEs concentrate computation in fewer dominant features. Correlations with

|r|\geq 0.5

are shown in bold .

Arch	Width	N	r
Standard ReLU	128k	48	0.674
Standard ReLU	32k	48	0.732

Appendix

Table 32: Correlation between sparsity level and top 10 dominance for Jbloom SAEs, controlling for width and architecture. For each architecture and dictionary size (width), we compute Pearson correlation between sparsity level (higher values = more sparse, fewer active features) and dominance across different sparsity parameter settings. A positive correlation indicates that as sparsity increases, dominance increases—more sparse SAEs concentrate computation in fewer dominant features. Correlations with

|r|\geq 0.5

are shown in bold .

Architecture	Median Max Firing Freq.
ReLU	0.990
TopK	0.597
BatchTopK	0.749
PAnneal	0.994
JumpReLU	0.998
Gated	0.999

Appendix

Table 33: Median maximum firing frequency by architecture across all SAE families.

Architecture	Median Fraction (90%)
ReLU	0.177
TopK	0.228
BatchTopK	0.189
PAnneal	0.235
JumpReLU	0.297
Gated	0.575

Appendix

Table 34: Median fraction of features needed to account for 90% of total magnitude by architecture across all SAE families. Lower values indicate higher concentration.

Architecture	Median Top 10 Dominance
ReLU	0.102
TopK	0.120
BatchTopK	0.151
PAnneal	0.155
JumpReLU	0.045
Gated	0.011

Appendix

Table 35: Median top 10 globally frequent features dominance by architecture across all SAE families.

Category	Word Count	Specificity	Unique Ratio	DAME Score
Q2-Q3 (Baseline)	22.6	0.738	0.988	0.682
Q1 High DAME	22.3	0.737	0.992	0.828
Q1 Low DAME	23.9	0.747	0.969	0.425
Q4 High DAME	22.1	0.747	0.993	0.862
Q4 Low DAME	23.3	0.717	0.986	0.558

Appendix

Table 36: Explanation Metrics by Category (Q2-Q3 as Baseline)

Figure 38: Top distinctive words in explanations by category. Q1 high DAME explanations emphasize general linguistic structures (conjunctions, articles), while Q1 low DAME explanations focus on technical patterns (symbols, XML-related terms).

Figure 39: Explanation complexity metrics by category. Metrics are similar across categories, but distributions reveal subtle differences in consistency.

Figure 40: Domain patterns by quantile. Q1 features show high proportion of mixed domains, combining multiple pattern types, which may contribute to interpretability challenges.

Figure 41: Surface coverage for 95% of data across all evaluated models. The plot shows median and maximum surface coverage across layers for each model. For comparison, a uniform distribution on a sphere would require 95% surface coverage to contain 95% of the data. Actual activation data requires much smaller surface coverage (typically 1–33%), demonstrating that activations cluster near a dominant mean direction rather than being uniformly distributed. This clustering behavior forces encoder weights to orient nearly orthogonally to the mean direction to achieve sparsity.

Figure 42: Distribution of surface coverage values across layers for each model. Each point represents one layer’s surface coverage for 95% of data. Box plots show quartiles, with median values marked. The consistent low surface coverage across all layers and models demonstrates that activation clustering is a universal property of transformer representations, not specific to particular layers or architectures. This geometric structure is what necessitates the orthogonal orientation of encoder weights.

Figure 43: Median angular distances between encoder–decoder, encoder–input, and decoder–input vectors across all SAE families. Encoder–decoder distances range from 24–50 ∘ (median), showing systematic mismatch between encoder and decoder directions. Both encoder–input and decoder–input distances are large (82–88 ∘ ), demonstrating that both encoder and decoder vectors orient nearly orthogonally to typical input activations. The encoder–input distances are typically slightly larger than decoder–input distances, consistent with encoders needing to be more orthogonal to satisfy sparsity constraints.

Figure 44: Distribution of angular distances across SAE families, showing encoder–decoder, encoder–input, and decoder–input distances separately. Error bars show standard deviation across SAEs within each family. The consistent pattern across families demonstrates that these geometric relationships are universal properties of SAE training, not artifacts of specific architectures or training procedures. Encoder–input distances are consistently larger than decoder–input distances, reflecting the stronger orthogonality requirement for encoders.

Figure 45: Scatter plot comparing encoder–input vs decoder–input angular distances for individual SAEs. Each point represents one SAE. Points above the diagonal (dashed line) indicate SAEs where encoder–input distance exceeds decoder–input distance. The majority of points lie above the diagonal, demonstrating that encoders are systematically more orthogonal to input activations than decoders. This pattern holds across all SAE families, with canrager showing the largest differences (5–6 ∘ ) and jbloom_oai showing smaller but still consistent differences.

Figure 46: Median angular distances of encoder and decoder weights from the activation mean direction across all SAE families. Encoder weights are nearly orthogonal to the mean direction (median angles of 88–93 ∘ ), demonstrating that they must orient orthogonally to satisfy sparsity constraints. Decoder weights also diverge substantially (median angles of 90–94 ∘ ), but this divergence serves a different purpose: encoding distinct semantic directions that are not aligned with the mean activation direction. This geometric separation between encoder/decoder weights and the activation mean is natural and necessary for SAEs to function effectively.

Figure 47: Distribution of encoder and decoder divergence from activation mean across SAE families. Error bars show standard deviation across SAEs within each family. Both encoder and decoder weights show consistent near-orthogonality to the mean direction across all families, with encoders typically slightly more orthogonal than decoders. This pattern demonstrates that the geometric constraints identified in the activation clustering and angular relationship analyses manifest consistently in the learned weight distributions.

Figure 48: Scatter plot comparing encoder vs decoder divergence from activation mean for individual SAEs. Each point represents one SAE. Points above the diagonal (dashed line) indicate SAEs where encoder divergence exceeds decoder divergence. The majority of points cluster near the diagonal but slightly above it, showing that encoders are consistently slightly more orthogonal to the mean than decoders. This systematic difference reflects the different roles: encoders must satisfy sparsity constraints (requiring stronger orthogonality), while decoders encode semantic content (allowing slightly less orthogonality).

Figure 49: Controlled comparison of per-feature scoring rules on 80 synthetic features with generator-fixed ground truth. Bars and the “Strict” column give the number of features whose correct explanation is scored strictly above the reversed one; the tie-credit rate counts a tie as one half.

Explainer	Judge	Mean Q1 $-$ Q2–Q4 (pp)	SAEs with a Q1 deficit
Original Mistral-7B	Mistral-7B	$-11.26$	6/6
GPT-5.6 Sol	Mistral-7B	$-3.25$	6/6
Original Mistral-7B	GPT-5.6 Sol	$-4.04$	5/6
GPT-5.6 Sol	GPT-5.6 Sol	$-18.84$	6/6

Appendix

Table 37: Explainer-by-judge check on six SAEs.

Explanation	Q1 $-$ Q2–Q4	SAEs with	Q1 mean	Q2–Q4 mean	Q1 $-$ Q2–Q4 length
instruction	accuracy (pp)	lower Q1	words	words	(paired $p$ )
15 words / 40 tokens	$-4.53$	21/28	23.63	22.82	$+0.81$ (.00049)
30 words / 80 tokens	$-4.05$	20/28	30.07	29.00	$+1.07$ (.00064)
60 words / 160 tokens	$-4.78$	23/28	35.80	35.33	$+0.47$ (.443)

Appendix

Table 38: Explanation length budget on a fixed panel of 28 SAEs (2,240 features).

Group	Highest single-variable correlation	AND	OR
Q1	.447	.143	.616
Q2–Q4	.273	.067	.363
Q1 $-$ Q2–Q4	$+.174$	$+.075$	$+.253$

Appendix

Table 39: Held-out correlation with the selected single-variable, AND, and OR targets on 105 synthetic SAEs.

SAE width	Q1	Q2–Q4	Q1 $-$ Q2–Q4
128	.094	.104	$-.010$
192	.088	.105	$-.018$
256	.087	.106	$-.019$
512	.084	.108	$-.023$
768	.082	.106	$-.024$
1024	.085	.104	$-.019$

Appendix

Table 40: Prevalence-corrected held-out support containment by SAE width (35 SAEs per width).

Comparison	Matched Q1–control	Firing-frequency standardized	Q1 $-$ control accuracy
	pairs retained	mean difference	(pp, 95% CI)
Unmatched Q1 versus Q2–Q4	—	0.715	$-1.59$ $(-2.30,-0.89)$
Matched, 0.05 rank caliper	55.2%	0.034	$-2.45$ $(-3.67,-1.24)$
Matched, 0.10 rank caliper	69.6%	0.080	$-2.07$ $(-3.07,-1.07)$
Matched, 0.20 rank caliper	86.2%	0.165	$-2.00$ $(-2.85,-1.16)$

Appendix

Table 41: Frequency-matched comparison of Q1 features with Q2–Q4 control features.

Held-out mean $L_{0}$	Sharding-compatible pair rate	SAEs with at least one such pair
$<50$	2.6%	56.3%
50–200	3.8%	42.9%
$\geq 200$	3.3%	30.4%

Appendix

Table 42: Sharding-compatible pairs by held-out mean

L_{0}

.

The committee’s primary	His ultimate
The organization has one clear	The campaign’s central
The proposal’s central	The department’s stated
The team’s shared †	The paper’s broader
The research program has a single	The initiative’s guiding
Her stated	Our main
The policy’s underlying †	The reform’s practical

Appendix

Table 43: Complete fixed target-prompt family.

\dagger

marks prompts excluded because feature 46571 was naturally active at the intervention position.

Base model	Public SAE family	SAEs with a Q1 deficit
Qwen3-8B	BatchTopK	41.7%
Qwen3-14B	BatchTopK	50.0%
Qwen3-32B	BatchTopK	41.7%
Gemma-2-9B	Gemma Scope residual	75.0%

Appendix

Table 44: Q1 token perfect-ranking deficits in public larger-model SAE panels. Percentages are computed within each model’s evaluated SAE panel.

Support pattern	Full high-cosine Mean [95% CI]	Matched high-cosine	Low-cosine controls
Partition-like	21.5% [17.6, 25.6]	21.6%	80.0%
Nested-like	15.3% [11.5, 19.3]	16.1%	1.2%
Duplicate-like	29.9% [25.1, 34.9]	27.4%	2.1%
Partial overlap	33.3% [28.8, 37.8]	34.9%	16.7%

Appendix

Table 45: Support geometry: mean within-SAE pair percentages. Intervals describe the full eligible high-cosine pool; the last two columns use matched pairs.

Audit panel	Stable sibling reading	Also fresh support	Insufficient evidence
All screened targets	15.8%	13.3%	51.4%
Control-matched targets	13.3%	13.3%	60.0%
Low-cosine controls	3.3%	3.3%	56.7%

Appendix

Table 46: Two-pass context judgments. “Fresh support” additionally requires the partition criterion to replicate in the new context corpus. The insufficient column means that at least one pass could not assess both features.

Feature 2081: they	Feature 3844: he / she
… the NBA, they like them young, …	… declared that he will be a better leader for the sake of his friends …
… after having suffered heavy casualties, they returned to Tagaung.	Radcliffe has supported various charities. He designed …
… have stated numerous times that they do not condone …	… Madonna starts fighting with the men. She starts bending backwards …

Appendix

Table 47: Selected activation contexts for the Pythia-160M BatchTopK-4k pair (decoder cosine 0.759; fresh-support Jaccard 0). Bold marks the activating token. Contexts are independently selected for each feature; ellipses mark cropped text.

Symbol / term	Meaning
$x$	An input token position.
$\phi(x)\in\mathbb{R}^{d_{\text{in}}}$	Model activation at the hooked layer; $d_{\text{in}}$ is the activation dimension.
$m$	SAE width: the number of features in the dictionary.
$f_{i}(x)$ , $d_{i}$ , $w_{i}$	Activation, decoder direction, and encoder row of feature $i$ .
$\mathcal{C}$ , $U$	Target concept set fixed for an analysis at a stated explanatory granularity; set of interpretable units to which attribution is assigned.
Atomic explanatory unit	A basic coordinate to which attribution is assigned directly, without further decomposition within the chosen explanatory interface.

Appendix

Table 96

Rethinking SAE Evaluation: An Atomic Interpretable Unit Framework Reveals Hidden Polysemanticity and Redundancy

Abstract

Figures & tables

Appendix

Sparse Autoencoders

The Rate-Distortion-Polysemanticity Tradeoff in SAEs

Measuring Monosemanticity in Sparse Autoencoders via Latent Activation Coherence

Sparse Autoencoders for Mechanistic Interpretability in NLP: A Critical Survey of Recoverability, Interpretability, and Utility

Architecture	Q1	Q2	Q3	Q4	Diff
BatchTopK	61.5	65.2	67.1	69.9	-5.9
Gated	60.8	57.8	58.9	60.6	- 1.7
JumpReLU	58.7	58.0	56.9	58.0	- 1.1
Matryoshka	61.6	64.5	65.8	66.5	-4.0
PAnneal	67.1	67.4	71.0	72.3	-3.2
Standard ReLU	60.5	60.8	61.9	62.1	-1.1

Abstract

Sparse Autoencoders

Explore similar work

The Rate-Distortion-Polysemanticity Tradeoff in SAEs

Measuring Monosemanticity in Sparse Autoencoders via Latent Activation Coherence

Sparse Autoencoders for Mechanistic Interpretability in NLP: A Critical Survey of Recoverability, Interpretability, and Utility