Representation Redistribution in Sparse Autoencoders

Ziye Mei; Haotian Ma

cs.LGOct 15, 2026

Representation Redistribution in Sparse Autoencoders

Authors: Ziye Mei, Haotian Ma

Organizations: Department of Computer Sciences, University of Wisconsin–Madison

Abstract

Sparse autoencoders (SAEs) decompose language model activations into sparse features, but prompt transformations designed to preserve the underlying request can redistribute activity across them. A feature change observed on one transformed prompt need not recur, and even a recurring change need not affect a downstream computation. We introduce the redistribution sensitivity score (RSS), which identifies large feature responses that recur in a consistent direction, and evaluate the resulting supports on variants excluded from selection. Across 27 pretrained SAEs spanning eight language models, the tested prompt transformations produce a reproducible redistribution profile on these excluded variants. We then separate recurring representation change from signed readout contribution and decision failure. For a fixed linear readout, an exact decomposition shows how response magnitude, readout coefficients, directional alignment, and the clean margin determine downstream consequences. In two frozen safety readouts, combining RSS support erosion with the full clean margin yields recurrent failure AUROCs of 0.935 and 0.958 using median supports of 20 and 19 features, compared with 0.965 and 0.964 using full erosion. RSS therefore provides a way to identify recurring sparse representation changes and quantify their contribution to a downstream readout without equating representation change with failure. Code is available at https://github.com/OpenSAE/Redistribution-Sensitivity-Score

Figures & tables

Figure 1: RSS under prompt transformations. In the illustrated five-variant selection, boxed features change in the same direction in at least four variants. MeanRSS averages their absolute response magnitudes.

Figure 2: Normalized MeanRSS across six prompt transformations and 27 pretrained SAEs. Each SAE’s six setting means are divided by their average, so the plot compares transformation profiles rather than absolute activation scales. Large outlined diamonds show averages across SAEs.

Figure 3: Response contrast on variants excluded from feature selection. Each point is the normalized context and wrapper mean minus the local edit mean for the indicated feature set and SAE layer. Clean fixed: features selected from clean prompts. Global fixed: features selected by activity across the clean corpus. Dictionary mean: equal average over all SAE features.

Figure 4: Feature validation on held out variants. A, response rate in the direction chosen on selection variants, comparing RSS with magnitude alone. B, mean

|e_{f}^{\mathrm{ho}}|

for RSS features and unselected features matched by readout coefficient magnitude. C–D, MCR as the support cap

K

varies. The

|e_{f}|

ranking uses contributions from the selection variants.

Figure 5: Readout change and recurrent readout failure on held out variants. A, Spearman correlation with held out margin erosion and AUROC for recurrent failure. Correlation measures erosion transfer, while AUROC measures failure ranking; residual margin correlations are oriented toward increasing risk. B, clean margin versus full residual margin for Gemma 3 4B, colored by held out recurrent failure. The dashed line marks zero average selection margin.

Figure 6: Utility of RSS supports selected for each prompt and transformation within the frozen readout. A,

R^{2}

for reconstructed held out logit changes. B, readout errors reversed by restoring selected feature values. C–D, errors and harmful false negatives captured at a 20% offline review budget using known labels. Error bars show 95% bootstrap confidence intervals.

Symbol	Meaning
$x_{0}$ , $x_{v}$	clean prompt and its $v$ th transformed variant
$\delta_{v,f}$	change in SAE feature $f$ from $x_{0}$ to $x_{v}$
$h_{f}$	mean absolute feature change used for RSS magnitude ranking
$S$	responsive support selected by RSS
MeanRSS	average $h_{f}$ over the responsive support
$\widetilde{\delta}_{v,f}$	standardized feature change used by the frozen probe

Table S1: Core notation. Quantities are defined for one pretrained SAE, layer, base prompt, and transformation setting unless stated otherwise.

Parameter	Value
Selector observations, $n$	5
Response rate threshold, $\rho$	0.8
Directional consistency threshold, $\kappa$	0.8
Response threshold, $\tau$	$10^{-5}$
Candidate features, $M$	400
Support cap, $K$	20

Table S2: Selection and uncertainty parameters. The main representation protocol has five operator observations derived from fifteen variants per setting. The readout audit selects from five variants and evaluates on ten excluded variants.

Model	Layers	SAE checkpoint	Width	Hook	Architecture / sparsity
GPT-2 Small	4, 6, 8, 10	GPT2-Small-SAEs-Reformatted	24,576	resid_pre	ReLU, $32\times$ ; L1 coef. $8\times 10^{-5}$
Pythia-70M	1, 3, 5	pythia-70m-deduped__res-sm	32,768	resid_post	ReLU, standard; sparsity n.r.
Gemma 3 270M	5, 9, 12, 15	gemma-scope-2-270m-pt	16,384	resid_post	JumpReLU; $L_{0}=18$ at layer 5 and 20 elsewhere
Gemma 3 1B	7, 13, 17, 22	gemma-scope-2-1b-pt	16,384	resid_post	JumpReLU; $L_{0}=18$ at layer 7 and 20 elsewhere
Gemma 3 4B	9, 17, 29	gemma-scope-2-4b-pt	65,536	resid_post	JumpReLU; nominal $L_{0}=53,60,60$
Mistral 7B	8, 16, 24	mistral_7b_layer_L	65,536	resid_pre	ReLU, $16\times$ ; measured $L_{0}=92,74,75$

Table S3: Pretrained SAE specifications (JumpReLU as in Rajamanoharan et al. [2024] ). “n.r.” means not reported.

Figure S1: Normalized MeanRSS profiles for the six transformations across all sampled layers of the eight models. Normalization follows Figure 2 .

Figure S2: Overlap between the feature supports selected under different transformations, averaged over all units and 1,000 prompts. A, pairwise Jaccard overlap between transformations; B, averages within each model. IC denotes irrelevant context, DC distractor context, MW metalinguistic wrapper, FW format wrapper, PN punctuation noise, and TL typed lexical substitution.

Setting	Available	Mean $\|S\|$	Median $\|S\|$	$\|S\|=20$
Irrelevant context	27,000	19.36	20	90.3%
Distractor context	27,000	18.77	20	82.1%
Metalinguistic wrapper	27,000	19.06	20	86.6%
Format wrapper	27,000	18.56	20	82.1%
Punctuation noise	27,000	18.01	20	71.1%
Typed lexical substitution	26,996	16.18	20	54.6%

Table S4: Support availability and size across transformation settings. Each setting contains 27,000 expected groups.

Treatment	Gap	95% CI	Order
Omitted	0.8740	[0.8671, 0.8810]	Reference
Assigned zero	0.8741	[0.8673, 0.8810]	Unchanged

Table S5: Sensitivity to the treatment of unavailable groups. Gap is the context and wrapper mean minus the local-edit mean; brackets give 95% confidence intervals.

Setting	Overlap	Newly active	Inactive	Positive	Negative
Irrelevant context	0.383	0.462	0.469	0.368	0.632
Distractor context	0.361	0.490	0.487	0.304	0.696
Metalinguistic wrapper	0.394	0.453	0.451	0.375	0.625
Format wrapper	0.381	0.496	0.478	0.230	0.770
Punctuation noise	0.692	0.198	0.187	0.406	0.594
Typed lexical substitution	0.776	0.133	0.133	0.381	0.619

Table S6: Changes in active SAE coordinates between clean and transformed prompts. Overlap is Jaccard similarity, newly active is normalized by the transformed active set, and inactive is normalized by the clean active set. Positive and negative shares partition the absolute activation change within the RSS support.

Figure S3: Normalized MeanRSS by prompt topic. A, topic level means for each transformation; B, the ratio of the context and wrapper mean to the local edit mean for each topic. Transformation abbreviations as in Figure S2 .

Figure S4: Profile of the six transformations after removing one model at a time, compared with the profile over all units. Transformation abbreviations as in Figure S2 .

Setting	Normalized MeanRSS
Irrelevant context	1.466 [1.459, 1.474]
Distractor context	1.341 [1.337, 1.346]
Metalinguistic wrapper	1.199 [1.192, 1.205]
Format wrapper	1.159 [1.154, 1.164]
Punctuation noise	0.525 [0.519, 0.532]
Typed lexical substitution	0.309 [0.305, 0.314]

Table S7: Aggregate normalized MeanRSS profile with equal weight for each SAE. Within each SAE, the six setting means are divided by their average. Brackets give 95% confidence intervals.

Criterion	Prompts	Share
Context/wrapper mean $>$ local-edit mean	1,000 / 1,000	100.0%
Every context/wrapper setting $>$ both local edits	997 / 1,000	99.7%

Table S8: Prompt-level consistency over the 1,000 representation-corpus prompts.

Figure S5: Comparison of the main feature selection rule with an alternative rule that treats all fifteen variants individually. A, mean support size; B, rates of undersized and empty selections; C, Jaccard overlap between the supports chosen by the two rules; D, unit level normalized MeanRSS under the two rules. Transformation abbreviations as in Figure S2 .

Setting	Spearman	Minimum	Gap	Mean $\|S\|$	Jaccard
Reference	1.00	1.00	0.626 [0.604, 0.646]	13.2	1.00
$\rho=0.6$	1.00	0.94	0.632 [0.610, 0.652]	13.8	0.90
$\kappa=0.6$	1.00	0.77	0.595 [0.571, 0.617]	17.4	0.61
$K=10$	1.00	0.89	0.702 [0.680, 0.723]	8.6	0.75
$K=50$	1.00	0.94	0.537 [0.514, 0.558]	17.5	0.89

Table S9: Selector parameter ablation on a stratified prompt pool. The reference uses

\rho=\kappa=0.8

and

K=20

; other rows change one parameter. Gap is the context and wrapper mean minus the local-edit mean, with 95% confidence intervals in brackets.

Criterion	Specifications
Full six-setting order preserved	17 / 25
Top setting preserved	20 / 25
Bottom setting preserved	25 / 25

Table S10: Transformation-ranking robustness across 25 combinations of magnitude summaries and normalization schemes.

$\tau$	Available	Profile Spearman	Gap	Positive units
0.001	99.99%	0.943	0.835	27/27
0.01	99.94%	0.943	0.817	27/27
0.05	99.65%	0.943	0.756	27/27
0.1	99.40%	0.943	0.706	27/27

Table S11: RSS after scaling feature changes using clean-prompt activations. Profile Spearman is relative to the main result; Gap is the context and wrapper mean minus the local-edit mean.

Setting and level	Definition	Tokens	Edits
Irrelevant context 1	4–8 added tokens	4.00	0
Irrelevant context 2	12–16 added tokens	12.00	0
Irrelevant context 3	24–32 added tokens	24.00	0
Irrelevant context 4	48–64 added tokens	48.00	0
Format wrapper 1	single label or quote	3.24	10.4
Format wrapper 2	envelope with two fields	8.92	28.2

Table S12: Intensity-level definitions and realized values. Tokens and edits are mean counts.

Sel.	Eval.	Reliability	Profile	Sign	CV	Jac.
3	12	0.773 [0.763, 0.782]	0.805	0.960	0.221	0.393
5	10	0.802 [0.792, 0.810]	0.827	0.964	0.189	0.539
8	7	0.714 [0.703, 0.724]	0.773	0.946	0.268	0.527
10	5	0.681 [0.670, 0.691]	0.765	0.946	0.279	0.654

Table S13: Split-half reliability by selection size. Brackets give 95% confidence intervals. Profile is stability against the 15-variant reference; Sign is transformation-group sign stability; Jac. is split-to-split support Jaccard.

Method	Sel.	Reliability	Profile	Sign
RSS support	3	0.773	0.805	0.959
RSS support	5	0.802	0.827	0.964
RSS support	8	0.714	0.773	0.946
RSS support	10	0.681	0.765	0.946
Clean support	3	0.772	0.809	0.921
Clean support	5	0.752	0.792	0.918

Table S14: Reliability by selection method and selection size. Column definitions follow Table S13 .

Fold	Avail.	Profile	Gap	Pos. units	Same dir.	Group sign	Pairwise
0	99.985%	1.000	0.909	27/27	0.940	99.92%	93.49%
1	99.985%	1.000	0.896	27/27	0.957	99.86%	92.00%
2	99.989%	1.000	0.897	27/27	0.959	99.87%	92.15%

Table S15: Fixed-support evaluation with one realization per operator used for selection. Profile is Spearman correlation with the 15-variant reference; Gap is the context and wrapper mean minus the local-edit mean; Pos. units is the number of SAEs with positive Gap; Group sign and Pairwise are prompt-level ordering rates.

Support	Absolute response	95% CI	Same direction	95% CI
RSS	107.670	[106.749, 108.597]	0.952	[0.951, 0.952]
Random, 100 draws	5.797	[5.748, 5.846]	0.758	[0.758, 0.759]

Table S16: Held out response for RSS and random supports matched in candidate pool and support size. Brackets give 95% confidence intervals.

Selection requirement	Available	Same direction	Gap
4 of 4 operators	99.42%	0.894	0.925
3 of 4 operators	100.00%	0.848	0.896

Table S17: Evaluation with one transformation operator excluded from support selection. Gap is the context and wrapper mean minus the local-edit mean.

Model	Unit	$\lambda$	Weights	Clean AUROC	Harmful FNR	Benign FPR
Gemma 3 4B	L17	0.01	415	0.941 [0.914, 0.963]	23.3%	7.3%
Llama 3.1 8B	L24	0.001	862	0.919 [0.884, 0.949]	33.3%	6.0%

Table S18: Safety probes and clean test performance. Brackets give 95% confidence intervals for AUROC; FNR and FPR are false-negative and false-positive rates.

Figure S6: Probe AUROC for the two safety readouts on untransformed test prompts and under each transformation. Transformation abbreviations as in Figure S2 . Error bars show 95% bootstrap confidence intervals.

Figure S7: Normalized MeanRSS and margin erosion on held out variants for each transformation, evaluated on the audited safety units. Transformation abbreviations as in Figure S2 . Error bars show 95% bootstrap confidence intervals.

Model	Setting	Norm. MeanRSS	Erosion	Flip	AUROC
Gemma 3 4B	Irrelevant context	1.118 [1.102,1.134]	0.132 [0.029,0.235]	6.1%	0.928
	Distractor context	1.263 [1.244,1.284]	0.138 [0.024,0.249]	5.6%	0.935
	Metalinguistic wrapper	1.113 [1.098,1.128]	0.231 [0.121,0.344]	6.6%	0.926
	Format wrapper	1.190 [1.172,1.208]	0.037 [-0.084,0.152]	8.0%	0.935
	Punctuation noise	0.524 [0.506,0.541]	0.158 [0.098,0.218]	4.2%	0.933
	Typed lexical substitution	0.792 [0.759,0.825]	0.146 [0.068,0.223]	4.6%	0.938

Table S19: MeanRSS and probe outcomes by transformation setting. Brackets give 95% confidence intervals for normalized MeanRSS and erosion. Erosion is mean held out margin loss; Flip is the fraction of predictions that differ from the clean prediction.

Figure S8: Effect of transformation strength on the audited safety units. A, margin erosion under three transformations rebuilt at four increasing intensity levels, with 95% bootstrap confidence intervals. B, MeanRSS for the same series, relative to the weakest level.

Model	Setting	Eligible	MeanRSS $\rho$	MeanRSS mono.	Erosion $\rho$	Erosion mono.
Gemma 3 4B	Irrelevant context	300	0.8	78/254	0.4	51/254
	Format wrapper	300	1.0	203/254	0.4	63/254
	Punctuation noise	265	1.0	137/222	0.4	37/222
Llama 3.1 8B	Irrelevant context	300	0.8	67/241	0.2	39/241
	Format wrapper	300	1.0	192/241	0.4	51/241
	Punctuation noise	265	1.0	138/212	0.6	62/212

Table S20: Prompt-level intensity calibration in the safety corpus. Monotonic counts use prompts whose mean response is nondecreasing across the four levels.

Model	Risk signal	Erosion $\rho$	Failure AUROC	AP
Gemma 3 4B	Clean margin	-0.282 [-0.370,-0.192]	0.863 [0.821,0.900]	0.294
	Full $\ell_{1}$	0.101 [0.025,0.173]	0.520 [0.444,0.595]	0.072
	Full cosine	0.083 [0.006,0.157]	0.486 [0.398,0.572]	0.063
	MeanRSS	0.116 [0.040,0.188]	0.526 [0.446,0.599]	0.069
	Probe exposure	0.104 [0.029,0.178]	0.597 [0.510,0.683]	0.107
	MeanRSS $\times$ exposure	0.130 [0.049,0.210]	0.606 [0.524,0.689]	0.091

Table S21: Risk signals on the common held out cohort. Brackets give 95% confidence intervals. Erosion

\rho

is Spearman correlation with held out margin erosion; AUROC and AP use recurrent failure. Residual-margin correlation is reported in the direction of increasing risk.

Model	RSS	Magnitude only
Gemma 3 4B	0.876 [0.872, 0.879]	0.816 [0.810, 0.822]
Llama 3.1 8B	0.871 [0.867, 0.874]	0.794 [0.788, 0.800]

Table S22: Held out response in the direction selected by RSS compared with magnitude-only selection. Brackets give 95% confidence intervals.

Model	RSS contribution	Matched contribution	Difference	Direction agreement
Gemma 3 4B	0.2397	0.0221	0.2177 [0.2005, 0.2356]	97.4%
Llama 3.1 8B	0.3702	0.0220	0.3483 [0.3235, 0.3739]	97.1%

Table S23: Held out feature contribution after matching selected and unselected features by readout coefficient magnitude. Contribution columns report mean absolute contribution. Direction agreement compares observed contribution sign with

\operatorname{sign}(-yw_{f}s_{f})

. Brackets give 95% confidence intervals for the paired difference.

Model	RSS	Mismatched RSS	$\|e_{f}\|$
Gemma 3 4B	0.430 [0.411, 0.450]	0.326 [0.308, 0.346]	0.942 [0.936, 0.948]
Llama 3.1 8B	0.562 [0.547, 0.577]	0.435 [0.417, 0.453]	0.919 [0.912, 0.925]

Table S24: Margin Contribution Retention at

K=20

. Mismatched RSS uses a support selected for another transformation;

|e_{f}|

ranks features by contributions computed from

V_{\mathrm{sel}}

. Brackets give 95% confidence intervals.

Model	Feature set	Erosion $\rho$	Failure AUROC
Gemma 3 4B	Full readout	0.923 [0.909, 0.935]	0.785 [0.721, 0.842]
Gemma 3 4B	RSS support	0.737 [0.700, 0.771]	0.753 [0.686, 0.815]
Llama 3.1 8B	Full readout	0.896 [0.878, 0.911]	0.705 [0.639, 0.766]
Llama 3.1 8B	RSS support	0.793 [0.761, 0.820]	0.735 [0.666, 0.797]

Table S25: Margin erosion results for the full readout and RSS support. Erosion

\rho

is correlation with held out erosion, and Failure AUROC ranks recurrent failure using the corresponding erosion score. Brackets give 95% confidence intervals.

Model	Feature set	Residual $\rho$	Residual AUROC
Gemma 3 4B	Full readout	0.169 [0.077, 0.261]	0.965 [0.939, 0.982]
Gemma 3 4B	RSS support	0.009 [-0.081, 0.105]	0.935 [0.901, 0.961]
Llama 3.1 8B	Full readout	0.152 [0.065, 0.243]	0.964 [0.940, 0.982]
Llama 3.1 8B	RSS support	0.080 [-0.006, 0.171]	0.958 [0.924, 0.981]

Table S26: Residual margin results for the same cohort. Both feature sets use the full clean margin. Correlation with held out erosion is oriented toward increasing risk; AUROC evaluates recurrent failure. Brackets give 95% confidence intervals.

Model	Feature set	Size	Erosion $\rho$	Residual AUROC	Errors	Harmful FN
Gemma 3 4B	Full readout	415	0.923	0.965	94.1%	90.9%
Gemma 3 4B	RSS support	20	0.737	0.935	89.2%	87.3%
Llama 3.1 8B	Full readout	862	0.896	0.964	91.9%	93.5%
Llama 3.1 8B	RSS support	19	0.793	0.958	92.8%	94.4%

Table S27: Summary of the full frozen readout and RSS supports selected for each case on the common evaluation cohort. Size is the full nonzero coefficient count or median support size. Both residual margin signals use the full clean margin. Errors and Harmful FN are capture rates at 20% offline review with known labels.

Layer	$\|S\|$	Same direction	MCR	Residual AUROC	$R^{2}$	Restoration
8	20	0.960	0.628	0.905	0.814	0.566
16	20	0.970	0.578	0.806	0.692	0.807

Table S28: Functional results for two additional Llama 3.1 8B layers under the main audit protocol.

Model	Support	Size	$R^{2}$	Errors reversed	$Q_{\mathrm{res}}$
Gemma 3 4B	RSS support	20	0.579 [0.532, 0.621]	0.722 [0.635, 0.803]	0.618
	Mismatched setting	20	0.438 [0.386, 0.481]	0.603 [0.512, 0.683]	0.368
	Global instability	20	0.017 [-0.008, 0.035]	0.067 [0.018, 0.151]	0.000
	Random matched	20	0.014 [-0.006, 0.022]	0.012 [0.002, 0.026]	0.002
Llama 3.1 8B	RSS support	19	0.627 [0.568, 0.675]	0.718 [0.610, 0.807]	0.800
	Mismatched setting	19	0.490 [0.406, 0.560]	0.611 [0.487, 0.721]	0.590

Table S29: Reconstruction and readout-counterfactual results. Baseline supports are matched to RSS support size; brackets give 95% confidence intervals.

Q_{\mathrm{res}}

is the share of probe logit change recovered from the support.

Ranking signal	Capture@10	Capture@20	Harmful FN@20	Residual error	$\Delta$ pAURC
Residual margin $m_{\mathrm{res}}$	85.1%	94.1%	90.9%	0.25%	+0.0276 [0.0175,0.0385]
Low clean margin	41.3%	68.9%	60.9%	1.30%	+0.0186 [0.0117,0.0267]
+ probe weights	32.1%	47.6%	49.5%	2.20%	+0.0109 [0.0046,0.0180]
MeanRSS	11.0%	18.8%	18.2%	3.40%	+0.0007 [-0.0026,0.0046]

Table S30: Selective-review summary for the Gemma probe. Capture columns give the fraction of errors reviewed at the stated budget. Residual error is reported at a 20% review budget. pAURC is the normalized partial area under the residual error curve from 0 to 40% review, and

\Delta

pAURC is the random ranking pAURC minus the listed value; brackets give 95% confidence intervals.

Model	Ranking signal	Errors	Harmful FN
Gemma 3 4B	Residual margin $m_{\mathrm{res}}$	94.1 [90.2, 96.5]	90.9 [81.9, 95.7]
	RSS residual margin $m_{\mathrm{res}}^{S}$	89.2 [81.8, 93.5]	87.3 [72.4, 94.9]
	Mismatched RSS residual	86.5 [78.7, 92.0]	85.9 [69.8, 93.5]
	Margin erosion $E$	63.6 [49.3, 74.8]	62.3 [35.4, 78.2]
	RSS margin erosion $E_{S}$	64.2 [50.1, 75.0]	66.8 [44.4, 81.1]
Llama 3.1 8B	Residual margin $m_{\mathrm{res}}$	91.9 [87.6, 95.6]	93.5 [88.7, 97.4]

Table S31: Selective review at a 20% budget for the rankings in Figure 6 C–D. Values are capture percentages; brackets give 95% confidence intervals.

Model	Method	Clean	Mean	Worst	Worst FNR	Gap
Gemma 3 4B	MeanRSS sampling	0.945	0.943	0.938	0.191	0.0067
Gemma 3 4B	Uniform	0.933	0.934	0.926	0.238	0.0071
Llama 3.1 8B	MeanRSS sampling	0.911	0.910	0.901	0.256	0.0100
Llama 3.1 8B	Uniform	0.915	0.913	0.907	0.263	0.0082

Table S32: Retrained-probe results for Figure S9 A. Clean, Mean, and Worst are AUROC; Gap is clean minus worst AUROC.

Figure S9: Augmentation guided by MeanRSS compared with uniform sampling. A, differences between the two sampling methods in clean, mean, and worst transformation AUROC, together with the reduction in harmful false negative rate under the worst transformation. Error bars show 95% bootstrap confidence intervals. B, harmful false negative rate averaged over the six transformations across five runs. Bars show the run mean and whiskers the observed range.

Sparse Autoencoders

Sparse autoencoders decompose neural network activations into sparse features to study, interpret, and control learned representations.

See all 218 papers

Sparse but Wrong: Incorrect L0 Leads to Incorrect Features in Sparse Autoencoders
Aug 22, 2025David Chanin, Adrià Garriga-AlonsoActivation Sparsity LLM Interpretability
University College London · MATS Research
C $^{2}$ R: Cross-sample Consistency Regularization Mitigates Feature Splitting and Absorption in Sparse Autoencoders
Jun 29, 2026Haoran Jin, Xiting Wang, Shijie Ren +2LLM Interpretability Representation Disentanglement
University of Science and Technology of China · State Key Laboratory of Cognitive Intelligence · Gaoling School of Artificial Intelligence, Renmin University of China
Active Budget Can Kill Sensitivity: Diagnosing and Repairing TopK Sparse Autoencoder Reliability
Sep 29, 2026Zhenting Huang, Bo Jiang, Junnan Liu +2Activation Sparsity LLM Interpretability
Beihang University · Monash University · Zhongguancun Laboratory

Aug 22, 2025J/K move · Enter open · S save

Sparse but Wrong: Incorrect L0 Leads to Incorrect Features in Sparse Autoencoders

David Chanin, Adrià Garriga-Alonso

University College London · MATS Research

Activation Sparsity LLM Interpretability Representation Disentanglement Sparse Autoencoders Sparse Coding Hyperparameter Optimization

Open paper AskPDF arXiv

Sparse Autoencoders (SAEs) extract features from LLM internal activations, meant to correspond to interpretable concepts. A core SAE training hyperparameter is L0: how many SAE features should fire per token on average. Existing work compares SAE algorithms using sparsity-reconstruction tradeoff plots, implying L0 is a free parameter with no inherently correct value aside from its effect on reconstruction. In this work we study the effect of L0 on SAEs, and show that if L0 is not set correctly, the SAE fails to disentangle the underlying features of the LLM. If L0 is too low, the SAE will mix correlated features to improve reconstruction. If L0 is too high, the SAE finds degenerate solutions that also mix features. Further, we present a proxy metric that can help guide the search for the correct L0 for an SAE on a given training distribution. We show that our method finds the correct L0 in toy models and coincides with peak sparse probing performance in LLM SAEs. We find that most commonly used SAEs have an L0 that is too low. Our work shows that practitioners must set L0 correctly to train SAEs with monosemantic features.

Abstract

Sparse Autoencoders

Explore similar work

Sparse but Wrong: Incorrect L0 Leads to Incorrect Features in Sparse Autoencoders

C2^{2}2R: Cross-sample Consistency Regularization Mitigates Feature Splitting and Absorption in Sparse Autoencoders

Active Budget Can Kill Sensitivity: Diagnosing and Repairing TopK Sparse Autoencoder Reliability

C $^{2}$ R: Cross-sample Consistency Regularization Mitigates Feature Splitting and Absorption in Sparse Autoencoders