Activation storage is often a bottleneck in sparse autoencoder (SAE) training. We study how many unique activations an SAE needs when its total training budget is fixed. Quality improves rapidly as the buffer grows, then reaches a plateau where replaying the buffer is nearly as effective as using fresh activations throughout training. Beyond this point, additional unique activations bring little benefit to reconstruction or interpretability, so the buffer need not grow with the training budget. We describe this tradeoff between diversity and repetition with a scaling law for limited data. We validate it across dictionary sizes and token budgets, early, middle, and late layers of Llama-3.1, Qwen3 model sizes, and BatchTopK, TopK, and JumpReLU SAEs. Sweeps using mathematical text and Python code test the effect of changing the activation source. Evaluation covers explained variance (EV), reconstruction MSE, feature absorption, automated interpretability, and sparse probing. These results give a practical training rule: choose the smallest buffer that reaches the quality plateau and replay it for the remaining budget. In our experiments, this reduces activation storage by 8–64× with negligible quality loss.
Figures & tables
Figure 1: Fresh activations and replay at a fixed SAE optimization budget. The replay pipeline generates and stores Ubuf activations once and uses E=Ntotal/Ubuf passes. s denotes bytes per activation. Generation counts refer to token activations. Batching and sequence packing determine the number of calls to the base model.
Figure 2: Explained variance as a function of unique activation count under fixed total SAE training budgets. With total optimization tokens fixed in each panel, moving right increases activation diversity and decreases the number of epochs over the buffer. EV improves sharply when activation diversity is low and then saturates.
Budget
Dict. size
Plateau EV
Saturation interval
Storage reduction
128M
16k
0.6393
8M–16M
8–16 ×
128M
32k
0.6610
16M–32M
4–8 ×
128M
64k
0.6648
16M–32M
4–8 ×
128M
128k
0.6730
16M–32M
4–8 ×
640M
16k
0.6522
20M–40M
16–32 ×
640M
32k
0.6803
40M–80M
8–16 ×
Table 1: Saturation intervals for each dictionary size and total training budget ( ϵ=0.005 ).
Figure 3: Validation across layers and models ( D=32k , N=1.28B ). Llama-3.1 8B layer sweeps use BatchTopK SAEs, and sweeps across Qwen3 model sizes use TopK SAEs. All curves show the same saturation pattern as diversity increases despite changes in layer, model size, and SAE activation function.
Activation
Sparsity
Plateau EV
usat
Storage reduction
BatchTopK
average k=32
0.6803
40M
16×
TopK
k=32
0.6807
40M
16×
JumpReLU
L0≈100
0.7634
40M
16×
Table 2: Transfer across activation functions at D=32k , Ntotal=640M , Llama-3.1 8B layer 15. BatchTopK and TopK are matched at k=32 . JumpReLU converges to L0≈100 and is therefore less sparse. All three variants saturate at 40M unique activations, showing that the diversity–repetition tradeoff is not specific to BatchTopK.
Figure 4: SAEBench metrics versus the number of unique activations for the BatchTopK sweep at layer 15 with a budget of 640M tokens. Dotted vertical lines mark the buffer chosen from the EV plateau for each dictionary. Reconstruction MSE follows the EV plateau, and AutoInterp remains high beyond this buffer scale. Feature absorption measures disentanglement. Sparse probing changes little with buffer size. None of the evaluated downstream metrics drops sharply at the recommended buffer size. Lower MSE and absorption fraction are better. Higher AutoInterp and sparse probing accuracy are better.
Domain
10M
20M
40M
80M
640M
U0.005 / reduction
FineMath-3plus
0.6312
0.6523
0.6608
0.6649
0.6673
80M / 8×
Stack-Edu Python
0.7013
0.7081
0.7119
0.7143
0.7155
40M / 16×
Table 3: Additional domain sweeps: held-out EV at each tested buffer size, with 640M total presentations. The reference is each domain’s 640M run without replay.
Domain
ϵ=0.0025
ϵ=0.005
ϵ=0.010
FineMath-3plus
80M / 8×
80M / 8×
40M / 16×
Stack-Edu Python
80M / 8×
40M / 16×
20M / 32×
Table 4: First qualifying tested buffer / storage reduction at three absolute EV tolerances, relative to the domain’s 640M run.
Figure 5: Capacity-corrected EV ( EV+AD−α ) versus effective activation count Neff under the extended R∗(D) model. Data points from all four dictionary sizes and three total budgets collapse onto a single power-law curve, supporting the functional form within the fitted regime.
Budget
16k
32k
64k
128k
128M
7.7
5.0
6.1
8.1
640M
1.8
1.6
2.9
2.4
1.28B
5.9
2.5
1.6
5.2
Table 5: RMSE for each panel used in the fit ( ×10−3 EV), with eight buffer sizes per cell. Residuals concentrate at the smallest budget and extreme widths at the largest budget.
Appendix figures & tables4 assets
Supplementary material from the paper’s appendix.
Appendix
Model
E∞
A
α
B
β
R∗ / c
γ
Δ AIC
Const. R∗
0.738
0.387
0.688
0.432
0.466
13.7
—
0
Extended R∗(D)
0.760
0.328
0.486
0.440
0.477
103.3
0.526
−77.3
Appendix
Table 6: Fitted parameters for the model with constant R∗ and the extended R∗(D) model, fit on all 96 data points from the main grid. The extended model is decisively preferred by AIC.
Quantity
D=16 k
D=32 k
D=64 k
Explained variance (main)
20M
40M
80M
Reconstruction MSE
80M
80M
80M
Feature absorption fraction
80M
40M
80M
Automated interpretability score
40M
160M
160M
SAE sparse probing accuracy
20M
20M
20M
Appendix
Table 7: SAEBench saturation thresholds at Ntotal=640M , Llama-3.1 8B layer 15, BatchTopK. The EV row is reproduced from Table 1 .
Table 9: Buffer recommendations at 640M total presentations for three EV tolerances. The reference is the best final EV on each sweep over eight buffer sizes. Each cell gives the smallest qualifying buffer and the resulting storage reduction.
Sparse autoencoders (SAEs) are typically trained on hundreds of millions to billions of activation tokens, yet little work has tested whether this scale is necessary. We study how much activation data SAEs need by training four SAE architectures on three language models with budgets ranging from 5M tokens to a 500M-token reference. We measure two notions of convergence: functional behavior on standard evaluations and one-to-one reproducibility of learned features. Functional data requirements are strongly metric-dependent: reconstruction often approaches the full-data reference at substantially smaller budgets, whereas feature reproducibility continues to improve with data. For Gemma-2-2B BatchTopK, the fraction of strictly matched features rises from 2% at 5M tokens to 29% at 50M. Data selection also matters: random sampling better preserves the original activation distribution and generally favors reconstruction, whereas diversity-based sampling frequently improves AutoInterp. A theoretical analysis separates statistical consistency of subsampled training from the stronger identifiability and optimization conditions required for one-to-one feature recovery. Our results show that SAE training can often be made far more data-efficient when budgets are matched to the goal, and feature-level claims are backed by explicit reproducibility checks, paving the way toward cheaper and more reliable SAE research.
Sambit Ghosh, Madelyn Mathai, R. Venkatesh Babu +1
IBM · Indian Institute of Science · University of Virginia
To train a sparse autoencoder (SAE), you need a massive cache of model activations. We look into the effects that compression has on an SAE in two ways: we keep the original fp32-trained SAE and pass the compressed cache into the same SAE to see how many feature activations stay the same (the frozen-SAE test), and we retrain an SAE on the compressed cache and see how many of the original features still exist (the retraining test). We found that in the retraining test, there is no clear scale for retained features. On a GPT-2 cache, rounding everything to fp16 keeps 60.6% of features, but changing one value in a million by the smallest step fp32 allows (a change eight orders of magnitude smaller) keeps only 70.5%. Meanwhile, changing the random seed keeps 23.0%. For the frozen-SAE test, the frozen SAE's correlation with the original increases in order of how precise the codecs are, but plain L2 distance gives nearly the same ordering. We also see the same frozen reading correspond to different percentages retained at different sites. This leaves the question of the right measurement. We build one by calibrating: retraining on identical bytes sets the ceiling, and a reseed sets the floor. We then measure the effect of each compressed codec against its own site's floor. With TopK SAEs on GPT-2 small and Pythia-1.4B at research-scale budgets, naïve per-token int4 loses more features than a reseed everywhere we tested, as do PCA and random projection on GPT-2. Standardizing each channel before quantizing brings int4 back above the floor in terms of retained features, at the same raw file size. At our smallest budget, a bf16 forward pass, which some pipelines use to save memory, loses slightly more features than rotated 8-bit storage. Calibrating a site costs four training runs and two evaluation passes.
Sparse Autoencoders (SAEs) extract features from LLM internal activations, meant to correspond to interpretable concepts. A core SAE training hyperparameter is L0: how many SAE features should fire per token on average. Existing work compares SAE algorithms using sparsity-reconstruction tradeoff plots, implying L0 is a free parameter with no inherently correct value aside from its effect on reconstruction. In this work we study the effect of L0 on SAEs, and show that if L0 is not set correctly, the SAE fails to disentangle the underlying features of the LLM. If L0 is too low, the SAE will mix correlated features to improve reconstruction. If L0 is too high, the SAE finds degenerate solutions that also mix features. Further, we present a proxy metric that can help guide the search for the correct L0 for an SAE on a given training distribution. We show that our method finds the correct L0 in toy models and coincides with peak sparse probing performance in LLM SAEs. We find that most commonly used SAEs have an L0 that is too low. Our work shows that practitioners must set L0 correctly to train SAEs with monosemantic features.