Scaling Laws for SAE Training Data

Nikita Koriagin; Nikita Balagansky; Daniil Gavrilov

cs.LGNov 5, 2026

Scaling Laws for SAE Training Data

Authors: Nikita Koriagin, Nikita Balagansky, Daniil Gavrilov

Organizations: Yandex · T-Tech

Abstract

Activation storage is often a bottleneck in sparse autoencoder (SAE) training. We study how many unique activations an SAE needs when its total training budget is fixed. Quality improves rapidly as the buffer grows, then reaches a plateau where replaying the buffer is nearly as effective as using fresh activations throughout training. Beyond this point, additional unique activations bring little benefit to reconstruction or interpretability, so the buffer need not grow with the training budget. We describe this tradeoff between diversity and repetition with a scaling law for limited data. We validate it across dictionary sizes and token budgets, early, middle, and late layers of Llama-3.1, Qwen3 model sizes, and BatchTopK, TopK, and JumpReLU SAEs. Sweeps using mathematical text and Python code test the effect of changing the activation source. Evaluation covers explained variance (EV), reconstruction MSE, feature absorption, automated interpretability, and sparse probing. These results give a practical training rule: choose the smallest buffer that reaches the quality plateau and replay it for the remaining budget. In our experiments, this reduces activation storage by $8\text{--}64\times$ with negligible quality loss.

Figures & tables

Panel of Figure 1 — Figure 1: Fresh activations and replay at a fixed SAE optimization budget. The replay pipeline generates and stores $U_{\mathrm{buf}}$ activations once and uses $E=N_{\mathrm{total}}/U_{\mathrm{buf}}$ passes. $s$ denotes bytes per activation. Generation counts refer to token activations. Batching and sequence packing determine the number of calls to the base model.

Figure 2: Explained variance as a function of unique activation count under fixed total SAE training budgets. With total optimization tokens fixed in each panel, moving right increases activation diversity and decreases the number of epochs over the buffer. EV improves sharply when activation diversity is low and then saturates.

Budget	Dict. size	Plateau EV	Saturation interval	Storage reduction
128M	16k	$0.6393$	8M–16M	8–16 $\times$
128M	32k	$0.6610$	16M–32M	4–8 $\times$
128M	64k	$0.6648$	16M–32M	4–8 $\times$
128M	128k	$0.6730$	16M–32M	4–8 $\times$
640M	16k	$0.6522$	20M–40M	16–32 $\times$
640M	32k	$0.6803$	40M–80M	8–16 $\times$

Table 1: Saturation intervals for each dictionary size and total training budget (

\epsilon=0.005

Figure 3: Validation across layers and models (

D=32\mathrm{k}

N=1.28\mathrm{B}

). Llama-3.1 8B layer sweeps use BatchTopK SAEs, and sweeps across Qwen3 model sizes use TopK SAEs. All curves show the same saturation pattern as diversity increases despite changes in layer, model size, and SAE activation function.

Activation	Sparsity	Plateau EV	$u_{\mathrm{sat}}$	Storage reduction
BatchTopK	average $k=32$	$0.6803$	40M	$16\times$
TopK	$k=32$	$0.6807$	40M	$16\times$
JumpReLU	$L_{0}\approx 100$	$0.7634$	40M	$16\times$

Table 2: Transfer across activation functions at

D=32\mathrm{k}

N_{\mathrm{total}}=640\mathrm{M}

, Llama-3.1 8B layer 15. BatchTopK and TopK are matched at

k=32

. JumpReLU converges to

L_{0}\approx 100

and is therefore less sparse. All three variants saturate at 40M unique activations, showing that the diversity–repetition tradeoff is not specific to BatchTopK.

Figure 4: SAEBench metrics versus the number of unique activations for the BatchTopK sweep at layer 15 with a budget of 640M tokens. Dotted vertical lines mark the buffer chosen from the EV plateau for each dictionary. Reconstruction MSE follows the EV plateau, and AutoInterp remains high beyond this buffer scale. Feature absorption measures disentanglement. Sparse probing changes little with buffer size. None of the evaluated downstream metrics drops sharply at the recommended buffer size. Lower MSE and absorption fraction are better. Higher AutoInterp and sparse probing accuracy are better.

Domain	10M	20M	40M	80M	640M	$U_{0.005}$ / reduction
FineMath-3plus	0.6312	0.6523	0.6608	0.6649	0.6673	80M / $8\times$
Stack-Edu Python	0.7013	0.7081	0.7119	0.7143	0.7155	40M / $16\times$

Table 3: Additional domain sweeps: held-out EV at each tested buffer size, with 640M total presentations. The reference is each domain’s 640M run without replay.

Domain	$\epsilon=0.0025$	$\epsilon=0.005$	$\epsilon=0.010$
FineMath-3plus	80M / $8\times$	80M / $8\times$	40M / $16\times$
Stack-Edu Python	80M / $8\times$	40M / $16\times$	20M / $32\times$

Table 4: First qualifying tested buffer / storage reduction at three absolute EV tolerances, relative to the domain’s 640M run.

Figure 5: Capacity-corrected EV (

\mathrm{EV}+AD^{-\alpha}

) versus effective activation count

N_{\mathrm{eff}}

under the extended

R^{*}(D)

model. Data points from all four dictionary sizes and three total budgets collapse onto a single power-law curve, supporting the functional form within the fitted regime.

Budget	16k	32k	64k	128k
128M	7.7	5.0	6.1	8.1
640M	1.8	1.6	2.9	2.4
1.28B	5.9	2.5	1.6	5.2

Table 5: RMSE for each panel used in the fit (

\times 10^{-3}

EV), with eight buffer sizes per cell. Residuals concentrate at the smallest budget and extreme widths at the largest budget.

Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

Model	$E_{\infty}$	$A$	$\alpha$	$B$	$\beta$	$R^{*}$ / $c$	$\gamma$	$\Delta$ AIC
Const. $R^{*}$	$0.738$	$0.387$	$0.688$	$0.432$	$0.466$	$13.7$	—	$0$
Extended $R^{*}(D)$	$0.760$	$0.328$	$0.486$	$0.440$	$0.477$	$103.3$	$0.526$	$-77.3$

Appendix

Table 6: Fitted parameters for the model with constant

R^{*}

and the extended

R^{*}(D)

model, fit on all 96 data points from the main grid. The extended model is decisively preferred by AIC.

Quantity	$D=16$ k	$D=32$ k	$D=64$ k
Explained variance (main)	20M	40M	80M
Reconstruction MSE	80M	80M	80M
Feature absorption fraction	80M	40M	80M
Automated interpretability score	40M	160M	160M
SAE sparse probing accuracy	20M	20M	20M

Appendix

Table 7: SAEBench saturation thresholds at

N_{\mathrm{total}}=640\mathrm{M}

, Llama-3.1 8B layer 15, BatchTopK. The EV row is reproduced from Table 1 .

Budget	Dict. size	$u_{\mathrm{sat}}$	Epochs	Storage reduction
128M	16k	8M	16	$16\times$
128M	32k	16M	8	$8\times$
128M	64k	16M	8	$8\times$
128M	128k	16M	8	$8\times$
640M	16k	20M	32	$32\times$
640M	32k	40M	16	$16\times$

Appendix

Table 8: Recommended activation buffer sizes and implied storage reductions (

\epsilon=0.005

). Epochs

=N_{\mathrm{total}}/u_{\mathrm{sat}}

Width	$\epsilon=0.0025$	$\epsilon=0.005$	$\epsilon=0.010$
16k	40M / $16\times$	20M / $32\times$	20M / $32\times$
32k	80M / $8\times$	40M / $16\times$	20M / $32\times$
64k	80M / $8\times$	80M / $8\times$	40M / $16\times$
128k	160M / $4\times$	80M / $8\times$	40M / $16\times$

Appendix

Table 9: Buffer recommendations at 640M total presentations for three EV tolerances. The reference is the best final EV on each sweep over eight buffer sizes. Each cell gives the smallest qualifying buffer and the resulting storage reduction.

Sparse Autoencoders

Sparse autoencoders decompose neural network activations into sparse features to study, interpret, and control learned representations.

See all 233 papers

Diminishing Returns of Data in Sparse Autoencoders: Functional Behavior does not Guarantee Feature Reproducibility
Oct 14, 2026Sambit Ghosh, Madelyn Mathai, R. Venkatesh Babu +1Sparse Autoencoders Data-Efficient Learning
IBM · Indian Institute of Science · University of Virginia
How Small Can We Go? Calibrating Activation-Cache Compression for SAE Training
Nov 5, 2026Abraham YeungActivation Compression Sparse Autoencoders
Stanford University
Sparse but Wrong: Incorrect L0 Leads to Incorrect Features in Sparse Autoencoders
Aug 22, 2025David Chanin, Adrià Garriga-AlonsoActivation Sparsity LLM Interpretability
University College London · MATS Research

Oct 14, 2026J/K move · Enter open · S save

Diminishing Returns of Data in Sparse Autoencoders: Functional Behavior does not Guarantee Feature Reproducibility

Sambit Ghosh, Madelyn Mathai, R. Venkatesh Babu, Chirag Agarwal

IBM · Indian Institute of Science · University of Virginia

Sparse Autoencoders Data-Efficient Learning ML Reproducibility Data Selection

Open paper AskPDF arXiv

Sparse autoencoders (SAEs) are typically trained on hundreds of millions to billions of activation tokens, yet little work has tested whether this scale is necessary. We study how much activation data SAEs need by training four SAE architectures on three language models with budgets ranging from 5M tokens to a 500M-token reference. We measure two notions of convergence: functional behavior on standard evaluations and one-to-one reproducibility of learned features. Functional data requirements are strongly metric-dependent: reconstruction often approaches the full-data reference at substantially smaller budgets, whereas feature reproducibility continues to improve with data. For Gemma-2-2B BatchTopK, the fraction of strictly matched features rises from 2% at 5M tokens to 29% at 50M. Data selection also matters: random sampling better preserves the original activation distribution and generally favors reconstruction, whereas diversity-based sampling frequently improves AutoInterp. A theoretical analysis separates statistical consistency of subsampled training from the stronger identifiability and optimization conditions required for one-to-one feature recovery. Our results show that SAE training can often be made far more data-efficient when budgets are matched to the goal, and feature-level claims are backed by explicit reproducibility checks, paving the way toward cheaper and more reliable SAE research.

Abstract

Sparse Autoencoders

Explore similar work

Diminishing Returns of Data in Sparse Autoencoders: Functional Behavior does not Guarantee Feature Reproducibility

How Small Can We Go? Calibrating Activation-Cache Compression for SAE Training

Sparse but Wrong: Incorrect L0 Leads to Incorrect Features in Sparse Autoencoders