Behavioral Compositionality of Sparse Autoencoder Representations

Haotian Ma; Ziye Mei; Frederic Sala

cs.LGOct 15, 2026

Behavioral Compositionality of Sparse Autoencoder Representations

Authors: Haotian Ma, Ziye Mei, Frederic Sala

Organizations: Department of Computer Sciences, University of Wisconsin–Madison

Abstract

Sparse autoencoders (SAEs) represent model activations with sparse, linearly decoded features, but the downstream effects of those features need not compose additively. We introduce SAE Additivity Residuals (SAR), which measure the behavioral variation left unexplained above a chosen interaction order. Under independent feature subset sampling, population SAR is exactly the normalized energy of interactions above that order and bounds coalition ordering error for the optimal low order approximation. To distinguish compositionality associated with the SAE representation from that induced by downstream computation, we compare learned SAE coordinates with sum preserving random rotations of the same selected subspace and find lower first order residuals for the learned coordinates across all 18 model and behavior settings. The coordinates are nevertheless not fully additive: pairwise terms improve held out coalition ordering in every setting and outperform parameter matched models using randomly selected third order terms. In a combined steering experiment, pairwise modeling raises rank fidelity from 0.5874 to 0.7790 and reduces ordering error from 0.1444 to 0.0606. Together, these results show that SAE coordinates support comparatively compositional behavioral descriptions while faithful joint explanations and interventions still require interactions beyond individual feature effects. Code is available at https://github.com/OpenSAE/SAE-Behavioral-Compositionality.

Figures & tables

Figure 1: Behavioral compositionality in SAE coordinates. (a) Selected decoded contributions are retained or removed while all other activation content is fixed, and the patched activation is propagated through the frozen upper layers to obtain

g_{x}(S)

. (b, c) Nearly additive and interacting responses, showing the first order fit, pairwise interactions, and variance decomposition.

\mathrm{SAR}_{1}

is the residual beyond individual features and

\rho_{2}=\Delta_{2}/\mathrm{SAR}_{1}

its pairwise share.

Figure 2: Prompt median

\widehat{\mathrm{SAR}}_{1}

for six behaviors in Gemma-3 4B layer 22, Mistral-7B layer 24, and Llama-3.1 8B layer 21. Error bars show 95% prompt bootstrap intervals.

Figure 3: Paired first order SAR for learned SAE coordinates and sum preserving random rotations of the same selected activation subspace.

Figure 4: First order explanation fidelity across

\widehat{\mathrm{SAR}}_{1}

deciles within each model.

Figure 5: Held out coalition ordering for first order and pairwise approximations.

Figure 6: Depth profiles for simple copying, CounterFact, and IOI.

Approximation	Residual estimate	Rank fidelity	Ordering error
First order	0.6424	0.5874	0.1444
Pairwise	0.3891	0.7790	0.0606

Table 1: Held out residual estimates and coalition ordering for MMLU-Pro steering response.

	Sign error		Selection regret
Behavior	$\mathrm{FVU}_{x}$	$\smash[t]{\widehat{\mathrm{SAR}}}_{1}$	$\mathrm{FVU}_{x}$	$\smash[t]{\widehat{\mathrm{SAR}}}_{1}$
Simple copying	-0.080	0.269	0.011	0.220
Subject verb agreement	-0.114	0.430	0.005	0.235
IOI	-0.013	0.396	0.028	0.199

Table 2: Prompt level Spearman correlations pooled over six SAEs (

n=1{,}800

per behavior).

Appendix figures & tables24 assets

Supplementary material from the paper’s appendix.

Appendix

Symbol	Meaning
$x$	Input prompt.
$h_{x}$	Model activation at the selected layer and token position.
$z_{i}(x),d_{i}$	Activation and decoder direction of SAE feature $i$ .
$\delta_{i}(x)$	Decoded contribution $z_{i}(x)d_{i}$ .
$N_{x}$	Set of top $K$ active SAE features for prompt $x$ .
$S$	Coalition of retained features, $S\subseteq N_{x}$ .

Appendix

Table 3: Notation used throughout the paper.

Diagnostic	Result
Maximum monomial and Walsh prediction difference	$1.67\times 10^{-12}$
Order 1 full rank prompts	2,700 / 2,700
Order 2 full rank prompts	2,700 / 2,700
Order 2 Gram condition number, median	41.02
Order 2 Gram condition number, 95th percentile	49.88
Relative split variation of first order residual, median	2.29%

Appendix

Table 4: Finite sample diagnostics for the 256 fitting and 128 evaluation coalition protocol. Split variation is the standard deviation across 20 random splits divided by the corresponding split mean. The final two rows compare a 256 coalition fit with a fit using all 384 sampled coalitions and normalize the mean squared prediction difference by behavioral variance.

Fit coalitions	Condition	$R^{\mathrm{var}}_{2}$	$F_{\mathrm{rank}}^{(2)}$	$\mathcal{E}_{2}$	Rank improved
160	580.3	0.03071	0.9783	0.001085	9 / 18
192	134.3	0.01516	0.9885	0.0003920	17 / 18
224	65.43	0.01104	0.9915	0.0002436	18 / 18
256	41.02	0.009152	0.9929	0.0001839	18 / 18

Appendix

Table 5: Fitting budget diagnostics with the original evaluation coalitions held fixed. The reported values aggregate the 18 model and behavior combinations.

Behavior	Prompt or source	Scalar behavior $\psi$
Affine control	Simple copying prompts	Linear readout of the intervened activation
Simple copying	Repeated name prompts	Target name logit difference
Subject verb agreement	Agreement prompts ending before the verb	Correct verb logit difference
Gendered pronoun preference	Biography prompts ending before the pronoun	Pronoun logit difference
CounterFact	Factual prompts from CounterFact ( Meng et al., 2022 )	Target object token logit
IOI	Indirect object identification prompts ( Wang et al., 2023 )	Indirect object logit difference

Appendix

Table 6: Behavior definitions used in the main experiments.

Index	Layer	Feature ID	Steering coefficient
1	26	22133	0.813
2	19	6432	0.966
3	22	11286	0.697
4	18	2624	0.478
5	20	28406	0.628
6	17	26604	0.469

Appendix

Table 7: Fixed SAE features used in the MMLU-Pro steering experiment.

Model	SAE family	Layers analyzed
GPT-2 Small	Bloom GPT-2 SAE	4, 6, 8, 10
Pythia-70M	Marks Pythia SAE	1, 3, 5
Gemma-3 270M	Gemma Scope 2	5, 9, 12, 15
Gemma-3 1B	Gemma Scope 2	7, 13, 17, 22
Gemma-3 4B	Gemma Scope 2	9, 17, 22, 29
Mistral-7B	Engels residual SAE	8, 16, 24

Appendix

Table 8: Model and SAE pipeline coverage.

Game	$\widehat{\mathrm{SAR}}_{1}$	$\widehat{\mathrm{SAR}}_{2}$
Additive	$6.1\times 10^{-12}$	$2.6\times 10^{-10}$
Pairwise with target residual 0.1	0.100	$\approx 0$
Pairwise with target residual 0.3	0.300	$\approx 0$
Pairwise with target residual 0.5	0.500	$\approx 0$
Third order	0.180	0.066

Appendix

Table 9: Synthetic checks for the SAR estimator.

Model	Behavior	SAE $\smash[t]{\widehat{\mathrm{SAR}}}_{1}$	Rotation $\smash[t]{\widehat{\mathrm{SAR}}}_{1}$	SAE below rotation median
Gemma-3 4B	Simple copying	0.04097	0.08333	0.98
	Subject verb agreement	0.03332	0.07638	0.99
	Gendered pronoun preference	0.03872	0.08080	1.00
	CounterFact	0.05558	0.09428	0.92
	IOI	0.02132	0.02402	0.66
	Greater than	0.02165	0.03198	0.98

Appendix

Table 10: Estimated first order SAR for learned SAE coordinates and random coordinate rotations within the same selected activation subspace. Each row uses 100 prompts and eight rotations per prompt. The final column gives the fraction of prompts whose SAE estimate lies below the median of its rotations.

Pipeline and layer	Behavior	$n$	Median $\smash[t]{\widehat{\mathrm{SAR}}}_{1}$	Median $\widehat{\rho}_{2}$
Gemma-3 4B L22	Simple copying	1000	0.03668 [0.03523, 0.03791]	0.6530 [0.6368, 0.6687]
	Subject verb agreement	1000	0.03732 [0.03496, 0.03877]	0.6274 [0.5978, 0.6518]
	Gendered pronoun preference	1000	0.03532 [0.03426, 0.03634]	0.8296 [0.8247, 0.8348]
	CounterFact	1000	0.05275 [0.05128, 0.05423]	0.7348 [0.7190, 0.7447]
	IOI	1000	0.04174 [0.03867, 0.04507]	0.7346 [0.7229, 0.7412]
	Greater than	1000	0.02121 [0.02090, 0.02162]	0.2814 [0.2653, 0.2949]

Appendix

Table 11: Fixed layer behavior estimates with 1,000 prompts for each combination. Brackets give 95% confidence intervals from prompt bootstrap resampling.

Pipeline	Behavior	Early $\to$ late $\smash[t]{\widehat{\mathrm{SAR}}}_{1}$	Reduction	Early $\to$ late $\widehat{\rho}_{2}$
Gemma-3 4B	Simple copying	0.1122 $\to$ 0.01155	9.711 [9.147, 10.29]	0.5835 $\to$ 0.7085
	CounterFact	0.1723 $\to$ 0.07646	2.254 [2.112, 2.483]	0.6929 $\to$ 0.8614
	IOI	0.2255 $\to$ 0.07320	3.080 [2.948, 3.248]	0.5615 $\to$ 0.8358
Mistral-7B	Simple copying	0.1771 $\to$ 0.02029	8.728 [8.232, 9.242]	0.5261 $\to$ 0.7882
	CounterFact	0.2658 $\to$ 0.05535	4.801 [4.432, 5.201]	0.4415 $\to$ 0.8255
	IOI	0.1270 $\to$ 0.01510	8.412 [7.865, 8.934]	0.3411 $\to$ 0.7552

Appendix

Table 12: Paired early to late estimates for the three detailed depth profiles. Brackets in the reduction column give 95% confidence intervals from paired prompt bootstrap resampling.

Figure 7: Simple copying at the earliest and latest sampled layers across seven model and SAE pipelines. Open markers denote the earliest layer and filled markers denote the latest layer. The left panel reports prompt median

\widehat{\mathrm{SAR}}_{1}

and the right panel reports prompt median

\widehat{\rho}_{2}

Pipeline	Early layer	Late layer	Early $\to$ late $\smash[t]{\widehat{\mathrm{SAR}}}_{1}$	Early $\to$ late $\widehat{\rho}_{2}$
GPT-2 Small	4	10	0.2978 $\to$ 0.01309	0.3437 $\to$ 0.9491
Pythia-70M	1	5	0.1373 $\to$ 0.0006608	0.5946 $\to$ 0.9979
Gemma-3 270M	5	15	0.1738 $\to$ 0.006676	0.6962 $\to$ 0.9854
Gemma-3 1B	7	22	0.1543 $\to$ 0.01521	0.7032 $\to$ 0.9763
Gemma-3 4B	9	29	0.1122 $\to$ 0.01155	0.5835 $\to$ 0.7085
Mistral-7B	8	24	0.1771 $\to$ 0.02029	0.5261 $\to$ 0.7882

Appendix

Table 13: Estimated simple copying endpoints across seven model and SAE pipelines.

Pipeline	Layer	Median $\smash[t]{\widehat{\mathrm{SAR}}}_{1}$	Median $\widehat{\mathrm{SAR}}_{2}$	Median $\widehat{\rho}_{2}$
Gemma-3 4B	9	0.2045	0.07585	0.6223
	17	0.1436	0.09039	0.3809
	22	0.02121	0.01495	0.2814
	29	0.06386	0.009314	0.8551
Llama-3.1 8B	8	0.3227	0.1005	0.6623
	16	0.1302	0.03133	0.7385

Appendix

Table 14: Estimated greater than depth profiles in the two pipelines with complete depth measurements.

	Rank fidelity		Selection regret
Model	Lowest decile	Highest decile	Lowest decile	Highest decile
Gemma-3 4B	0.9857	0.9539	0.01812	0.03623
Mistral-7B	0.9939	0.9413	0.009118	0.04684
Llama-3.1 8B	0.9790	0.9289	0.02711	0.05173

Appendix

Table 15: First order explanation fidelity in the lowest and highest

\smash[t]{\widehat{\mathrm{SAR}}}_{1}

deciles within each model.

		Sign error		Selection regret
Model	Behavior	First order	Pairwise	First order	Pairwise
Gemma-3 4B	Simple copying	0.0892	0.0420	0.0150	0.0044
	Subject verb agreement	0.0784	0.0566	0.0175	0.0083
	Gendered pronoun preference	0.1070	0.0415	0.0261	0.0050
	CounterFact	0.0457	0.0418	0.0176	0.0034
	IOI	0.1275	0.0514	0.0288	0.0041

Appendix

Table 16: First order and pairwise explanation errors for all 18 model and behavior combinations.

Model	Behavior	$F_{\mathrm{rank}}^{(1)}$	$F_{\mathrm{rank}}^{(2)}$
Gemma-3 4B	Simple copying	0.9857	0.9970
	Subject verb agreement	0.9883	0.9967
	Gendered pronoun preference	0.9690	0.9964
	CounterFact	0.9851	0.9961
	IOI	0.9751	0.9959
	Greater than	0.9744	0.9887

Appendix

Table 17: Mean prompt level rank fidelity of the first order and pairwise approximations for the 18 model and behavior combinations.

Model	Behavior	$\widehat{\mathcal{E}}_{1}$	$\widehat{\mathcal{E}}_{2}$
Gemma-3 4B	Simple copying	0.0009103	0.0002062
	Subject verb agreement	0.0005988	0.0001896
	Gendered pronoun preference	0.001169	0.0001176
	CounterFact	0.0009869	0.0002584
	IOI	0.001248	0.0002133
	Greater than	0.0007911	0.0005375

Appendix

Table 18: Held out behavior weighted ordering error for first order and pairwise approximations in the 18 model and behavior combinations used in the fidelity analysis.

Model	Behavior	Pairwise rank	Matched rank	Pairwise $\widehat{\mathcal{E}}$	Matched $\widehat{\mathcal{E}}$
Gemma-3 4B	Simple copying	0.9934	0.9802	0.0002062	0.001488
	Subject verb agreement	0.9934	0.9845	0.0001896	0.0009812
	Gendered pronoun preference	0.9925	0.9597	0.0001176	0.001838
	CounterFact	0.9926	0.9772	0.0002584	0.001730
	IOI	0.9925	0.9690	0.0002133	0.001799
	Greater than	0.9800	0.9682	0.0005375	0.001151

Appendix

Table 19: Held out ordering for the pairwise model and parameter matched models that replace pairwise terms with randomly sampled third order Walsh terms. All columns report prompt medians; matched quantities first take the median over 50 random models for each prompt.

Domain	$\smash[t]{\widehat{\mathrm{SAR}}}_{1}$	$F_{\mathrm{rank}}^{(1)}$	$\widehat{\mathrm{SAR}}_{2}$	$\widehat{\rho}_{2}$	$F_{\mathrm{rank}}^{(2)}$	Rank fidelity change
Biology	0.5815	0.6463	0.5600	0.0370	0.6684	+0.0221
Business	0.6666	0.5417	0.3878	0.4183	0.6784	+0.1367
Chemistry	0.9179	0.3170	0.5650	0.3844	0.7130	+0.3960
Computer Science	0.5420	0.6841	0.5356	0.0119	0.6962	+0.0121
Economics	0.5781	0.6558	0.6150	-0.0639	0.6465	-0.0093
Engineering	0.2238	0.8731	0.1871	0.1637	0.8689	-0.0042

Appendix

Table 20: Held out residual estimates and rank fidelity for steering combinations on MMLU-Pro. The first order and pairwise models are fitted on one set of sampled coalitions and evaluated on a separate set. The final column reports

F_{\mathrm{rank}}^{(2)}-F_{\mathrm{rank}}^{(1)}

Domain	$\widehat{\mathcal{E}}_{1}$	$\widehat{\mathcal{E}}_{2}$
Biology	0.1187	0.1080
Business	0.1287	0.06926
Chemistry	0.3078	0.09323
Computer Science	0.1043	0.09596
Economics	0.1071	0.1073
Engineering	0.02049	0.02116

Appendix

Table 21: Held out behavior weighted ordering error for the steering responses.

Figure 8: Median prompt level

\mathrm{FVU}_{x}

and median

\widehat{\mathrm{SAR}}_{1}

for six SAEs at Gemma-3 1B layer 17. Panels show simple copying, subject verb agreement, and IOI.

Figure 9: Behavior scores for four examples under ablations of individual features and feature pairs, together with first order and pairwise predictions when available.

Panel	Model	Behavior	Prompt ID	$\smash[t]{\widehat{\mathrm{SAR}}}_{1}$	$\widehat{\mathrm{SAR}}_{2}$	$\widehat{\rho}_{2}$
(a)	Mistral-7B	Simple copying	simple_copying/02819	0.00325	0.00159	0.509
(b)	Llama-3.1 8B	Gendered pronoun preference	gendered_pronoun/00310	0.1677	0.0384	0.771
(c)	Llama-3.1 8B	Subject verb agreement	subject_verb_agreement/03544	0.00765	0.00103	0.865
(d)	Llama-3.1 8B	Gendered pronoun preference	gendered_pronoun/00072	0.1412	0.0417	0.705

Appendix

Table 22: Prompts used in Figure 9 .

	Number of selected features $K$
Behavior or setting	6	8	10	12	14	16	18	20
Affine control	6.87	6.40	5.70	5.16	5.03	5.43	5.07	4.84
Greater than	0.0460	0.0575	0.0634	0.0707	0.0763	0.0823	0.0868	0.0872
IOI	0.0554	0.0559	0.0597	0.0640	0.0652	0.0662	0.0664	0.0693
Simple copying	0.0213	0.0217	0.0234	0.0245	0.0270	0.0299	0.0307	0.0320
MMLU-Pro steering	0.2577	0.4951	0.6595	0.6350	0.5804	0.7768	0.8096	0.7861

Appendix

Table 23: Sensitivity of

\smash[t]{\widehat{\mathrm{SAR}}}_{1}

to the number of selected SAE features. The active feature rows report prompt median

\smash[t]{\widehat{\mathrm{SAR}}}_{1}

at Gemma-3 1B layer 17. Affine control values are reported in units of

10^{-14}

. The steering row reports

\smash[t]{\widehat{\mathrm{SAR}}}_{1}

for the overall MMLU-Pro accuracy game. Feature sets are nested across

K

Sparse Autoencoders

Sparse autoencoders decompose neural network activations into sparse features to study, interpret, and control learned representations.

See all 217 papers

Oct 11, 2026

What Do Sparse Autoencoder Feature Discovery Scores Identify? From Activations to Interventions

Sparse autoencoder (SAE) features are typically discovered from how their activations co-vary with behaviour, yet they are often used through interventions such as steering or ablation. We take intervention attribution, the first-order effect of a specified intervention on the model output, as our estimand. For a broad class of response functions, we show that observational feature-discovery evidence generally does not identify this attribution, and derive an exact gradient-based decomposition of the gap between linear-probe predictions and intervention attribution. Across four model-SAE pairs, median probe-to-attribution magnitude ratios range from tens to hundreds, while attributions closely track small-step steering. For single-latent ablations with nonzero effects, gradient predictions match the realized sign in 91-97% of cases, compared with 17-35% for probe-implied changes. Attribution-selected sets also produce larger absolute mean ablation effects on held-out prompts than sets selected by correlation, rank tests or probe coefficients. Together, these results establish intervention attribution as a principled, gradient-estimable target for selecting SAE features for intervention.

Yifeng Guo, Yanran Li, Xiang Chen

St. Jude Children’s Research Hospital · Columbia University

Sparse Autoencoders Mechanistic Interpretability Causal Interventions in Language Models Gradient-Based Attribution

Ask PDF

Jul 27, 2026

Sparse Autoencoders Encode Both Concepts and Functions: The Downstream Geometry of Feature Effects

The wide-scale use of sparse autoencoders (SAEs) as interpretability tools is limited by inconsistent links between SAE features and model behavior. Features with clear activation descriptions may have weak or unexpected causal effects; steering can vary across prompts or oppose the intended direction; and activation-based feature selection can miss features that produce the desired output change. Prior work has studied feature geometry inside the model, where features are computed. We instead study the geometry of changes in model logits caused by feature interventions. We introduce Feature-Effect Geometry Analysis (FEGA), an unsupervised framework that removes the same active SAE feature across contexts and analyzes the resulting cloud of logit changes. Across SAE variants, consistent one-dimensional effects are rare: few features behave like reusable directions. To interpret this variation, we distinguish value-like features, tied to static information such as factual attributes, from pointer-like features, associated with context-dependent operations. Value-like features more often exhibit structured, low-dimensional effects, although these effects typically span several directions. Pointer-like features, by contrast, predominantly exhibit diffuse effects. Our results show that a feature can be interpretable and causally relevant without providing a stable direction for steering.

Phu Gia Hoang, Anwoy Chatterjee, Tanmoy Chakraborty +2

UKP Lab, Technical University of Darmstadt · Department of Electrical Engineering Indian Institute of Technology Delhi, India

Sparse Autoencoders Mechanistic Interpretability Causal Interventions in Language Models

Ask PDF

May 28, 2025

Kronecker Factorization Improves Efficiency and Interpretability of Sparse Autoencoders

Sparse Autoencoders (SAEs) decompose language-model activations into sparse, interpretable features, but standard encoders usually treat the latent dictionary as a flat set of independent coordinates, leaving hierarchy and feature interactions to emerge only implicitly. We propose KronSAE, a design that factorizes the latent space into heads and forms post-latent features as pairwise compositions of lower-dimensional pre-latents using mAND, a differentiable AND-like interaction. This imposes a compositional co-activation prior while remaining compatible with standard SAE objectives and variants such as TopK, Matryoshka, and Switch SAEs. KronSAE matches strong baselines on the EV-FLOPs frontier, improves interpretability of the latents, better captures the underlying correlated feature structure, and reduces encoder computational cost as an additional benefit. Code is available at https://github.com/corl-team/kronsae.

Vadim Kurochkin, Yaroslav Aksenov, Daniil Laptev +2

T-Tech

Representation Learning Activation Sparsity Hierarchical Representation Learning Sparse Autoencoders Mechanistic Interpretability Sparse Coding+1

Ask PDF

Abstract

Sparse Autoencoders

Explore similar work

What Do Sparse Autoencoder Feature Discovery Scores Identify? From Activations to Interventions

Sparse Autoencoders Encode Both Concepts and Functions: The Downstream Geometry of Feature Effects

Kronecker Factorization Improves Efficiency and Interpretability of Sparse Autoencoders