ScopeSAE: Model-Scope Feature Discovery with Interpretable Layer Selection

Qingwen Zeng; Zehao Fu; Shuyu Meng; Linghan Huang; Jiayi Zhang; Chenglin Wu; Ling Chen; Huaming Chen

cs.LGOct 4, 2026

ScopeSAE: Model-Scope Feature Discovery with Interpretable Layer Selection

Authors: Qingwen Zeng, Zehao Fu, Shuyu Meng, Linghan Huang, Jiayi Zhang, Chenglin Wu, Ling Chen, Huaming Chen

Organizations: The University of Sydney · The Hong Kong University of Science and Technology (Guangzhou) · DeepWisdom · University of Technology Sydney

Abstract

Sparse autoencoders (SAEs) are a central tool in mechanistic interpretability. However, existing SAEs are primarily trained per layer. The modeling subspace is therefore fixed by layer identity, independent of which token-layer states actually drive each prediction. We argue that this constraint contributes to several limitations observed in layer-wise SAEs, including low feature utilization, high dictionary redundancy, and features that lack direct behavioral grounding. In this paper, we propose ScopeSAE, which selects the modeling subspace per token by attributing each prediction to its most influential token-layer state via normalized gradient-based attribution, and learns features over the resulting prediction-relevant subspace. Empirically, ScopeSAE yields an effect we term reconstruction-better-than-original. Written-back reconstructions of the SAE produce lower next-token cross-entropy than the original activations, an outcome that, to our knowledge, has not previously been reported for SAEs. Through interventional analyses and a KL fine-tuning counter-experiment, we show that this effect is attributable to ScopeSAE's prediction-relevant subspace itself rather than to architectural changes. ScopeSAE further improves effective feature count, interpretability, utilization, and dictionary redundancy over existing layer-based baselines, suggesting that choosing the SAE modeling subspace by predictive relevance leads to more useful and behaviorally meaningful features.

Figures & tables

Figure 1: Overview of the ScopeSAE framework. For each token, normalized gradient-based attribution scores identify the most prediction-relevant token-layer state, over which a sparse autoencoder is trained. At inference, the reconstruction is written back only to the selected position, enabling behaviorally grounded feature discovery while remaining compatible with standard SAE evaluation and intervention pipelines.

	Chance	Residual	Original	Reconstruction
Mean	0.0176	0.0103	0.0329	0.0420

Table 1: Mean absolute cosine similarity

|\cos|

between four vector types and the prediction gradient, averaged over all tokens and candidate layers (L3–L11). Chance is the expected alignment for random vectors in

\mathbb{R}^{d}

	$R_{\parallel}$	$R_{\perp}$
Median	1.2808	0.8735

Table 2: Median retention rates of the activation components parallel and perpendicular to the prediction gradient, computed over all tokens and candidate layers.

	SAE Recon	Recon + $r_{\parallel}$	Recon + $r_{\perp}$
CE	3.778	3.667	4.108
DeltaCE	0	-0.111	+0.330

Table 3: Cross-entropy of SAE reconstruction, reconstruction plus parallel residual, and reconstruction plus perpendicular residual.

	KL Div	DeltaCE	Recovered
ScopeSAE	$495$	$-0.20$	$1.03$
KL Fine-tuning	$98$	$+0.07$	$0.99$

Table 4: Effect of KL fine-tuning on ScopeSAE across three metrics vs. the standard checkpoint.

Appendix figures & tables10 assets

Supplementary material from the paper’s appendix.

Appendix

Metric	ScopeSAE	TopK	RouteSAE	ReLU	Gated
PPL ratio	1.60x	4.34x	1.98x	1.67x	1.67x
Entropy Change	1.73x	1.75x	1.99x	2.64x	2.65x
Cosine Similarity	2.08x	2.15x	2.52x	3.04x	3.04x
Top-1 Preservation Rate	1.38x	1.41x	1.59x	1.93x	1.93x

Appendix

Table 8: Degradation ratios of four side-effect metrics when intervention strength is doubled from

10

20

. Each entry is the ratio of the metric’s value at strength

20

to its value at strength

10

; smaller ratios indicate that side effects scale more gracefully with intervention strength. ScopeSAE attains the smallest ratio on every metric.

Type	Shallow (L3–L5)	Middle (L6–L8)	Deep (L9–L11)
A: Word / word-family bound	$55.1\%$	$10.1\%$	$34.8\%$
B: Semantic concept across words	$74.7\%$	$1.5\%$	$23.8\%$
C: Contextual / discourse pattern	$75.7\%$	$22.3\%$	$2.0\%$
D: Token-level / syntactic / noise	$25.5\%$	$34.4\%$	$40.1\%$

Appendix

Table 9: Distribution of selected layers within each feature type, aggregated over the top-activating tokens of all features in that type.

Layer	RouteSAE	Full-state ablation	Activation patching	ScopeSAE (ours)
L3	$99.40$	$100.00$	$17.54$	$15.44$
L4	$0.06$	$0.00$	$9.24$	$11.73$
L5	$0.20$	$0.00$	$8.38$	$9.41$
L6	$0.01$	$0.00$	$3.64$	$8.94$
L7	$0.01$	$0.00$	$5.94$	$8.37$
L8	$0.00$	$0.00$	$9.88$	$8.83$

Appendix

Table 10: Layer selection distribution (%) under four selection mechanisms.

Selection rule	DeltaCE	Recovered	DictCosSim	Feat. Util.	Feat. Count
ScopeSAE (first-order)	$-0.197$	$1.035$	$0.0202$	$99.8$	$5319$
Activation patching	$-0.117$	$1.018$	$0.0206$	$99.7$	$5040$
Full-state ablation	$+0.107$	$0.986$	$0.0224$	$97.6$	$3353$

Appendix

Table 11: Behavioral and reconstruction metrics for SAEs trained end to end under each selection rule, at

k{=}64

. Full-state ablation and Activation patching replace ScopeSAE’s first-order score with an exact ablation-based one; all other components are held fixed.

Method	NormMSE	KL	DeltaCE	Recovered	Feat. Count	Interp Score
ScopeSAE (ours)	$0.243$	$\mathbf{495.18}$	$\mathbf{-0.20}$	$\mathbf{1.035}$	$5{,}319$	$\mathbf{4.44}$
ScopeSAE-Top3-LCB	$\mathbf{0.239}$	$522.52$	$-0.005$	$1.002$	$\mathbf{5{,}429}$	$4.31$

Appendix

Table 12: Comparison of ScopeSAE with the Top3-LCB selection variant at

k{=}64

$k$	$\Delta\text{CE}$ (within-subspace)	Marginal gain over $k{=}1$
$1$	$-0.111$	—
$2$	$-0.113$	$-0.002$
$4$	$-0.128$	$-0.017$

Appendix

Table 13: Effect of increasing the prediction-subspace dimensionality

k

. Marginal gain is computed relative to

k{=}1

Selection target	DeltaCE	Recovered	DictCosSim	Util(%)	#Feat
Next token	$-0.197\pm 0.009$	$1.035\pm 0.0052$	$0.0202\pm 0.00007$	$99.8\pm 0.08$	$5319\pm 46$
Predicted token	$-0.206\pm 0.011$	$1.038\pm 0.0071$	$0.0205\pm 0.00008$	$99.8\pm 0.08$	$5281\pm 53$

Appendix

Table 17: ScopeSAE under next-token and predicted-token layer selection. Entries are mean

\pm

std over three training seeds.

Subset	Next-token selection	Predicted-token selection
$\hat{y}=y$	$-0.438$	$-0.469$
$\hat{y}\neq y$	$+0.144$	$+0.167$

Appendix

Table 18: DeltaCE on tokens where the base model predicts correctly (

\hat{y}=y

) and incorrectly (

\hat{y}\neq y

), under next-token and predicted-token selection.

Layer	DeltaCE	Recovered	DictCosSim	Util(%)	#Feat
L0	$+0.078$	$0.986$	$0.0223$	$98.1$	$2815$
L1	$+0.099$	$0.987$	$0.0225$	$97.5$	$3013$
L2	$+0.075$	$0.983$	$0.0224$	$97.8$	$2904$
L3	$+0.070$	$0.989$	$0.0223$	$97.6$	$3074$
L4	$+0.076$	$0.988$	$0.0226$	$96.5$	$3108$
L5	$+0.083$	$0.986$	$0.0227$	$95.1$	$3093$

Appendix

Table 19: Each independently trained per-layer TopK SAE, evaluated at its own layer, across the full depth range.

Method	DeltaCE	Recovered	DictCosSim	Util(%)	#Feat
Best single-layer TopK SAE	$+0.070$	$0.989$	$0.0223$	$97.6$	$3074$
Selection rule + per-layer TopK SAEs	$+0.081$	$0.987$	$0.0217$	$95.9$	$2931$
ScopeSAE	$-0.197$	$1.035$	$0.0202$	$99.8$	$5319$

Appendix

Table 20: The selection rule applied over independently trained per-layer TopK SAEs, compared against the best single such SAE and against ScopeSAE.

Transformer Interpretability

Transformer interpretability studies how attention, feed-forward layers, and hidden representations support model predictions and behavior.

See all 336 papers

Are Single-Token Sparse Autoencoder Features Causally Necessary? Layer-Depth and SAE-Family Effects
Jul 22, 2026Seonglae Cho, Zekun Wu, Kleyton Da Costa +3LLM Interpretability Sparse Autoencoders
Holistic AI · University College London
Subspace-Aware Sparse Autoencoders for Effective Mechanistic Interpretability
Jun 4, 2026Seyed Arshan Dalili, Mehrdad MahdaviRepresentation Disentanglement Sparse Autoencoders
The Pennsylvania State University
From Input to Output: A Flexible Agent for Dual-End Interpretation of Sparse Autoencoder Features
Sep 28, 2026Dewen Liu, Zixuan Li, Jonathan Pan +4Sparse Autoencoders Mechanistic Interpretability
Tsinghua University · Fudan University · University of Edinburgh

Jul 22, 2026J/K move · Enter open · S save

Are Single-Token Sparse Autoencoder Features Causally Necessary? Layer-Depth and SAE-Family Effects

Seonglae Cho, Zekun Wu, Kleyton Da Costa, Rishi Kalra+2

Holistic AI · University College London

LLM Interpretability Sparse Autoencoders Causal Interpretability Causal Interventions in Language Models

Open paper AskPDF arXiv

Sparse autoencoder (SAE) features are used to interpret and steer large language models, yet whether a feature's causal role is stable across SAE families remains untested. Single-token features that activate on one vocabulary item provide the diagnostic case where ground truth permits direct comparison. We analyze 3.9M features across six models and three SAE families using zero-ablation at full layer depth. Single-token features cluster 4.7x tighter in decoder space and concentrate in early layers (Layer 0 in GPT2-Small; L0-L4 in Gemma). Ablating them yields Benjamini-Hochberg-significant logit reductions in 178 of 208 full-layer conditions, with depth controlling whether damage cascades downstream or shapes the output directly. Cross-family causal differences exceed within-family scale effects: on the same base model, GemmaScope and BatchTopK features remain causally anchored, while LlamaScope features are locally redundant. The target token's rank recovers to within 2x baseline 96-98% of the time after the same ablation, and a controlled activation-function comparison reverses sign within the same model, leaving training recipe as the residual candidate. Cross-family interpretability claims are therefore sensitive to training methodology, not just activation function or scale.

Abstract

Transformer Interpretability

Explore similar work

Are Single-Token Sparse Autoencoder Features Causally Necessary? Layer-Depth and SAE-Family Effects

Subspace-Aware Sparse Autoencoders for Effective Mechanistic Interpretability

From Input to Output: A Flexible Agent for Dual-End Interpretation of Sparse Autoencoder Features