TaxoSAE: Rethinking Sparse Autoencoders through Taxonomic Logic

Xinyu Guo; Minglai Yang; Clayton T. Morrison; Eduardo Blanco; Mihai Surdeanu

cs.CLOct 22, 2026

TaxoSAE: Rethinking Sparse Autoencoders through Taxonomic Logic

Authors: Xinyu Guo, Minglai Yang, Clayton T. Morrison, Eduardo Blanco, Mihai Surdeanu

Organizations: University of Arizona · Scale AI

Abstract

Recent Sparse Autoencoders (SAEs) decompose the representations of large language models (LLMs) into interpretable features, enabling precise readout and control of encoded knowledge. However, existing SAEs treat each feature as a standalone unit, whereas real-world knowledge is often taxonomically organized. As a result, readout carries no guarantee of taxonomic consistency, and control lacks the granularity to specify the scope of an edit. We propose TaxoSAE, which explicitly encodes taxonomical information, built on is-a (hypernym-hyponym) relations, into the SAE feature space. TaxoSAE mirrors a taxonomy with a stack of SAEs, one per taxonomy level, and introduces a taxonomic alignment loss that trains each concept to be active whenever one of its hyponyms is. On two factual QA taxonomies, one over biographical relations and one over animals, TaxoSAE enables (1) more accurate readout through the taxonomy with concentrated activations, e.g., +31.3 points in reading out the hypernyms of abstract concepts never queried in training; (2) more precise control, with +25.5 points in swap success, while suppressing a concept removes only its own answers (0-2% accuracy vs. 91-100% on other concepts); and, more importantly, (3) readout of unseen concepts through their hypernyms, with up to +37.9 points in hypernym readout accuracy.

Figures & tables

Figure 1: A taxonomy: each edge $(a,b)$ is an is-a link from hypernym $a$ to hyponym $b$ ; dashed edges mark further levels and nodes. Gray labels: a $K=3$ example from the Animal taxonomy (full taxonomy in App. A ). $\mathcal{T}$ axoSAE uses one SAE per level $L_{k}$ (Fig. 2 ).

Figure 2: Overview of $\mathcal{T}$ axoSAE. We train stacked SAEs on a frozen LLM’s last-token hidden state. (1) Stacked SAEs: one SAE per taxonomy level ( $L_{1,\dots,K}$ in Fig. 1 ), each mapping the reconstruction of the level below onto its hypernyms. (2) Taxonomic alignment loss: each taxonomic slot is trained to fire when the input involves its node or any of the node’s hyponyms. Complement slots bind the complementary element; reconstruction, sparsity, and orthogonality losses complete the objective. Figure 8 shows the full slot layout of each level on both taxonomies.

Bio relations (layer 6)
	Diagonal Acc (%) $\uparrow$						EffFeat $\downarrow$			Top1C (%) $\uparrow$			Swap (%) $\uparrow$
SAE	$L_{1}$ ${\to}$ $L_{1}$	$L_{1}$ ${\to}$ $L_{2}$	$L_{1}$ ${\to}$ $L_{3}$	$L_{2}$ ${\to}$ $L_{2}$	$L_{2}$ ${\to}$ $L_{3}$	$L_{3}$ ${\to}$ $L_{3}$	$L_{1}$	$L_{2}$	$L_{3}$	$L_{1}$	$L_{2}$	$L_{3}$	$L_{1}$	$L_{2}$
Random Unsup. SAE	4.3	4.2	3.9	6.7	3.0	1.4	360.8	298.0	283.3	3.5	3.5	3.1	2.2	0.0
DiffMean Unsup. SAE	77.9	82.1	75.4	61.4	60.7	63.7	360.8	298.0	283.3	3.5	3.5	3.1	81.8	12.2
Hybrid SAE	100.0	71.0	77.1	54.9	54.5	69.2	3.2	188.2	184.9	74.8	4.4	3.5	84.7	16.3
$\mathcal{T}$ axoSAE	100.0	100.0	100.0	69.8	92.0	87.8	3.2	1.5	1.1	74.8	91.9	99.1	84.7	41.8

Table 1: Results on Bio and Animal.

L_{1}

L_{2}

L_{3}

: terminal/intermediate/top levels; L

q{\to}

r

: level-

r

readout on level-

q

questions (

L_{1}{\to}L_{r}

pools ID and OOD). Bold/underline: best/second-best;

\Delta

\mathcal{T}

axoSAE minus Random Unsup. SAE.

\mathcal{T}

axoSAE achieves the best readout (Diagonal Acc) and control (Swap) on both datasets. Layer choice: App. L . The same conclusions hold on Qwen2.5-0.5B (App. B ).

Figure 3: t-SNE of

L_{2}

activations on Animal, colored by

L_{2}

node: squares are trained animals, triangles unseen ones. A triangle inside its color’s contour is placed under the right hypernym.

\mathcal{T}

axoSAE does this for most unseen animals; DiffMean’s regions overlap. Mollusc is omitted due to heavy overlap in the baseline ( App. C ).

Figure 4: Normalized activations ( $L_{1}$ / $L_{2}$ ) and binding matrices ( $L_{1}$ – $L_{3}$ ) on Bio. Rows group questions by their terminal node and its hypernyms. Wide panels show activations over all units: the unsupervised SAE spreads each concept over many features, while $\mathcal{T}$ axoSAE concentrates it in bind slots. Square panels show each concept’s assigned unit; a diagonal means perfect binding, and “acc.” is the binding accuracy.

Figure 5: Activation fragmentation (EffFeat) and concentration (Top1C) per taxonomy level on Animal and Bio; dashed lines mark the ideal single unit.

\mathcal{T}

axoSAE activations are the most concentrated.

	$L_{1}$ swap				$L_{2}$ swap
	Answer		Read		Answer	Read
SAE	Strict	In-set	$L_{2}$	$L_{3}$	Target-only	$L_{3}$
Random Unsup.	2.2	4.0	18.0	34.3	0.0	31.0
DiffMean Unsup.	81.8	83.0	71.1	55.9	12.2	69.5
Hybrid	84.7	88.1	66.2	44.6	16.3	65.8
$\mathcal{T}$ axoSAE	84.7	88.1	100.0	93.2	41.8	100.0

Table 2: Bio swaps at each system’s best strength. Answer: target-value accuracy; Read: target-hypernym readout. Underlines: best baselines (excluding Hybrid for

L_{1}

, sharing our

L_{1}

SAE);

\uparrow

: gains over them.

Figure 6: Selectivity on Animal (top) and Bio (bottom): row $i$ deactivates element $i$ ’s unit; column $j$ is the exact match on element $j$ . Ideally the diagonal is white and the rest dark, as for $\mathcal{T}$ axoSAE; DiffMean also lightens off-diagonal cells, especially at $L_{2}$ . Own ( $\downarrow$ )/Other ( $\uparrow$ ): mean accuracy on target/non-target elements. Labeled version: Fig. 12 .

Figure 7: Ablation on Bio (upper right is better). Removing taxonomic alignment collapses both metrics to random; removing others mainly lowers swap success.

Appendix figures & tables23 assets

Supplementary material from the paper’s appendix.

Appendix

	Bio (relation taxonomy)	Animal (entity taxonomy)
Nodes at $L_{1}$ / $L_{2}$ / $L_{3}$	18 / 6 / 3	18 / 6 / 2
Complementary elements	20 people	20 properties
Terminal facts	$20\times 18=360$	$18\times 20=360$
SAE training questions	14,400 (10 templates, 80 name forms)	25,014 (54 templates, 108 animal synonyms)
Readout, $L_{1}$ questions	800 in-distribution / 800 new template / 800 new name / 720 both new	1,080 in-distribution / 720 new template / 720 new synonym / 720 both new
Readout, $L_{2}$ questions	240 training templates / 240 new templates / 800 extra templates	600 seen templates / 240 new templates / 600 new synonyms / 240 both new / 1,440 extra surface forms

Appendix

Table 3: Dataset statistics. Only terminal-level questions are used to train

\mathcal{T}

axoSAE; intermediate- and top-level questions appear only in the readout tests.

Figure 8: Slot layout of the three stacked SAEs on Animal (top) and Bio (bottom). Each level has one taxonomic slot per node at that level, one complement slot per complementary element, and 10,000 free slots;

\mathrm{SAE}_{k+1}

reads the reconstruction of

\mathrm{SAE}_{k}

. Dark cells mark the slots that the example question should activate: its terminal node and that node’s hypernyms in the taxonomic slots, and its complementary element at every level.

Bio relations , Qwen2.5-0.5B (layer 12)
	Diagonal Acc (%) $\uparrow$						EffFeat $\downarrow$			Top1C (%) $\uparrow$			Swap (%) $\uparrow$
SAE	$L_{1}$ ${\to}$ $L_{1}$	$L_{1}$ ${\to}$ $L_{2}$	$L_{1}$ ${\to}$ $L_{3}$	$L_{2}$ ${\to}$ $L_{2}$	$L_{2}$ ${\to}$ $L_{3}$	$L_{3}$ ${\to}$ $L_{3}$	$L_{1}$	$L_{2}$	$L_{3}$	$L_{1}$	$L_{2}$	$L_{3}$	$L_{1}$	$L_{2}$
Random Unsup. SAE	1.7	0.0	0.0	0.0	0.0	0.0	138.6	135.9	132.5	6.4	7.5	6.8	1.1	0.3
DiffMean Unsup. SAE	51.5	73.5	60.6	48.9	56.2	55.8	138.6	135.9	132.5	6.4	7.5	6.8	44.1	4.3
Hybrid SAE	100.0	63.6	66.2	52.1	46.5	49.2	3.4	123.4	101.3	71.9	4.6	8.1	60.5	4.0
$\mathcal{T}$ axoSAE	100.0	100.0	100.0	67.7	88.9	82.5	3.4	1.3	1.0	71.9	95.3	99.3	60.5	13.7

Appendix

Table 4: Results on Bio with Qwen2.5-0.5B (layer 12), in the format of Table 1 . Swap:

L_{1}

Strict accuracy over all pairs and

L_{2}

Target-only coverage@

|U|

over pairs from different top-level nodes, each system at its best

\alpha

. Hybrid SAE shares the

L_{1}

module of

\mathcal{T}

axoSAE, so its

L_{1}

columns are identical. Bold/underline: best/second-best;

\Delta

\mathcal{T}

axoSAE minus Random Unsup. SAE. The conclusions of Table 1 hold on Qwen.

	Diagonal Acc (%) $\uparrow$			Swap (%) $\uparrow$
Layer	$L_{1}$	$L_{2}$	$L_{3}$	$L_{1}$	$L_{2}$
$L2$	95.1	76.3	74.3	3.9	0.3
$L4$	96.4	71.7	71.3	10.1	1.0
$L6$	100.0	74.4	69.5	26.2	0.5
$L8$	99.9	75.9	82.5	49.6	7.5
$L10$	100.0	72.2	67.7	50.3	9.2

Appendix

Table 5: Layer-wise performance of

\mathcal{T}

axoSAE on Bio with Qwen2.5-0.5B (24 layers), in the format of Table 19 . Shading marks layer 12, used in Table 4 . Readout is stable across layers, and control is best at layers 16–18.

$L_{2}$ node	Unseen terminal concepts
Animal (24 animals, 100 questions each)
mammal	pig, dog, horse, lion
bird	sparrow, owl, parrot, pigeon
amphibian	newt, axolotl, caecilian, mudpuppy
fish	tuna, trout, goldfish, cod
insect	ant, butterfly, beetle, mosquito

Appendix

Table 6: Unseen terminal concepts, four per intermediate node. None of them appears in SAE training; each unseen concept is compared with 100 questions of each trained terminal concept.

Animal (layer 9)
	$L_{1}$ read		Upper levels read (%) $\uparrow$
SAE	Top score	Abstain (%)	$L_{2}$	$L_{3}$	$L_{2}$ \| $L_{1}$ off
DiffMean Unsup. SAE	0.5 $\times$	27.6	26.0	56.8	28.2
Hybrid SAE	0.5 $\times$	12.9	16.7	50.5	1.4
$\mathcal{T}$ axoSAE	0.5 $\times$	12.9	63.9	76.1	31.1
Chance	–	–	16.7	50.0	–

Appendix

Table 7: Readout of unseen terminal concepts (Table 6 ). Top score: median of the highest

L_{1}

score, relative to the trained concepts.

L_{2}

L_{1}

off:

L_{2}

accuracy on questions whose

L_{1}

read falls in another intermediate node. Hybrid SAE shares the

L_{1}

module of

\mathcal{T}

axoSAE. The upper levels of

\mathcal{T}

axoSAE place most unseen concepts under the right intermediate and top-level node on both datasets.

Figure 9: Bio counterpart of Fig. 3 : t-SNE of the

L_{2}

readout scores for seen and unseen terminal relations. The extended-family node is omitted for readability; Table 7 includes it.

Figure 10: Diagonal accuracy on terminal-level questions read at

L_{1}

–

L_{3}

, with seen templates (ID, top) and unseen rephrasings or synonyms (OOD, bottom).

\mathcal{T}

axoSAE is best at every level in both settings.

Bio (20 people)
SAE	In-dist.	New template	New name/syn.	Both new
Random Unsup. SAE	3.7	3.6	4.2	3.5
DiffMean Unsup. SAE	40.6	39.0	39.6	38.9
$\mathcal{T}$ axoSAE	100.0	100.0	100.0	99.3
Animal (20 properties)
Random Unsup. SAE	4.2	4.9	4.4	4.7

Appendix

Table 8: Complement readout (Diagonal Acc, %) on terminal-level questions, by question tier: in-distribution, new templates, new names (Bio) or synonyms (Animal), and both new.

	(A) Diagonal Acc (%) $\uparrow$						(D) EffFeat $\downarrow$			(D) Top1C (%) $\uparrow$
Variant	$L_{1}{\to}L_{1}$	$L_{1}{\to}L_{2}$	$L_{1}{\to}L_{3}$	$L_{2}{\to}L_{2}$	$L_{2}{\to}L_{3}$	$L_{3}{\to}L_{3}$	$L_{1}$	$L_{2}$	$L_{3}$	$L_{1}$	$L_{2}$	$L_{3}$
$\mathcal{T}$ axoSAE	100.0	100.0	100.0	69.8	92.0	87.8	3.21	1.47	1.06	74.8	91.9	99.1
w/o taxonomic alignment	0.0	5.4	0.0	9.8	4.0	4.7	4.32	3.11	2.40	26.8	58.6	60.5
w/o complement alignment	100.0	100.0	100.0	67.6	85.1	80.7	3.56	1.60	1.18	70.2	89.0	95.8
w/o value loss	100.0	100.0	100.0	68.1	76.9	78.3	4.02	1.70	1.28	64.9	86.5	93.5
w/o taxonomic–complement ortho.	100.0	100.0	100.0	69.0	88.9	88.0	2.84	1.41	1.05	79.6	93.0	99.3

Appendix

Table 9: Loss ablation on Bio (layer 6): taxonomy readout and activation concentration. Each variant retrains Stage 2 of

\mathcal{T}

axoSAE from the same Stage 1 checkpoint with one loss component removed.

L_{1}

L_{2}

L_{3}

denote terminal/intermediate/top levels. (A) Diagonal Acc:

L_{q}{\to}L_{r}

denotes readout of level-

r

taxonomic elements from level-

q

questions. (D) EffFeat and Top1C measure activation concentration. Bold marks the best value in each column. Removing taxonomic alignment collapses readout; the other losses change it far less.

Variant	$\alpha$	Strict (%) $\uparrow$	In-set (%) $\uparrow$	$\alpha$	Cov. (%) $\uparrow$	Exact (%) $\uparrow$	$L_{1}$ Own $\downarrow$	$L_{1}$ Other $\uparrow$	$L_{2}$ Own $\downarrow$	$L_{2}$ Other $\uparrow$
	(B) $L_{1}$ swap			(C) $L_{2}$ swap			(E) Selectivity (%)
$\mathcal{T}$ axoSAE	100	84.7	88.1	100	41.8	41.8	0.0	100.0	0.0	100.0
w/o taxonomic alignment	200	1.3	2.4	20	2.2	2.2	77.8	77.8	75.0	75.7
w/o complement alignment	100	60.1	63.7	100	42.0	42.0	0.8	100.0	0.0	100.0
w/o value loss	100	82.0	82.8	200	15.2	15.2	0.0	100.0	0.8	99.6
w/o taxonomic–complement ortho.	100	80.6	86.5	100	37.0	37.0	4.7	100.0	0.0	100.0

Appendix

Table 10: Loss ablation on Bio (layer 6): steering and selectivity for the same variants as Table 9 . Each swap test uses the variant’s best steering strength

\alpha

. (B)

L_{1}

swap: strict and in-target-set accuracy. (C)

L_{2}

swap on pairs from different top-level nodes: Target-only coverage@

|U|

and Changed exact-set@

|U|

. (E) Selectivity: Own (

\downarrow

) / Other (

\uparrow

) exact match when the taxonomic slot corresponding to one

L_{1}

L_{2}

element is ablated. Bold marks the best value in each column. Removing any loss lowers at least one swap score, while selectivity is preserved except without taxonomic alignment.

Bio (layer 6)
	Diagonal Acc (%) $\uparrow$			Swap (%) $\uparrow$		EffFeat $\downarrow$
Value head	$L_{2}{\to}L_{2}$	$L_{2}{\to}L_{3}$	$L_{3}{\to}L_{3}$	$L_{1}$	$L_{2}$	$L_{2}$
Bilinear ( $\mathcal{T}$ axoSAE)	69.8	92.0	87.8	84.7	41.8	1.47
Per-slot	69.5	91.5	87.2	77.1	38.3	1.47
Animal (layer 9)
Bilinear ( $\mathcal{T}$ axoSAE)	47.9	86.6	50.6	43.6	27.6	1.68

Appendix

Table 11: Value head ablation. Per-slot: one head per taxonomic slot, reading only that slot, as in AlignSAE. Swap: terminal-level Strict accuracy and intermediate-level Target-only coverage@

|U|

on pairs from different top-level nodes, each at the best

\alpha

. Terminal-level readout (

L_{1}{\to}L_{1}/L_{2}/L_{3}

) is identical for the two heads on Bio (100%) and within 0.2 points on Animal.

Figure 11: Slot layout of the flat AlignSAE baseline on Bio: a single SAE whose taxonomic slots cover all three levels (

L_{1}

L_{2}

L_{3}

: 18/6/3), followed by complement and free slots. Dark cells mark the slots that the example question should activate. Compare the stacked layout of

\mathcal{T}

axoSAE in Fig. 8 .

	(A) Diagonal Acc (%) $\uparrow$			(B) $L_{1}$ swap $\uparrow$		(C) $L_{2}$ swap $\uparrow$
SAE	$L_{2}{\to}L_{2}$	$L_{2}{\to}L_{3}$	$L_{3}{\to}L_{3}$	Strict	$L_{2}$ read	Cov.	$L_{3}$ read
Flat AlignSAE	69.3	89.0	91.7	5.5	0.0	18.3	0.0
$\mathcal{T}$ axoSAE	69.8	92.0	87.8	84.7	100.0	41.8	100.0

Appendix

Table 12: Flat AlignSAE vs.

\mathcal{T}

axoSAE on Bio (layer 6). Both read terminal-level questions perfectly at every level (

L_{1}{\to}L_{1}/L_{2}/L_{3}

, not shown). Swaps use each model’s best

\alpha

(flat: 20 for

L_{1}

, 50 for

L_{2}

;

\mathcal{T}

axoSAE: 100 for both).

L_{2}

read: the SAE reads the target’s intermediate node after a terminal swap across intermediate nodes;

L_{3}

read: it reads the target’s top-level node after an intermediate swap. For the flat SAE, both reads use the edited code itself, which has no higher level to pass the edit to. Readout is similar, but only

\mathcal{T}

axoSAE keeps the levels consistent after an edit.

	$L_{1}$ swap (all pairs)					$L_{2}$ swap (different top-level nodes)
		Answer (%) $\uparrow$		Read after edit (%) $\uparrow$			Answer (%) $\uparrow$	Read after edit (%) $\uparrow$
SAE	$\alpha$	Strict	In-set	$L_{2}$	$L_{3}$	$\alpha$	Target-only	$L_{3}$
Bio ( $L_{1}$ : 2,550 pairs; $L_{2}$ : 600 pairs)
Random Unsup. SAE	–	2.2	4.0	18.0	34.3	–	0.0	31.0
DiffMean Unsup. SAE	200	81.8	83.0	71.1	55.9	200	12.2	69.5
Hybrid SAE	100	84.7	88.1	66.2	44.6	200	16.3	65.8

Appendix

Table 13: Swap results with the levels the SAE reads after the edit, each system at its best edit strength

\alpha

L_{1}

swap over all source–target pairs: Strict, the answer is the target terminal node’s value; In-set, any value the target terminal node holds; Read after edit, the SAE reads the target’s intermediate (

L_{2}

) and top-level (

L_{3}

) hypernyms.

L_{2}

swap on pairs from different top-level nodes: Target-only coverage@

|U|

, and whether the SAE reads the target intermediate node’s top-level hypernym (

L_{3}

) after the edit. Hybrid SAE shares the

L_{1}

module of

\mathcal{T}

axoSAE, so its

L_{1}

answers are identical, but it reads

L_{2}

and

L_{3}

with its own unsupervised SAEs; Random Unsup. SAE is the mean over four fixed assignments. After every swap,

\mathcal{T}

axoSAE reads the target’s hypernyms in 93.2–100% of cases, against at most 71.1% for the baselines.

Figure 12: Fig. 6 with concept labels. Row $i$ deactivates the unit of element $i$ ; column $j$ is the exact match on element $j$ .

Bio: training data	#
Terminal-level factual QA	14,400
Hypernymy QA	159
terminal ${\to}$ intermediate	90
terminal ${\to}$ top-level	36
intermediate ${\to}$ top-level	24
hyponym enumeration	9

Appendix

Table 14: Base-model training data for Bio. Mapping questions include yes/no checks (e.g., “Is ‘birth city’ a place of birth fact?”); hyponym enumeration asks for the members of an intermediate or top-level node. Biographies are plain-text descriptions of terminal-level facts.

Animal: training data	#
Terminal-level factual QA	25,014
Hypernymy QA	1,126
terminal ${\to}$ intermediate	864
terminal ${\to}$ top-level	162
intermediate ${\to}$ top-level	66
hyponym enumeration	34

Appendix

Table 15: Base-model training data for Animal. Mapping questions include yes/no checks (e.g., “Is a cow a mammal?”); hyponym enumeration asks for the members of an intermediate or top-level node (e.g., “Which animals are mammals?”). Descriptions state terminal-level facts in plain text.

Bio: fine-tuning
Base model	GPT-2 small
Training steps	13k
biographies only	first 1k
biography share afterwards	15%
Batch size	64
Optimizer	AdamW ( $\epsilon{=}10^{-6}$ )

Appendix

Table 16: Base-model fine-tuning for Bio. GPT-2 small has 12 transformer layers and a hidden dimension of 768. The loss is computed on answer tokens for QA and on all tokens for biographies. Activations are the hidden state at the final prompt token.

Animal: fine-tuning
Base model	GPT-2 small
Training steps	30k
descriptions	mixed into the data
Batch size	32
Optimizer	AdamW ( $\epsilon{=}10^{-6}$ )
Learning rate	$2{\times}10^{-5}$

Appendix

Table 17: Base-model fine-tuning for Animal. GPT-2 small has 12 transformer layers and a hidden dimension of 768. The loss is computed on answer tokens for QA and on all tokens for descriptions. Activations are the hidden state at the final prompt token.

Loss term	Stage 1	Stage 2
Reconstruction	1	1
Sparsity, free slots	$10^{-3}$	$10^{-3}$
Sparsity, taxonomic slots	$10^{-3}$	$10^{-3}$
Sparsity, complement slots	$10^{-3}$	$10^{-3}$
Taxonomic alignment, terminal (CE)	0	1.0
Taxonomic alignment, non-terminal (CE)	0	5.0

Appendix

Table 18: Loss weights of

\mathcal{T}

axoSAE. Both datasets ( Bio and Animal ) use the same weights. Stage 2 is initialized from the Stage 1 checkpoint.

	Diagonal Acc (%) $\uparrow$			Swap (%) $\uparrow$		$\downarrow$
Layer	$L_{1}$	$L_{2}$	$L_{3}$	$L_{1}$	$L_{2}$	Recon NMSE
$L0$	5.3	21.3	35.5	2.9	0.3	0.372
$L1$	90.1	72.1	98.0	0.9	0.0	0.310
$L2$	100.0	69.0	79.8	11.4	0.3	0.252
$L3$	99.6	70.4	85.8	47.5	14.0	0.129
$L4$	100.0	71.8	89.0	65.5	35.2	0.082

Appendix

Table 19: Layer-wise performance of

\mathcal{T}

axoSAE on Bio. Layer 0 is the embedding output. Bold marks column-best results; shading marks layer 6, which achieves the highest mean swap score. Steering emerges only from the middle layers.

	Diagonal Acc (%) $\uparrow$			Swap (%) $\uparrow$		$\downarrow$
Layer	$L_{1}$	$L_{2}$	$L_{3}$	$L_{1}$	$L_{2}$	Recon NMSE
$L0$	3.8	17.9	50.0	5.1	5.2	0.967
$L1$	69.0	27.1	47.1	4.3	8.6	2.605
$L2$	76.4	46.2	50.4	4.1	6.2	1.775
$L3$	76.2	41.7	51.1	5.0	7.9	0.879
$L4$	75.3	36.7	50.7	6.0	7.6	0.824

Appendix

Table 20: Layer-wise performance of

\mathcal{T}

axoSAE on Animal, in the format of Table 19 . Layer 0 is the embedding output. Bold marks column-best results; shading marks layer 9, used in the main experiments. Steering emerges only from the middle layers.

Sparse Autoencoders

Sparse autoencoders decompose neural network activations into sparse features to study, interpret, and control learned representations.

See all 220 papers

Dec 1, 2025

AlignSAE: Concept-Aligned Sparse Autoencoders

Large Language Models (LLMs) encode factual knowledge within hidden parametric spaces that are difficult to inspect or control. While Sparse Autoencoders (SAEs) can decompose hidden activations into more fine-grained, interpretable features, they often struggle to reliably align these features with human-defined concepts, resulting in entangled and distributed feature representations. To address this, we introduce AlignSAE, a method that aligns SAE features with a predefined ontology through a "pre-train, then post-train" curriculum. After an initial unsupervised training phase, we apply supervised post-training to bind specific concepts to dedicated latent slots while preserving the remaining capacity for general reconstruction. This separation creates an interpretable interface where specific concepts can be inspected and controlled without interference from unrelated features. Empirical results on GPT-2 across synthetic biographical QA (1-hop, 6 concepts) and compositional relational reasoning (2-hop, 20 concepts) demonstrate that AlignSAE enables precise causal interventions, such as reliable concept swaps (94.8% success in 1-hop), by targeting single, semantically aligned slots. AlignSAE achieves disentanglement improvements of

9

\infty\times

over standard SAEs (measured via Mutual Information Gap), and further supports multi-hop reasoning and a mechanistic probe of grokking-like generalization dynamics. Although our evaluation focuses on GPT-2, AlignSAE itself is model-agnostic.

Minglai Yang, Xinyu Guo, Zhengliang Shi +4

University of Arizona · Shandong University · LMU Munich +1

Sparse Autoencoders LLM Interpretability Disentangled Representation Learning

Ask PDF

Aug 22, 2025

Sparse but Wrong: Incorrect L0 Leads to Incorrect Features in Sparse Autoencoders

Sparse Autoencoders (SAEs) extract features from LLM internal activations, meant to correspond to interpretable concepts. A core SAE training hyperparameter is L0: how many SAE features should fire per token on average. Existing work compares SAE algorithms using sparsity-reconstruction tradeoff plots, implying L0 is a free parameter with no inherently correct value aside from its effect on reconstruction. In this work we study the effect of L0 on SAEs, and show that if L0 is not set correctly, the SAE fails to disentangle the underlying features of the LLM. If L0 is too low, the SAE will mix correlated features to improve reconstruction. If L0 is too high, the SAE finds degenerate solutions that also mix features. Further, we present a proxy metric that can help guide the search for the correct L0 for an SAE on a given training distribution. We show that our method finds the correct L0 in toy models and coincides with peak sparse probing performance in LLM SAEs. We find that most commonly used SAEs have an L0 that is too low. Our work shows that practitioners must set L0 correctly to train SAEs with monosemantic features.

David Chanin, Adrià Garriga-Alonso

University College London · MATS Research

Activation Sparsity LLM Interpretability Representation Disentanglement Sparse Autoencoders Sparse Coding Hyperparameter Optimization

Ask PDF

Apr 26, 2026

Domain-Filtered Knowledge Graphs from Sparse Autoencoder Features

Sparse autoencoders (SAEs) extract millions of interpretable features from a language model, but flat feature inventories aren't very useful on their own. Domain concepts get mixed with generic and weakly grounded features, while related ideas are scattered across many units, and there's no way to understand relationships between features. We address this by first constructing a strict domain-specific concept universe from a large SAE inventory using contrastive activations and a multi-stage filtering process. Next, we build two aligned graph views on the filtered set: a co-occurrence graph for corpus-level conceptual structure, organized at multiple levels of granularity, and a transcoder-based mechanism graph that links source-layer and target-layer features through sparse latent pathways. Automated edge labeling then turns these graph views into readable knowledge graphs rather than unlabeled layouts. In a case study on a biology textbook, these graphs recover coherent chapter and subchapter-level structure, reveal concepts that bridge neighboring topics, and transform messy sentence-level activity containing thousands of features into compact, readable views that illustrate the model's local activity. Taken together, this reframes a flat SAE inventory as an internal knowledge graph that converts feature-level interpretability into a global map of model knowledge and enables audits of reasoning faithfulness.

John Winnicki, Abeynaya Gnanasekaran, Eric Darve

Institute for Computational and Mathematical Engineering, Stanford University, Stanford, CA, USA

KG Construction LLM Interpretability Sparse Autoencoders Mechanistic Interpretability Knowledge Representation Knowledge Graphs

Ask PDF

Abstract

Sparse Autoencoders

Explore similar work

AlignSAE: Concept-Aligned Sparse Autoencoders

Sparse but Wrong: Incorrect L0 Leads to Incorrect Features in Sparse Autoencoders

Domain-Filtered Knowledge Graphs from Sparse Autoencoder Features