Finding Belief Geometries with Sparse Autoencoders

Matthew Levinson

cs.LGApr 3, 2026

Finding Belief Geometries with Sparse Autoencoders

Authors: Matthew Levinson

Organizations: Independent Researcher

Abstract

Understanding the geometric structure of internal representations is a central goal of mechanistic interpretability. Prior work has shown that transformers trained on sequences generated by hidden Markov models encode probabilistic belief states as simplex-shaped geometries in their residual stream, with vertices corresponding to latent generative states. Whether large language models trained on naturalistic text develop analogous geometric representations remains an open question. We introduce a pipeline for discovering candidate simplex-structured subspaces in transformer representations, combining sparse autoencoders (SAEs), $k$ -subspace clustering of SAE features, and simplex fitting using AANet. We validate the pipeline on a transformer trained on a multipartite hidden Markov model with known belief-state geometry. Applied to Gemma-2-9B, we identify 13 priority clusters exhibiting candidate simplex geometry ( $K \geq 3$ ). A key challenge is distinguishing genuine belief-state encoding from tiling artifacts: latents can span a simplex-shaped subspace without the mixture coordinates carrying predictive signal beyond any individual feature. We therefore adopt barycentric prediction as our primary discriminating test. Among the 13 priority clusters, 3 exhibit a highly significant advantage on near-vertex samples (Wilcoxon $p < 10^{-14}$ ) and 4 on simplex-interior samples. Together 5 distinct real clusters pass at least one split, while no null cluster passes either. One cluster, 768_596, additionally achieves the highest causal steering score in the dataset. This is the only case where passive prediction and active intervention converge. We present these findings as preliminary evidence that genuine belief-like geometry exists in Gemma-2-9B's representation space, and identify the structured evaluation that would be required to confirm this interpretation.

Figures & tables

Figure 1: Overview of the belief geometry discovery pipeline. Text sequences are processed through Gemma-2-9B, and residual-stream activations at layer 20 are encoded by a GemmaScope JumpReLU SAE. Decoder directions of SAE latents are clustered into candidate latent groups using $k$ -subspace clustering. Each candidate group is then fit with AANet to test for simplex structure and recover barycentric coordinates. The resulting candidate belief geometries are evaluated using barycentric predictive advantage and causal steering as the primary validation analyses.

Cluster	$m$	Semantic Confidence	Vertex interpretation (summary)
512_17	15	HIGH	Text register (instructional / promotional / metadata)
512_229	22	MEDIUM	Syntactic complexity (simple / formal-specific / complex-embedded)
512_261	18	MEDIUM	Grammatical function (content words / function words / technical register)
512_504	22	MEDIUM	Syntactic role (function words / content nouns / predicates)
768_140	26	MEDIUM	Referential specificity (generic / specific / anaphoric)
768_210	4	MEDIUM	Syntactic complexity (complex NP / simple / intermediate)

Table 1: The 13 priority clusters. Clusters are identified by the number of clusters for the

k

-subspace clustering (512 or 768) and their cluster index.

m

denotes the number of SAE latents in the cluster. Vertex consistency is labeled as consistent, mixed, or inconsistent. Confidence is derived from 20-iteration autointerp synthesis: HIGH

=\geq

2 consistent vertices; MEDIUM

=

no inconsistent vertices; LOW

=\geq

1 inconsistent vertex.

		Near-vertex (NV)		Simplex-interior (SI)
Cluster	$n$	Frac. wins	$p$	Frac. wins	$p$
512_17	15	0.00	1.00	0.04	1.00
512_22	6	0.00	1.00	0.04	1.00
512_67	25	0.00	1.00	0.00	1.00
512_181	26	1.00	$<10^{-15}$	1.00	$<10^{-15}$
512_229	22	1.00	$<10^{-15}$	0.58	$0.003$

Table 2: Barycentric vs. best-latent predictive advantage, both evaluation splits. Frac. bary wins : fraction of top-50 highest-variance tokens for which the barycentric coordinate achieves higher

R^{2}

than the best individual latent. Wilcoxon

p

-values test the one-sided hypothesis that barycentric

R^{2}

is stochastically greater. NV = near-vertex samples; SI = simplex-interior samples. No null cluster passes on NV. One null cluster passes SI; see text.

Cluster	Steering score	Best type	Best scale
768_596	0.419	type2	20
512_22	0.356	type1	20
768_210	0.336	type2	1
768_140	0.289	type1	20
512_67	0.250	type1	20
512_261	0.242	type1	1

Table 3: Causal steering scores. Null cluster scores overlap with those of most qualifying real clusters; 768_596 is the only cluster where a significant barycentric advantage and a positive steering score converge.

Figure 2: Cluster 768_596: per-latent centroid positions. Mean barycentric centroid of each of the six latents. The latents partition across the three vertices, consistent with vertex-specialized feature coding.

Appendix figures & tables11 assets

Supplementary material from the paper’s appendix.

Appendix

Cluster	Assigned component	$R^{2}$
0	tom_quantum	0.551
1	mess3_2	0.314
2	tom_quantum_1	0.577
3	mess3_1	0.719
4	mess3	0.893
5	(noise)	—

Appendix

Table 4: Toy model clustering recovery (

R^{2}

), layer 1. Each cluster is assigned to the generative component whose belief state it best predicts (linear regression, held-out data). All five components are recovered with no conflicts.

R^{2}

values are modest but uniformly positive, consistent with genuine but partial recovery under the high-entropy multipartite setting.

Panel of Figure 3 — Figure 3: Per-sub-component PCA projections, toy model layer 1. Each panel shows the PCA subspace that best reveals one sub-component’s geometry, colored by that component’s true discrete output token. Every panel also exhibits clear separation for at least one other sub-component, demonstrating cross-component entanglement.

Panel of Figure 4 — Figure 4: Two representative latents from Cluster 4 (multipartite toy model, layer 1, TopK $K=12$ ). Each panel shows KDE-smoothed activation density over all five component belief geometries. The assigned component is marked $\star$ ( $R^{2}=0.89$ ). Top: Latent 4 fires near the top vertex of each Mess3 simplex and at the centre of the Tom Quantum disks. Bottom: Latent 26 fires near the base vertices and along the annular rim of the Tom Quantum disks. The two latents tile complementary regions of the same geometry, demonstrating geometry-consistent partitioning of belief-state information.

Figure 5: Cluster 512_181: barycentric vs. best-latent $R^{2}$ . Barycentric coordinates (mean $R^{2}=0.612$ ) outperform the best individual latent (mean $R^{2}=0.539$ ) for every one of the 50 tokens (Wilcoxon $p<10^{-15}$ ).

Figure 6: Cluster 512_229: barycentric vs. best-latent $R^{2}$ . Barycentric coordinates (mean $R^{2}=0.378$ ) outperform the best individual latent (mean $R^{2}=0.244$ ) for all 50 tokens (Wilcoxon $p<10^{-15}$ ), with a $\sim$ 55% relative improvement.

Figure 7: Cluster 768_596: barycentric vs. best-latent $R^{2}$ . Barycentric coordinates (mean $R^{2}=0.269$ ) outperform the best individual latent (mean $R^{2}=0.217$ ) for 98% of tokens (Wilcoxon $p<10^{-14}$ ). This cluster also has the highest causal steering score (0.419).

Cluster	$m$	KL ratio	Bary wins	Steering	Interp.
512_17	15	1.054	0.00	0.118	HIGH
512_22	6	1.058	0.00	0.356	LOW
512_67	25	1.229	0.00	0.250	LOW
512_181	26	1.037	1.00 (sig.)	—	LOW
512_229	22	1.166	1.00 (sig.)	—	MED
512_261	18	1.032	0.00	0.242	MED

Appendix

Table 5: Full results across all four validation strands. KL ratio: cross/same symmetric KL divergence ratio. Bary wins: fraction of top-50 tokens where

R^{2}_{\mathrm{bary}}>R^{2}_{\mathrm{best\,latent}}

(“sig.” if Wilcoxon

p<0.001

). Steering: best-combination score (“—” if cluster did not qualify). Interp.: frontier-LLM vertex label consistency (HIGH/MED/LOW).

Latent	Neuronpedia interp (GPT-4o-mini)	Active at (pie%)
L2149	Occurrences of research methodologies and results in scientific texts	V1 (19%)
L2318	Instances of error logging and debugging messages in code	—
L2759	Programming constructs and function calls associated with database management	—
L3510	Mathematical expressions related to derivatives	—
L3954	Mathematical concepts related to derivatives and calculus	—
L4321	References to privilege and inequality in social contexts	V1 (73%)

Appendix

Table 13

Latent	Neuronpedia interp (GPT-4o-mini)	Active at (pie%)
L449	Special characters, particularly underscores and certain punctuation marks	—
L1234	Various forms of punctuation and mathematical symbols	—
L1367	References to academic papers and their citation formats	—
L1462	Quotes or dialogue from characters or people	V0 (5%)
L1575	References to experimental findings and numerical data	—
L1993	Punctuations and metadata references within the text	—

Appendix

Table 14

Latent	Neuronpedia interp (GPT-4o-mini)	Active at (pie%)
L4269	Code-related syntax and structures	—
L7398	Conversational expressions and phrases	V1 (2%), V2 (100%)
L9729	Punctuation marks and their contexts within sentences	—
L12076	Questions related to religious practices and their origins	V1 (2%)
L13939	References to non-profit organizations and charitable donations	V1 (97%)
L14895	Code-related terms and programming concepts	V0 (99%)

Appendix

Table 15

Latent	Neuronpedia interp (GPT-4o-mini)	Active at (pie%)
L1189	Snippets of mathematical or programming expressions and logical statements	—
L1277	References to individuals in positions of authority or influence	—
L1425	References to historical events and their impacts on society	—
L1481	References to educational institutions and their operational details	V1 (16%)
L1540	Words indicative of financial or economic conditions and events	V1 (2%)
L1818	Punctuation and markup elements in text	—

Appendix

Table 16

Representation Geometry in Language Models

Representation geometry in language models studies how embeddings and hidden states organize concepts, computation, and behavior.

See all 417 papers

SMIXAE: Towards Unsupervised Manifold Discovery in Language Models
May 9, 2026Collin FrancelAutoencoders LLM Interpretability
Department of Arts and Sciences, University of Alabama, Tuscaloosa, AL, United States.
Trajectory Geometry of Transformer Representations Across Layers
Jun 8, 2026Vishal Pandey, Gopal Singh, Yacine MahdidTransformer Interpretability Neural Representation Geometry
London, UK · Athens, GR · Montreal, CA
Fully Interpretable Minimal Transformers: From Geometry to Algorithm
Oct 7, 2026Raneem Mahajne, Toviah MoldwinTransformer Interpretability Mechanistic Interpretability
Edmond and Lily Safra Center for Brain Sciences, The Hebrew University of Jerusalem

May 9, 2026J/K move · Enter open · S save

SMIXAE: Towards Unsupervised Manifold Discovery in Language Models

Collin Francel

Department of Arts and Sciences, University of Alabama, Tuscaloosa, AL, United States.

Autoencoders LLM Interpretability Sparse Autoencoders Manifold Learning

Open paper AskPDF arXiv

Sparse autoencoders (SAEs) have been used widely to decompose and interpret neural network activations, especially those of transformer language models. One key issue with SAEs is their inability to directly model multidimensional features. Instead, SAEs may tile such features by a set of independent directions that must be grouped together after the SAE training phase, impeding discoverability and interpretation of learned feature representations. We begin to address this issue by introducing the Sparse MIXture of Autoencoders (SMIXAE) architecture. Empirically, we provide evidence that SMIXAE models have success both in directly learning previously identified manifold structures, as well as finding novel structures, within the open source Gemma 2 2B and 9B models. Finally, we discuss several limitations and point towards areas for future work.

Abstract

Representation Geometry in Language Models

Explore similar work

SMIXAE: Towards Unsupervised Manifold Discovery in Language Models

Trajectory Geometry of Transformer Representations Across Layers

Fully Interpretable Minimal Transformers: From Geometry to Algorithm