cs.LGNov 3, 2026

When Are Two SAE Latents the Same Feature?

Authors: Alexandra Suvorikova, Ilgam Latypov, Anton Trukhan, Yuriy Dorn

Organizations: Independent researcher · MSU Institute for Artificial Intelligence

Abstract

Comparing sparse autoencoders (SAEs) requires connecting feature correspondence, differences between whole dictionaries, and uncertainty from finite evaluation data. We develop a framework that connects these tasks through an energy-weighted optimal-transport plan under a specified similarity geometry. The same plan defines a dictionary-level discrepancy, decomposes it into latent contributions, and quantifies how each latent's mass is distributed across counterparts. We characterize relationships between decoder, activation, and first-order causal similarities; idealized models clarify the interpretation of splitting and absorption. For fixed trained SAEs, we derive joint asymptotic inference for transport costs and fractions, accounting for activation similarities and both energy marginals estimated from the same documents. A joint document bootstrap gives asymptotically valid simultaneous confidence bands. Experiments across four language models locate cross-seed discrepancy and separate context-dependent changes.

Figures & tables

Appendix figures & tables14 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. SharedSAE: One Feature Dictionary Across Language Models

    Sep 3, 2026Daniil Ognev, Célian Vasson, Lijie Hu +2LLM InterpretabilitySparse Autoencoders