We introduce sparse autoencoder neural operators (SAE-NOs), a new class of sparse autoencoders that operate in function spaces rather than fixed-dimensional Euclidean representations. We formalize the functional representation hypothesis, where data are explained through sparse compositions of structured functions. Unlike standard SAEs that represent concepts with scalar activations, SAE-NOs parameterize concepts as functions, enabling representations that capture not only a concept's presence, but also how and where it is expressed across the input domain. We achieve this through joint sparsity: concept sparsity selects active concepts, while domain sparsity governs where they are expressed. We instantiate this framework using Fourier neural operators (SAE-FNOs), parameterizing concepts as integral operators in the Fourier domain. This functional and spectral parameterization is particularly advantageous when data exhibit spatial structure across scales or when concepts are frequency-structured. We characterize SAE-FNO on vision data and demonstrate that it learns localized patterns, uses concepts more efficiently, and exhibits stable concept characteristics across sparsity levels. We further show that SAE-FNO adapts to changes in domain size and generalizes across discretizations, operating at resolutions beyond those seen during training, where standard SAEs fail. We also introduce lifting into SAEs and show theoretically and empirically that it acts as a preconditioner that accelerates optimization. Overall, our results show that moving from vector-valued to functional parameterizations, with concept and domain sparsity, extends SAEs from representing concept presence to modeling structured concept expression, highlighting the importance of parameterization.
Figures & tables
Figure 1: Sparse Autoencoder Neural Operator (SAE-NO). (a) Conventional SAEs (SAE-MLP) parameterize concepts D in the Euclidean domain, assigning each concept a scalar activation indicating presence and strength. (b) SAE-FNOs use functional concept parameterization via Fourier integral operators, encoding concept sparsity and domain sparsity to capture how and where concepts are expressed. SAE-FNO treats inputs as samples from an underlying continuous function and supports querying of representations, reconstructions, and concepts at varying resolutions.
Figure 2: Encoding where and how concepts are expressed. (a) A digit translates across the domain over frames. (b) Top-10 concepts over time: SAE-MLP switches concepts as the digit moves, whereas SAE-FNO maintains stable concepts whose spatial activations shift. (c) Higher Intersection over Union (IoU) indicates stronger concept stability and consistency across time. SAE-FNO (blue) maintains a highly stable set of active concepts, whereas SAE-MLP (red) rapidly swaps its active concepts between frames. (d) Spatial feature-map trajectories of the top-3 SAE-FNO concepts follow the ground-truth motion; color denotes concept and brightness denotes time. (e) SAE-FNO exhibits stable concept characteristics across different sparsity levels. In contrast, SAE-MLP and SAE-CNN show significant changes in concept structure as sparsity varies, indicating undesirable sensitivity.
Figure 3: Concept recovery under known generative model. (a) SAE-FNO recovers the underlying concepts, with dictionary recovery error converging to zero. (b) Lifting to a higher-dimensional space acts as a preconditioner, accelerating recovery. (c) SAE-FNO outperforms SAE-CNN in recovering smooth concepts. Each point corresponds to a training run.
Figure 4: Structural stability, correlation of concepts, and expressivity. (a) Fraction of matched features across runs (identity consistency) of increasing concept sparsity shows that SAE-FNO learns concepts with stable structural characteristics while SAE-MLP’s concept structure changes as k increases. (b) SAE-FNO overcomes orthogonality bias and recovers more correlated concepts. We compare the distribution of pairwise cosine similarities between learned dictionary atoms across varying k. (c) Expressivity of SAE, R2 , as a function of dictionary size p .
Figure 5: Learned concepts across the dataset . Each panel visualizes the most utilized concepts (each square is one concept) for a given concept sparsity level k . Fewer displayed concepts indicate more efficient concept reuse across the data population. SAEs are trained on CIFAR-10 images ( 8×8 ) with TopK and p=1000 . (a) SAE-FNO exhibits stable concept characteristics across sparsity levels due to domain sparsity. As sparsity decreases, more concepts become active while preserving similar structure. (b) SAE-MLP concepts vary significantly with sparsity, losing structure and failing to capture locality. See also Figure 18 for 32×32 inputs.
Figure 6: Concept utilization and expressivity . First row: 8×8 , second row: 32×32 for CIFAR10 and TopK. (a,b,e,f) ℓ0 code statistics (y-axis) across the data population, plotted against concept indices sorted by decreasing utilization (x-axis). SAE-FNO shows a sharp decay, indicating efficient concept usage, while SAE-MLP exhibits a flatter distribution, requiring more concepts, an effect that worsens for larger patches. (c,g) R2 vs. k . (d,h) R2 vs. the number of used concepts; SAE-FNO achieves higher expressivity with fewer concepts, whereas SAE-MLP fails to scale to larger domains.
Appendix figures & tables36 assets
Supplementary material from the paper’s appendix.
Appendix
Setting
Symbol
Meaning
Continuous (FRH)
Z,X
Function (Banach) spaces of codes / data
GD∗:Z→X
Ground-truth generative operator
Hθ:X→Z
Approximate inverse (encoder) operator
Dc∈Rh
Spatial-domain kernel for concept c
Discretized
x∈Rm
Discretized data sample
zc∈Rd
Discretized code / feature map for concept c
Appendix
Table 1: Notation across continuous, discretized, Fourier-domain, and lifted spaces.
r
(k,s)
Final cosine similarity
0.20
(1, 2)
0.89±0.08
0.30
(3, 1)
0.94±0.03
0.40
(2, 2)
0.89±0.07
0.59
(3, 2)
0.89±0.12
0.99
(5, 2)
0.82±0.14
1.19
(3, 4)
0.75±0.13
Appendix
Table 2: Stripe-sparsity identifiability. Recovery of a ground-truth dictionary of p=10 single-frequency kernels ( h=99 , m=1000 ), reported as final cosine similarity to the ground-truth dictionary (mean ± std, 5 seeds; initial cosine similarity 0.33±0.02 throughout), as a function of the expected number of active coefficients per receptive field r=ksh/m , for concept sparsity k and domain sparsity s .
Figure 7: Top concept activations over time for higher concept sparsity levels. Extended visualizations of Figure 2 b for (a) k=25 and (b) k=50 . Each cell’s color represents the concept ID active at a given time step. Consistent with lower sparsity levels, SAE-FNO maintains stable concept identities over time (horizontal bands), using domain sparsity to handle spatial translations. In contrast, SAE-MLP constantly activates new, distinct concepts to represent the moving digit.
Figure 8: SAE-FNO concept stability across sparsity levels. Intersection over Union (IoU) of the active concept sets between consecutive frames in translating MNIST digits (see Figure 2 (a)-(d)). At each time step t , we compute the IoU between the top k concepts (ranked by their spatial L2 norm) activated in frame t and those activated in frame t+1 , averaged across batch. Higher IoU indicates stronger concept stability and consistency, meaning the model tracks the moving digit using a consistent subset of concepts. While SAE-MLP (red) rapidly swaps its active concepts between frames, SAE-FNO (blue) maintains a highly stable set of active concepts ( IoU≈1 ).
Figure 9: Lifting. Lifted SAE is equivalent to SAE when learning with LTL=I .
Figure 10: Learning the lifting operator L accelerates model recovery. Comparing SAE-MLP with L-SAE-MLP.
Figure 11: Lifted concepts visualized in the input domain. CIFAR TopK-SAE-FNO (modes = 6, spatially 3-sparse).
Figure 12: Lifting accelerates learning. Lifting acts as a preconditioner introducing an implicit acceleration to learning. In real-world settings, this can result in a better fit to the data by reducing the impact of noise on learned concepts. The impact of lifting applies to both SAE-FNO and SAE-MLP.
Figure 13: Stability of concept characteristics across concept sparsity levels (MNIST). Distribution of Hoyer scores of learned SAE-MLP and SAE-FNO (spatial sparsity 0.007) concepts across concept sparsity levels k (densities normalized to unit area). (a) SAE-FNO’s distribution is stable across k for full and truncated modes. (b) SAE-MLP’s distribution shifts drastically with k .
Figure 14: Stability of concept characteristics across Fourier modes and domain sparsity levels (CIFAR-10). Distribution of spatial Hoyer scores across concept sparsity levels k . (a) SAE-FNO with varying Fourier modes and fixed domain sparsity (3-sparse). (b) SAE-FNO with varying domain sparsity and 6 Fourier modes. (c) SAE-MLP for comparison. SAE-FNO maintains stable concept characteristics across k , whereas SAE-MLP shows larger changes in its Hoyer score distributions.
Figure 15: Stability of concept characteristics of SAE-CNN depends on kernel size. The distribution of SAE-CNN atoms becomes more stable as kernel size decreases, consistent with the stronger locality bias imposed by smaller receptive fields.
Model
Var(mean)
Var(median)
Var(mode)
R2 (max)
SAE-FNO (24 modes)
6.40
1.04
0.43
0.795
SAE-FNO (full, 32 modes)
1.61
0.03
0.03
0.945
SAE-CNN ( 5×5 )
7.11
6.66
4.04
0.602
SAE-CNN ( 10×10 )
18.9
35.5
58.8
0.678
SAE-CNN ( 15×15 )
26.3
69.7
91.5
0.710
SAE-CNN ( 20×20 )
6.55
71.0
64.6
0.845
Appendix
Table 3: SAE-FNO has higher structural stability than SAE-CNN as k changes. Variance of the mean, median, and mode of the Hoyer-score distribution ( ×10−4 ), together with the best R2 across k , on CIFAR 32×32 (domain sparsity of 5 pixels). Lower variance indicates concept structure that is more stable as k varies.
Figure 16: Identity consistency across concept sparsity levels. Fraction of active concepts from an anchor model ( k=50 ) that can be matched to concepts learned at larger k , using a similarity threshold of 0.7. Results are shown for TopK, BatchTopK, and Matryoshka SAE variants. SAE-FNO preserves a larger fraction of concepts across sparsity levels than SAE-MLP.
Figure 17: SAE-FNO overcomes orthogonality bias and recovers a broader range of correlated concepts. Distribution of pairwise cosine similarities between learned SAE-FNO (blue) and SAE-MLP (red) dictionary atoms across concept sparsity levels. SAE-FNO displays a broader correlation distribution with heavier tails compared to SAE-MLP.
Figure 18: Learned concepts across sparsity levels on CIFAR 32×32 images . Most frequently used concepts learned by (a) SAE-FNO-TopK and (b) SAE-MLP-TopK, with p=1000 and k=25,50,200 from left to right. Each square represents one concept. Fewer displayed concepts indicate more efficient usage (see R2 vs. used atoms in Figure 6 ). (a) SAE-FNO learns localized frequency structures, consistent with natural image statistics [ 49 ] . SAE-FNO concept characteristics remain stable across sparsity levels, enabled by domain sparsity and adaptability to input size. As sparsity decreases, more concepts are activated. (b) SAE-MLP concepts change more substantially with sparsity, losing structure and failing to capture locality. SAE-FNO requires fewer effective concepts than SAE-MLP, indicating superior expressivity for structured, large-domain inputs.
Figure 19: Learned concepts across the dataset for CIFAR 8×8 patches for SAE-FNO (modes = 6) as domain sparsity changes. From left to right, concept sparsity is k=25,50,200 . Same visualization setup as Figure 18 . For concept utilization and efficiency, see Figure 26 a.
Figure 20: Learned concepts across the dataset for CIFAR 8×8 patches for SAE-FNO (spatial sparsity = 0.05) as the number of Fourier modes changes. From left to right, concept sparsity is k=25,50,200 . Same visualization setup as Figure 18 . Corresponding concept utilization and efficiency plots are shown in Figure 26 b.
Figure 21: SAE-FNO learns concepts with stable structural properties across concept sparsity levels, whereas SAE-MLP concepts lose structure as sparsity decreases. Learned concepts on CIFAR-10 8×8 patches, with k=10,100,700,1000 (left to right).
Figure 22: Supporting visualization demonstrating instability of SAE-MLP learned concept structure. ImageNet patches. Left to right: k=25,100,200,500 .
Figure 23: Supporting visualization demonstrating stability of SAE-FNO learned concept structure. SAE-FNO (modes = 12, spatially 5-sparse) on ImageNet patches. Left to right: k=25,100,200,500 .
Figure 24: Progressive reconstruction from active concepts across SAE variants. SAE-FNO (left) and SAE-MLP (right) are compared across concept sparsity levels. Within each panel, TopK, BatchTopK, and Matryoshka results are shown from top to bottom. For each variant, the three rows (from top to bottom) display spatial sparse codes, individual learned concepts, and cumulative reconstructions obtained by successively adding the top i active concepts ( i=1,…,20 ). The leftmost images show the ground-truth input and final reconstruction.
Figure 25: Concept utilization and efficiency on ImageNet patches ( 16×16 ). Normalized ℓ0 -norm of codes corresponding to each dictionary concept, sorted by decreasing utilization. The point at which the y-axis reaches zero corresponds to the effective total number of concepts used to explain the dataset. Across TopK, BatchTopK, and Matryoshka variants, SAE-FNO requires a much smaller active dictionary to represent the data compared to SAE-MLP.
Figure 26: Concept utilization and efficiency on CIFAR 8×8 patches for SAE-FNO-TopK across changing (a) spatial sparsity and (b) number of Fourier modes. Same visualization setup as Figure 25 . Visualization of learned concepts is shown in Figure 19 (change spatial sparsity) and Figure 20 (change modes).
Figure 27: Concept utilization and efficiency on CIFAR 32×32 for SAE-CNN (spatially 5-sparse) across changing kernel sizes. Same visualization setup as Figure 25 .
Figure 28: Expressivity across concept sparsity for SAE-CNN kernel sizes. R2 on CIFAR 32×32 patches as a function of (a) concept sparsity k and (b) number of used concepts. See Figure 6 (g-h) for SAE-FNO and SAE-MLP.
Figure 29: Expressivity of SAE-FNO as a function of Fourier modes and domain sparsity. (a) The R2 of SAE-FNO on CIFAR patches steadily increases with the number of Fourier modes. The frequency bandwidth directly controls the resolution and detail of the learned concepts. (b) SAE-FNO (6 modes) on CIFAR patches. Domain sparsity matters when k is small, where raising domain sparsity compensates for fewer active concepts. For large k , the curves collapse and R2 values stay within 0.005.
Figure 30: SAE-FNO vs. SAE as a function of input discretization . Models trained at 28×28 resolution are evaluated at higher resolutions (upsampled images). Reconstruction MSE is shown. Transparent lines represent individual runs, and solid lines represent their average. SAE-FNOs successfully infer the underlying representations that allow them to reconstruct data across resolutions.
Figure 31: Discretization Generalization of SAE-FNO. The underlying data structure has five convolutional concepts along with a 5 -sparse code (i.e., 1 -sparse representation for each concept). Each subplot shows (left) the input data and its reconstruction, and (right) the support sign (active sparse codes). We evaluate the ability of SAE-FNO (left column) and SAE-CNN (right column) to infer sparse representations and reconstruct data across varying discretization levels. Each row represents a different upsampling rate ( 1× to 4× applied to the input signal). For each panel, the left plot shows the spatial-domain signal reconstruction (red) against the input to the network (blue), and the right plot shows the inference of 1-sparse code supports across 5 kernels. (a-b) Original resolution ( 1× ): Baseline performance at training resolution. Both models accurately reconstruct the signal and recover the correct support. (c-f) Higher upsampling rates ( 2× , 4× ): As spatial resolution increases, SAE-FNO maintains accurate reconstruction and code support inference. In contrast, SAE-CNN fails to generalize; its support inference collapses, leading to poor reconstruction. SAE-FNO’s function-space parameterizations enables this generalization across discretizations.
1 × (length 1000)
2 × (500)
4 × (250)
5 × (200)
No LPF
0.975±0.03
0.952±0.04
0.910±0.05
0.845±0.07
With LPF
0.976±0.03
0.981±0.02
0.985±0.01
0.987±0.01
Appendix
Table 4: Effect of discretization on kernel recovery. Cosine similarity to the ground-truth kernel under decimation, with and without an anti-aliasing low-pass filter applied before decimation (setting of Figure 3(c) , 5 seeds, initial cosine similarity 0.671±0.079 ). Recovery degrades under decimation only in the unfiltered case. With aliasing removed, recovery does not degrade at all, identifying the cause as sampling rather than a limitation of the SAE-FNO parameterization.
Figure 32: High-frequency noise robustness. PSNR as a function of high-frequency noise strength on ImageNet 16×16 patches, with p=1000 and varying concept sparsity. SAE-FNO shows smaller degradation under distribution shift compared to SAE-MLP, indicating greater robustness.
Table 5: Reconstruction under pixel permutation. Reporting R2 .
Figure 33: Ablation of concept and domain sparsity. (a) Concepts learned by AE-FNO without either sparsification mechanism. (b) Concepts learned by SAE-FNO with concept sparsity but without domain sparsity (from left to right: k=25,50,200 ). Both fail to produce the localized, structured features observed with the full SAE-FNO. This confirms that the observed gains in mechanistic interpretability are specifically from the combination of functional sparse coding and domain sparsity, rather than the FNO architecture alone.
Init
SAE-MLP
SAE-FNO ( m= full)
SAE-FNO ( m=151 )
Clean
0.329 (0.210)
0.868 (0.303)
0.863 (0.324)
0.882 (0.612)
Noisy
0.329 (0.210)
0.798 (0.534)
0.853 (0.627)
0.890 (0.658)
Appendix
Table 6: Useful inductive bias of SAE-FNO in recovering frequency-oriented concepts. Synthetic experiment recovering 10 single-frequency concepts ( k=3 , domain sparsity 2). Values represent cross-correlation recovery scores, reported as mean (standard deviation) across 5 runs.
Table 7: Computational cost as a function of (a) data dimension and (b) model size p . Table reports training and inference throughput (images/sec, L40S GPU, batch size 256, 32 workers).
Table 8: Sensitivity of computational cost to (a) the number of Fourier modes and (b) domain sparsity. Training and inference throughput and memory (L40S GPU, batch size 256, 32 workers).
Model
Modes
p
Train (samp/s)
Train Mem (GB)
Infer (samp/s)
Infer Mem (GB)
MLP
–
10K
3,667
0.48
21,609
0.32
FNO
64
10K
286
8.58
556
8.59
FNO
2048
10K
200
9.03
439
9.18
MLP
–
16K
2,228
0.76
20,945
0.51
FNO
64
16K
181
13.72
353
13.73
FNO
2048
16K
126
14.43
279
14.67
Appendix
Table 9: Computational cost at SAE-deployment scale. Training and inference throughput and memory (L40S GPU, batch size 256, 32 workers) at a scale representative of current SAE deployments ( m=2048 , matching Gemma-2-2B activation dimensionality [ 92 ] ), across dictionary sizes p and modes. Data is synthetic. SAE-FNO remains feasible at this scale, with throughput and memory scaling predictably in mode count.
We propose a unified mathematical framework for a geometric understanding of concept learning and neuron interpretation in sparse autoencoders (SAEs). While SAEs improve interpretability of neural networks by learning sparse feature representations, a principled definition of ''concept'' and ''learning'' remains unclear. We formalize concepts as sets of data points and cast concept learning as a set-alignment problem between human-defined and model-induced concepts. This formulation distinguishes three increasingly strong notions of learning -- detection, separation, and approximation -- and yields geometric conditions, error bounds, and capacity constraints for when concepts can be represented by individual neurons or multi-neuron units. It also provides a set-theoretic account for common SAE phenomena, including feature splitting, feature absorption, feature families, and hierarchical concepts. Finally, we connect concept learning and neuron interpretation through formal concept analysis, showing that the two directions need not agree and that their many-to-many structure can be organized by concept lattices. Experiments on synthetic data with ReLU and Top-K SAEs illustrate the theory and reveal the effects of SAE size and sparsity on concept learning.
Chenhao Zhang, Chris Lin, Su-In Lee
Paul G. Allen School of Computer Science & Engineering, University of Washington, Seattle, USA.
Recently, sparse autoencoders (SAEs) have emerged as a promising technique for interpreting activations in foundation models by disentangling features into a sparse set of concepts. However, identifying the optimal level of sparsity for each neuron remains challenging in practice: excessive sparsity can lead to poor reconstruction, whereas insufficient sparsity may harm interpretability. While existing activation functions such as ReLU and TopK provide certain sparsity guarantees, they typically require additional sparsity regularization or cherry-picked hyperparameters. We show in this paper that dynamically sparse attention mechanisms using sparsemax can bridge this trade-off, due to their ability to determine the activation numbers in a data-dependent manner. Specifically, we first explore a new class of SAEs based on the cross-attention architecture with the latent features as queries and the learnable dictionary as the key and value matrices. To encourage sparse pattern learning, we employ a sparsemax-based attention strategy that automatically infers a sparse set of elements according to the complexity of each neuron, resulting in a more flexible and general activation function. Through comprehensive evaluation and visualization, we show that our approach successfully achieves lower reconstruction loss while producing high-quality concepts, particularly in top-n classification tasks.
Dongsheng Wang, Jinsen Zhang, Dawei Su +1
College of Computer Science and Software Engineering, Shenzhen University, China
Sparse autoencoders (SAEs) are widely used to extract interpretable features from neural network representations, often under the implicit assumption that concepts correspond to independent linear directions. However, a growing body of evidence suggests that many concepts are instead organized along low-dimensional manifolds encoding continuous geometric relationships. This raises three basic questions: what does it mean for an SAE to capture a manifold, when do existing SAE architectures do so, and how? We develop a theoretical framework that answers these questions and show that SAEs can capture manifolds in two fundamentally different ways: globally, by allocating a compact group of atoms whose linear span contains the entire manifold, or locally, by distributing it across features that each selectively tile a restricted region of the underlying geometry. Empirically, we find that SAEs suboptimally recover continuous structures, mixing the global subspace and local tiling solutions in a fragmented regime we call dilution. This explains why manifold structure is rarely visible at the level of individual concepts and motivates post-hoc unsupervised discovery methods that search for coherent groups of atoms rather than isolated directions. More broadly, our results suggest that future representation learning methods should treat geometric objects, not just individual directions, as the basic units of interpretability.
Usha Bhalla, Thomas Fel, Can Rager +9
aHarvard University · bNortheastern University · cTechnion IIT +1