Factor models are central to asset pricing and portfolio construction, but neural factors are often difficult to interpret, and their effect on portfolio risk is unclear. A sparse autoencoder (SAE) may address both concerns by describing daily returns across many assets with only a few active features. We ask whether this sparse representation produces stable financial factors and better-diversified portfolios. We study cryptocurrency returns, use equities as a secondary test, and compare the SAE with common benchmarks in a time-ordered evaluation. We find consistently lower volatility in SAE portfolios and link this result to lower correlations among the selected assets. However, the SAE does not outperform a simple contrarian strategy on risk-adjusted return, and its factor identities are not stable across training runs. Our contribution is to show that sparse coding can support portfolio diversification, while clearly separating this risk effect from claims about alpha or stable factor identities.
Figures & tables
Figure 1. Pipeline from the daily crypto return cross-section xt through a sparse code and reconstructed-return ranking to a contrarian quintile basket. Under strictly pre-fold selection, the chosen K=128 SAE reduces rank-scale long-short volatility by 23% on average relative to matched- K PCA ( 20 – 26% in all five folds). Pipeline from a daily cross-section of 433 cryptocurrency returns through a sparse code and a contrarian long-short basket. The final bars show lower confirmed rank-scale volatility for SAE than PCA.
Model
Return
Volatility
Sharpe
Naive
+47.9
7.7
+6.20
SAE- K=64
+13.9
4.2
+3.28
PCA- K=64
+18.1
6.8
+2.67
DenseAE- K=64
+16.3
6.4
+2.53
NMF- K=64
+14.1
7.0
+2.02
Table 1. Exploratory contrarian portfolios at matched K=64 on the crypto static test split. Rank-scale values support cross-method comparison only; all values are pre-cost. Bold marks the best factor model; PCA uses exact SVD.
Fold
SAE Vol.
PCA Vol.
DenseAE Vol.
Δρlf
0
6.6
9.7
10.2
−0.048
1
5.5
8.5
8.4
−0.032
2
5.2
7.3
7.3
−0.025
3
5.9
9.7
9.7
−0.039
4
4.5
7.4
7.1
−0.030
5
4.2
7.1
7.2
−0.030
Table 2. Exploratory K=64 ten-fold robustness analysis ( 994 -day initial training window, 99 -day test folds, rotating seeds). Volatility is annualised on the rank scale; Δρlf is the leak-free trailing 60 -day long-basket correlation gap (SAE − PCA).
Figure 2. Exploratory day- t -eligible raw-return results at matched K=64 . (a) SAE volatility reduction relative to PCA is positive in all five folds. (b) Pre-cost annualised Sharpe; the naive contrarian leads both reconstruction models. Panel a is a five-bar chart showing a positive SAE volatility reduction relative to PCA in every walk-forward fold. Panel b compares pre-cost raw-return Sharpe ratios for the naive, SAE, and PCA portfolios.
Figure 3. Exploratory K=64 diagnostics. (a) Mean within-basket pairwise correlations. (b) SAE–PCA volatility reduction across five universe definitions and ten folds; cells share underlying days. Panel a compares long- and short-basket correlations for SAE, DenseAE, and PCA, with SAE lowest on both sides. Panel b shows positive SAE volatility reduction in every fold under five universe definitions.
Rank-scale gap −2.91 ; day- t -eligible raw-return gap −2.48
not supported
Seed-reproducible identities
Matched cosine 0.13 compared with random 0.12 ; best match 0.23
not supported
Table 3. Summary of empirical findings. Strict confirmation applies only to the K=128 volatility result; the K=64 evidence is exploratory.
Figure 4. Factor stability and correlations. (a) Mean Hungarian-matched decoder cosine by fold; the shaded band is the random-dictionary 95% reference interval. (b) Descriptive equity-sample correlations with FF5+MOM ( max∣ρ∣=0.147 ), short of a joint spanning test. Panel a compares the mean matched decoder cosine in each fold with a narrow random-dictionary reference band; the observed values are only slightly above random. Panel b is a heat map of small positive and negative correlations between 64 equity-sample SAE factors and six FF5 plus momentum series.
Sparse autoencoders (SAEs) are widely used to interpret neural network representations, but their utility depends on whether the learned features are reproducible across training runs. We study this question through feature stability: for each SAE feature, we estimate the probability that a similar feature reappears in an independently trained SAE. This yields a scalable per-feature signal that separates stable from unstable features. In a large-scale study across seeds, models, layers, dictionary sizes, and SAE variants, we find a pronounced functional asymmetry: stable features carry most of the reconstruction- and prediction-relevant signal, while unstable features have weak marginal impact and are dominated by low-frequency surface-form triggers in both activation statistics and automatic explanations. Geometrically, unstable features are individually non-reproducible but concentrate in reproducible lower-rank subspaces, suggesting that seed dependence often reflects basis ambiguity within a shared region of activation space rather than pure noise. A controlled synthetic model makes this mechanism explicit, showing that low-rank ground-truth features can be recovered at the subspace level while remaining non-identifiable as individual SAE latents across seeds. Finally, by pooling unique cross-seed features, we construct more stable SAEs while preserving explained variance in this setting. Together, these results show that unstable features are not merely failed or noisy latents: they have weak individual functional impact, but reflect reproducible low-dimensional structure that standard SAEs resolve differently across seeds.
Sparse Autoencoders (SAEs) decompose language-model activations into sparse, interpretable features, but standard encoders usually treat the latent dictionary as a flat set of independent coordinates, leaving hierarchy and feature interactions to emerge only implicitly. We propose KronSAE, a design that factorizes the latent space into heads and forms post-latent features as pairwise compositions of lower-dimensional pre-latents using mAND, a differentiable AND-like interaction. This imposes a compositional co-activation prior while remaining compatible with standard SAE objectives and variants such as TopK, Matryoshka, and Switch SAEs. KronSAE matches strong baselines on the EV-FLOPs frontier, improves interpretability of the latents, better captures the underlying correlated feature structure, and reduces encoder computational cost as an additional benefit. Code is available at https://github.com/corl-team/kronsae.
Sparse autoencoders (SAEs) are typically trained on hundreds of millions to billions of activation tokens, yet little work has tested whether this scale is necessary. We study how much activation data SAEs need by training four SAE architectures on three language models with budgets ranging from 5M tokens to a 500M-token reference. We measure two notions of convergence: functional behavior on standard evaluations and one-to-one reproducibility of learned features. Functional data requirements are strongly metric-dependent: reconstruction often approaches the full-data reference at substantially smaller budgets, whereas feature reproducibility continues to improve with data. For Gemma-2-2B BatchTopK, the fraction of strictly matched features rises from 2% at 5M tokens to 29% at 50M. Data selection also matters: random sampling better preserves the original activation distribution and generally favors reconstruction, whereas diversity-based sampling frequently improves AutoInterp. A theoretical analysis separates statistical consistency of subsampled training from the stronger identifiability and optimization conditions required for one-to-one feature recovery. Our results show that SAE training can often be made far more data-efficient when budgets are matched to the goal, and feature-level claims are backed by explicit reproducibility checks, paving the way toward cheaper and more reliable SAE research.
Sambit Ghosh, Madelyn Mathai, R. Venkatesh Babu +1
IBM · Indian Institute of Science · University of Virginia