Representation learning models exhibit a surprising stability in their internal representations. Whereas most prior work treats this stability as a single property, we formalize it as two distinct concepts: statistical identifiability (consistency of representations across runs) and structural identifiability (alignment of representations with some unobserved ground truth). Recognizing that perfect pointwise identifiability is generally unrealistic for modern representation learning models, we propose new model-agnostic definitions of statistical and structural near-identifiability of representations up to some error tolerance ε. Leveraging these definitions, we prove a statistical ε-near-identifiability result for the representations of models with nonlinear decoders, generalizing existing identifiability theory beyond last-layer representations in e.g. generative pre-trained transformers (GPTs) to near-identifiability of the intermediate representations of a broad class of models including (masked) autoencoders (MAEs) and supervised learners. Although these weaker assumptions confer weaker identifiability, we show that independent components analysis (ICA) can resolve much of the remaining linear ambiguity for this class of models, and validate and measure our near-identifiability claims empirically. With additional assumptions on the data-generating process, statistical identifiability extends to structural identifiability, yielding a simple and practical recipe for disentanglement: ICA post-processing of latent representations. On synthetic benchmarks, this approach achieves state-of-the-art disentanglement using a vanilla autoencoder. With a foundation model-scale MAE for cell microscopy, it disentangles biological variation from technical batch effects, substantially improving downstream generalization.
Figures & tables
Figure 1: A simple isometric data-generating process.
Figure 2: Controlling the bi-Lipschitz constant L leads to improved identifiability (reduced ℓ2 error).
Model Pair
Permutation
Supervised
ICA (% eff.)
Rigid
Linear
Pythia-160M-0 → Pythia-160M-1
0.219
0.150
0.131
0.202 (25%)
MAE- timm → MAE-original
0.197
0.109
0.036
0.145 (59%)
CheXpert-small → CheXpert-base
0.218
0.104
0.048
0.175 (38%)
ResNet-18-fc-1 → ResNet-18-fc-2
0.382
0.206
0.175
0.312 (40%)
Table 1: Supervised and unsupervised alignment scores between pairs of models which measure empirical identifiability. The optimal transforms (permutation, rigid, linear, or ICA) are estimated between the two models, and average ℓ2 errors normalized by latent diameter are reported. For ICA, we also report its efficiency as the reduction of ℓ2 error from the permutation to the rigid transform.
model
aggregated
Shapes3D
MPI3D
Falcor3D
Isaac3D
( InfoM InfoE InfoC ) ↑
AE
(0.39 0.76 0.25 )
(0.34 0.99 0.16 )
(0.42 0.40 0.31 )
(0.37 0.83 0.20 )
(0.41 0.80 0.34 )
β -VAE*
(0.59 0.81 0.55 )
(0.59 0.99 0.49 )
(0.45 0.71 0.51 )
( 0.71 0.73 0.70 )
(0.60 0.80 0.51 )
β -TCVAE*
(0.58 0.72 0.59 )
(0.61 0.82 0.62 )
( 0.51 0.60 0.57 )
(0.66 0.74 0.71 )
(0.54 0.70 0.46 )
BioAE*
(0.54 0.75 0.36 )
(0.56 0.98 0.44 )
(0.45 0.66 0.36 )
(0.54 0.73 0.31 )
(0.63 0.65 0.33 )
AE + ICA (ours)
( 0.65 0.83 0.40 )
( 0.79 0.99 0.52 )
(0.44 0.66 0.31 )
( 0.71 0.83 0.33 )
( 0.64 0.82 0.43 )
Table 2: Disentanglement metrics ( InfoM , InfoE , InfoC ), of which InfoM and InfoE are the most important. AE + ICA performs comparably to some of the best disentanglement-specific neural networks, with almost no tuning. Results marked with (*) are quoted without reproduction from Hsu et al. (2023) . Bolded metrics have the highest point estimates. Full details in Appendix A.7.3 .
Mean AUROC ( ↑ )
Sparsity ( ↑ more sparse)
Gene
Base
PCA
PCA + ICA
PCA + Rand
Base
PCA
PCA + ICA
PCA + Rand
CYP11B1 (1)
0.663
0.692
0.709
0.678
0.184
0.204
0.237
0.188
EIF3H (1)
0.682
0.724
0.749
0.725
0.192
0.224
0.268
0.214
HCK (1)
0.670
0.693
0.711
0.668
0.156
0.208
0.241
0.166
MTOR (6)
0.663
0.690
0.705
0.679
0.166
0.201
0.233
0.186
PLK1 (6)
0.803
0.811
0.815
0.792
0.251
0.307
0.305
0.262
Table 3: Results from a downstream perturbation classification task. Each row represents a gene consisting of (#) separate experiments with different CRISPR guides. All experiments use the same set of 22,062 controls. Base = untransformed embeddings, PCA = whitened embeddings, PCA + ICA = whitened embeddings with ICA rotation applied, PCA + Rand = whitened embeddings with a random rotation applied.
Model
Concentration ( ↑ )
Base
0.163
PCA
0.332
PCA + ICA
0.386
PCA + Rand
0.287
Table 4: Concentration of biological variation in the top 25% of features.
Appendix figures & tables7 assets
Supplementary material from the paper’s appendix.
Appendix
Figure A.3: The distribution of sample-level bi-Lipschitz constant estimates B(z) tightens around 1 as α→0 .
Figure A.4: Controlling the bi-Lipschitz constant L leads to improved identifiability (reduced ℓ2 error). The proportionality does not appear to differ whether the max or mean bi-Lipschitz constant is estimated.
Figure A.5: Reconstruction error improves as the bi-Lipschitz constant grows (leak α→1 ). Notably, poor reconstruction does not inhibit identifiability.
Figure A.6: In digit-specific models, controlling the bi-Lipschitz constant L leads to improved identifiability (reduced ℓ2 error) with pattern similar to the full-dataset models.
Shapes3D
model
InfoM ↑
InfoE ↑
InfoC ↑
AE
0.41±0.03
0.98±0.01
0.28±0.01
β -VAE
0.59±0.02
0.99±0.02
0.49±0.03
β -TCVAE
0.61±0.03
0.82±0.02
0.62±0.02
BioAE
0.56±0.02
0.98±0.01
0.44±0.02
AE (reproduction)
0.36±0.05
1.00±0.00
0.18±0.06
Appendix
Table 5: Full results across datasets.
Hyperparameter
Value
is_unbalance
True
learning_rate
0.05
num_leaves
31
feature_fraction
0.6
reg_alpha
5.0
reg_lambda
1.0
Appendix
Table 6: LightGBM hyperparameters used in all experiments.
Representation identifiability studies when learned internal codes can be uniquely recovered from observations or behavior, and which ambiguities remain.
Identifiability in representation learning is commonly evaluated using standard metrics (e.g., MCC, DCI, R^2) on synthetic benchmarks with known ground-truth factors. These metrics are assumed to reflect recovery up to the equivalence class guaranteed by identifiability theory. We show that this assumption holds only under specific structural conditions: each metric implicitly encodes assumptions about both the data-generating process (DGP) and the encoder. When these assumptions are violated, metrics become misspecified and can produce systematic false positives and false negatives. Such failures occur both within classical identifiability regimes and in post-hoc settings where identifiability is most needed. We introduce a taxonomy separating DGP assumptions from encoder geometry, use it to characterise the validity domains of existing metrics, and release an evaluation suite for reproducible stress testing and comparison.
Shruti Joshi, Théo Saulus, Wieland Brendel +3
Mila - Québec AI Institute & Université de Montréal · Max-Planck-Institute for Intelligent Systems, ELLIS Institute Tübingen, University of Tübingen
Understanding when neural networks learn interpretable representations is central to mechanistic interpretability. We study this through identifiability: when can supervised learning recover the true latent variables underlying the data? We introduce two data generating processes distinguished by their causal direction where either labels cause latents (generative) or latents cause labels (discriminative). When the representation has at least as many dimensions as the latent space, we prove that the optimal encoders are linear in the latents. Conversely, when dimensions are insufficient, identifiability generically fails and encoders become nonlinear. We then show that additional constraints yield recovery of individual latent variables, not just linear mixtures. In the generative setting, non-negative activations and energy efficiency force each neuron to encode exactly one latent, providing a precise account of selective grandmother cell-like responses. In the discriminative setting, when latents are sparse and the true measurement matrix satisfies the spark condition, the learned encoder inherits compressed sensing structure, enabling sparse dictionary learning to perfectly recover individual latents. This constitutes, to our knowledge, the first identifiability result for undercomplete nonlinear independent component analysis. Our framework unifies nonlinear ICA, compressed sensing, and neural network interpretability, delineating exactly when representations transition from nonlinear distortions to linear representations and, under additional constraints or steps, to clean, axis-aligned codes. Theoretical predictions are validated on synthetic benchmarks and on activations drawn from vision models, large language models, and macaque inferotemporal cortex.
Given a generalist model, learning a task-relevant specialist representation is fundamental for downstream applications. Identifiability, the asymptotic guarantee of recovering the ground-truth representation, is critical because it sets the ultimate limit of any model, even with infinite data and computation. We study this problem in a completely nonparametric setting, without relying on interventions, parametric forms, or structural constraints. We first prove that the structure between time steps and tasks is identifiable in a fully unsupervised manner, even when sequences lack strict temporal dependence and may exhibit disconnections, and task assignments can follow arbitrarily complex and interleaving structures. We then prove that, within each time step, the task-relevant latent representation can be disentangled from the irrelevant part under a simple sparsity regularization, without any additional information or parametric constraints. Together, these results establish a hierarchical foundation: task structure is identifiable across time steps, and task-relevant latent representations are identifiable within each step. To our knowledge, each result provides a first general nonparametric identifiability guarantee, and together they mark a step toward provably moving from generalist to specialist models.