Diagnosing Generalization Failures from Representational Geometry Markers

Chi-Ning Chou; Artem Kirsanov; Yao-Yuan Yang; SueYeon Chung

cs.LGMar 2, 2026

Diagnosing Generalization Failures from Representational Geometry Markers

Authors: Chi-Ning Chou, Artem Kirsanov, Yao-Yuan Yang, SueYeon Chung

Organizations: Flatiron Institute · Harvard University · Google DeepMind

Abstract

Generalization, the ability to perform well beyond the training context, is a hallmark of biological and artificial intelligence, yet anticipating unseen failures remains a central challenge. Conventional approaches often take a bottom-up'' mechanistic route by reverse-engineering interpretable features or circuits to build explanatory models. While insightful, these methods often struggle to provide the high-level, predictive signals for anticipating failure in real-world deployment. Here, we propose using a top-down'' approach to studying generalization failures inspired by medical biomarkers: identifying system-level measurements that serve as robust indicators of a model's future performance. Rather than mapping out detailed internal mechanisms, we systematically design and test network markers to probe structure, function links, identify prognostic indicators, and validate predictions in real-world settings. In image classification, we find that task-relevant geometric properties of in-distribution (ID) object manifolds consistently forecast poor out-of-distribution (OOD) generalization. In particular, reductions in two geometric measures, effective manifold dimensionality and utility, predict weaker OOD performance across diverse architectures, optimizers, and datasets. We apply this finding to transfer learning with ImageNet-pretrained models. We consistently find that the same geometric patterns predict OOD transfer performance more reliably than ID accuracy. This work demonstrates that representational geometry can expose hidden vulnerabilities, offering more robust guidance for model selection and AI interpretability.

Figures & tables

Figure 1: A diagnostic, system-level paradigm for studying generalization failures in DNNs, with an example on image classification. See Section 1.1 for an overview.

Figure 2: Object manifolds and task-relevant geometric measures. a , Object manifolds are the per-class point clouds in the feature space. b , Critical dimension $N_{\textsf{crit}}$ quantifies the degree of manifold untangling/separability in an average-case sense via random projection. c , Anchor point distribution gives higher weight to points that are more important for linear classification. 4 4 footnotemark: 4 d , The degree of manifold separation (quantified by critical number of neurons $N_{\textsf{crit}}$ ) is analytically linked to three task-relevant geometric measures: effective dimension $D_{\textsf{eff}}$ , radius $R_{\textsf{eff}}$ and utility $\Psi_{\textsf{eff}}$ .

	$D_{\textsf{eff}}\geq 0$	$R_{\textsf{eff}}\geq 0$	$\Psi_{\textsf{eff}}\in[0,1]$
Geometric intuition	Quantify the task-relevant dimensionality of object manifolds.	Quantify the task-relevant spread within each manifold relative to their centers.	Quantify the amount of excessive compression of untangling manifolds.
Effect on linear separability	More separable when $D_{\textsf{eff}}$ is small.	More separable when $R_{\textsf{eff}}$ is small.	More separable when $\Psi_{\textsf{eff}}$ is large.
Example	$D_{\textsf{eff}}$ equals the dimension of uncor. random spheres	$R_{\textsf{eff}}$ equals the radius of uncor. random spheres	Collapsing manifolds to points yields $\Psi_{\textsf{eff}}\to 0$ .
Interpretation in feature learning 5 5 5 This figure is a schematic illustration of the non-uniform, task-relevant anchor point distribution. The 2D depiction is only intuitive and can be misleading, analogous to how in high dimensions Gaussian mass concentrates on the sphere rather than at the origin.	Low $D_{\textsf{eff}}$ indicates a smaller set of feature modes in use.	Low $R_{\textsf{eff}}$ indicates more similar feature usage across examples within a class.	Low $\Psi_{\textsf{eff}}$ indicates inefficient compression of within-class variability.

Table 1: Intuitions for GLUE measures.

Figure 3: Prognostic discovery for OOD generalization. a , We consider the image classification problem with an ID dataset and an OOD dataset with disjoint image classes. b , We trained DNNs on the ID dataset and evaluated the OOD performance as linear probe accuracy. c , Conventional performance and statistical measures on the ID dataset are weakly predictive of OOD performance, while some task-relevant geometric measures can robustly predict failures in OOD generalization.

Figure 4: All results on models trained on CIFAR-10, showing correlations between markers (x-axis) and OOD performance across a hyperparameter sweep. Numbers indicate Pearson $r$ ; asterisks denote significance ( ${}^{*}:p\leq 0.05$ ; ${}^{**}:p\leq 0.01$ ; ${}^{***}:p\leq 0.001$ ; ${}^{****}:p\leq 0.0001$ ).

Figure 5: Predict OOD transfer performance on ImageNet-pretrained models via $D_{\textsf{eff}}$ and $\Psi_{\textsf{eff}}$ . For the first block of models, our prognostic indicators predicted that v1 would outperform v2. For the second block of models, our prognostic indicators predicted the other way around.

Appendix figures & tables24 assets

Supplementary material from the paper’s appendix.

Appendix

Experimental Setting	In-Distribution (ID) Dataset	Out-of-Distribution (OOD) Datasets
Prognostic discovery ( Section 3 )	CIFAR-10	CIFAR-100 ImageNet-1k (resized to 32x32)
Transfer Learning Applications ( Section 4 )	ImageNet-1k	Flowers102 Stanford Cars Places365 Oxford-IIIT Pets Food-101 iNaturalist 2018 DTD FGVC-Aircraft HAM10000

Appendix

Table 2: Summary of dataset roles in our experiments.

Architecture	Layer name in PyTorch module
VGG13	features.34
VGG16	features.43
VGG19	features.52
ResNet	avgpool
DenseNet121	avg_pool2d
MobileNet	avg_pool2d

Appendix

Table 3: Exact layer names used for extracting feature representations.

	$D_{\textsf{eff}}\geq 0$	$R_{\textsf{eff}}\geq 0$	$\Psi_{\textsf{eff}}\in[0,1]$
Geometric intuition	Quantify the task-relevant dimensionality of object manifolds.	Quantify the task-relevant spread within each manifold relative to their centers.	Quantify the amount of excessive compression of untangling manifolds .
Effect on linear separability	More separable when $D_{\textsf{eff}}$ is small.	More separable when $R_{\textsf{eff}}$ is small.	More separable when $\Psi_{\textsf{eff}}$ is large.
Example	$D_{\textsf{eff}}$ equals the dimension of uncor. random spheres	$R_{\textsf{eff}}$ equals the radius of uncor. random spheres	Collapsing manifolds to points yields $\Psi_{\textsf{eff}}\to 0$ .
Formula for uncorrelated random spheres 6 6 footnotemark: 6	$\frac{1}{P}\sum_{\mu}\mathop{\mathbb{E}}\left[\left(\frac{\langle\mathbf{s}^{\mu}_{1}(\mathbf{t}),\mathbf{t}\rangle}{\\|\mathbf{s}^{\mu}_{1}(\mathbf{t})\\|_{2}}\right)^{2}\right]$	$\frac{1}{P}\sum_{\mu}\sqrt{\mathop{\mathbb{E}}\left[\left(\frac{\\|\mathbf{s}^{\mu}_{1}(\mathbf{t})\\|_{2}}{\\|\mathbf{s}^{\mu}_{0}\\|_{2}}\right)^{2}\right]}$	$\frac{1}{P}\sum_{\mu}\mathop{\mathbb{E}}\left[\left(\frac{\langle\mathbf{s}^{\mu}_{1}(\mathbf{t}),\mathbf{t}\rangle}{\langle\mathbf{s}^{\mu}(\mathbf{t}),\mathbf{t}\rangle}\right)^{2}\right]$
Interaction with correlations among manifolds	If within-manifold variations align along similar directions, $D_{\textsf{eff}}$ decreases.	If manifold centers move farther apart, $R_{\textsf{eff}}$ decreases.	If within-manifold variations reduce without improving separability, $\Psi_{\textsf{eff}}$ decreases.
Interpretation in feature learning 7 7 footnotemark: 7	Low $D_{\textsf{eff}}$ indicates a smaller set of feature modes in use.	Low $R_{\textsf{eff}}$ indicates more similar feature usage across examples within a class.	Low $\Psi_{\textsf{eff}}$ indicates inefficient compression of within-class variability.

Appendix

Table 4: Intuitions for GLUE measures.

Architecture	Initial learning rate list	Weight decay list
VGG (13/16/19)	[0.01000, 0.00333, 0.00111, 0.00037]	[0.0010000, 0.0003333, 0.0001111, 0.0000370]
ResNet (18/34/50)	[1.00000, 0.50000, 0.25000, 0.12500]	[0.0002000, 0.0001000, 0.0000500, 0.0000250]
DenseNet121	[0.05000, 0.01667, 0.00556, 0.00185]	[0.0005000, 0.0001667, 0.0000556, 0.0000185]
MobileNet	[0.20000, 0.06667, 0.02222, 0.00741]	[0.0001000, 0.0000333, 0.0000111, 0.0000037]
EfficientNetB0	[0.20000, 0.06667, 0.02222, 0.00741]	[0.0001000, 0.0000333, 0.0000111, 0.0000037]

Appendix

Table 5: Hyperparameter grid for SGD optimizer.

Architecture	Initial learning rate list	Weight decay list
VGG (13/16/19)	[0.02000, 0.00500, 0.00125, 0.00031]	[0.0100000, 0.0033333, 0.0011111, 0]
ResNet (18/34/50)	[0.10000, 0.02500, 0.00625, 0.00156]	[0.0100000, 0.0050000, 0.0025000, 0]
DenseNet121	[0.05000, 0.02500, 0.01250, 0.00625]	[0.0100000, 0.0033333, 0.0011111, 0]
MobileNet	[0.02000, 0.00500, 0.00125, 0.04000]	[0.0100000, 0.0033333, 0.0011111, 0]
EfficientNetB0	[0.10000, 0.05000, 0.01000, 0.00100]	[0.0010000, 0.0003333, 0.0001111, 0]

Appendix

Table 6: Hyperparameter grid for AdamW optimizer.

Figure label	Model set	Optimizer	ID split	OOD dataset
Figure 7	Five DNNs	SGD	Train	CIFAR-100
Figure 8	Five DNNs	SGD	Test	CIFAR-100
Figure 9	Five DNNs	AdamW	Train	CIFAR-100
Figure 10	Five DNNs	AdamW	Test	CIFAR-100
Figure 11	Three ResNets + Three VGGs	SGD	Train	CIFAR-100
Figure 12	Three ResNets + Three VGGs	SGD	Test	CIFAR-100

Appendix

Table 7: Organization of figures in Appendix C .

Figure 6: All results, measures computed on the ID train set.

Figure 7: Five DNN architectures, trained with SGD, measures computed on the ID train set.

Figure 8: Five DNN architectures, trained with SGD, measures computed on the ID test set.

Figure 9: Five DNN architectures, trained with AdamW, measures computed on the ID train set.

Figure 10: Five DNN architectures, trained with AdamW, measures computed on the ID test set.

Figure 11: Three ResNet and three VGG architectures, trained with SGD, measures computed on the ID train set.

Figure 12: Three ResNet and three VGG architectures, trained with SGD, measures computed on the ID test set.

Figure 13: ResNet18 and VGG19, trained with SGD, evaluated on ImageNet subset OOD, measures computed on the ID train set.

Figure 14: ResNet18 and VGG19, trained with SGD, evaluated on ImageNet subset OOD, measures computed on the ID test set.

Figure 15: Corruption type: gaussian noise. Five DNN architectures, trained with SGD, measures computed on the ID test set.

Figure 16: Corruption type: fog. Five DNN architectures, trained with SGD, measures computed on the ID test set.

Figure 17: Corruption type: brightness. Five DNN architectures, trained with SGD, measures computed on the ID test set.

Figure 18: Corruption type: contrast. Five DNN architectures, trained with SGD, measures computed on the ID test set.

Figure 19: Corruption type: pixelate. Five DNN architectures, trained with SGD, measures computed on the ID test set.

Figure 20: Corruption type: jpeg compression. Five DNN architectures, trained with SGD, measures computed on the ID test set.

Figure 21: Prediction accuracy of OOD performance using different markers (or marker combinations). a , Using single marker. b , Using a pair of markers.

Figure 22: Fine-tuning dynamics of ImageNet-pretrained networks on Flowers102 dataset from v1 and v2 weights. Insets show ID measures at initialization

Figure 23: Fine-tuning dynamics of ImageNet-pretrained networks on StanfordCars dataset from v1 and v2 weights. Insets show ID measures at initialization

Neural Network Generalization

Neural network generalization studies why trained models perform well on unseen data and how architecture, data, and training shape that ability.

See all 303 papers

Does Your Neural Network Extrapolate? Feature Engineering as Identifiability Bias for OOD Generalization
May 8, 2026Leonel Aguilar, Jan Nagler, Christoph Hoelscher +1OOD Generalization Inductive Bias
Chair of Cognitive Science, ETH Zürich · Centre for Human and Machine Intelligence, Frankfurt School · Aisot Technologies AG +1
Inside-Out: Measuring Generalization in Vision Transformers Through Inner Workings
Apr 9, 2026Yunxiang Peng, Mengmeng Ma, Ziyu Yao +1OOD Generalization Vision Transformer
University of Delaware · George Mason University
ILDR: Geometric Early Detection of Grokking
Apr 22, 2026Shreel GolwalaNeural Network Generalization Early Stopping
Virginia Tech

May 8, 2026J/K move · Enter open · S save

Does Your Neural Network Extrapolate? Feature Engineering as Identifiability Bias for OOD Generalization

Leonel Aguilar, Jan Nagler, Christoph Hoelscher, Nino Antulov-Fantulin

Chair of Cognitive Science, ETH Zürich · Centre for Human and Machine Intelligence, Frankfurt School · Aisot Technologies AG+1

OOD Generalization Inductive Bias

Open paper AskPDF arXiv

Successful deep neural networks discover salient features of data. We show when and why they fail to learn out-of-distribution (OOD)-relevant representations from an in-distribution (ID) training window. This requires decoupling feature learning from data-generating-process (DGP) identifiability. From a single training window, OOD extrapolation is non-identifiable: infinitely many DGPs are

\varepsilon

-observationally equivalent on the training data but diverge arbitrarily outside it, and no in-distribution criterion alone reliably breaks the tie. A structural commitment, the feature map, label map, and model class

(\varphi, ψ, \mathcal{M})

, dictates the assumed DGP and governs OOD generalization while leaving ID performance essentially unchanged. When architecture, pretraining, augmentation, input formats, or domain knowledge implicitly inject the missing commitment, the model succeeds. When it cannot infer OOD-relevant structure from ID evidence, it fails. Changing only the representation can make the same architecture, at the same in-distribution loss, differ by

{\sim}520\times

out of distribution. When the commitment is correct and identifiable, OOD error vanishes. For example, Fourier coordinates turn periodic extrapolation into interpolation on

\mathbb{S}^1

. The same mechanism predicts outcomes in three natural-science settings (mass-action chemistry; Kepler's-third-law exoplanet prediction,

n=2{,}362

; and cross-species coding-DNA detection) and in a 264-run positional-encoding study across Transformer, Mamba, and S4D. Finally, a controlled study shows: correct features are necessary but not sufficient. The model class must express the target, and the transformed training data must cover the relevant representation space.

Abstract

Neural Network Generalization

Explore similar work

Does Your Neural Network Extrapolate? Feature Engineering as Identifiability Bias for OOD Generalization

Inside-Out: Measuring Generalization in Vision Transformers Through Inner Workings

ILDR: Geometric Early Detection of Grokking