cs.LGOct 12, 2026
SaveFisher information and the geometry of memorization in neural networks
Organizations: Princeton University1; Lewis-Sigler Institute · Princeton University2; Joseph Henry Laboratories · University of Sydney3; School of Physics · CUNY Graduate Center4; Initiative for the Theoretical Sciences
Abstract
A fruitful approach towards understanding generalization in machine learning involves characterizing how models separate generalizable structure from idiosyncratic or even corrupted features of training data. Fisher information has emerged as a diagnostic tool for this separation, leveraging loss geometry to distinguish generalization from memorization. Here we investigate this connection in vision models trained on CIFAR-10 with controlled noise that can be learned only through memorization. We show that projecting model weights onto the top Fisher eigenvectors decouples general task performance from noisy sample memorization, and we investigate how this decoupling presents differentially across layers. We show that the same phenomena are present in small transformers trained to solve a synthetic in-context learning task, and are reproduced in closed form in a simple, analytically solvable linear model. Our results reveal that Fisher information captures aspects of the geometric structure underlying how neural networks allocate capacity between generalization and memorization.
Figures & tables
Figure 1: Layer-dependent decoupling of memorization and generalization via Fisher projection: memorized Gaussian images. Classification accuracy as a function of the number of retained top Fisher eigenvectors , computed over the full training set (CIFAR-10 and Gaussian images) for a single layer of a ResNet-20. Green: CIFAR-10 test; blue: CIFAR-10 train; red: memorized Gaussian samples. A ) In the first layer, CIFAR-10 train and test accuracy rise in tandem and reach a maximum at , while accuracy on memorized Gaussian samples remains at , demonstrating selective de-memorization at negligible cost to generalization. B ) In the last layer, all three curves rise sharply even for very small , with no separation between them at any point, indicating that memorized and structured representations have merged and can no longer be separated.
Figure 2: Layer-dependent decoupling of memorization and generalization via Fisher projection: noisy labels. Classification accuracy as a function of retained top Fisher eigenvectors for a ResNet-20 trained on CIFAR-10 with 10% label noise. Blue: train accuracy; Green: test accuracy; Red: accuracy on poisoned samples under their (wrong) training labels; Orange: accuracy on poisoned samples under their true labels. A&B) Fisher computed on clean (non-corrupted) data only. C&D) Fisher computed on the full training set. First layer , A&C) : top Fisher eigenspaces treat memorized and clean samples differently under both Fisher variants. Penultimate convolutional layer , B) : with the clean Fisher, de-memorization is successful — the red curve has not reached its maximum by the time train and test accuracy do — and the model partially recovers the true labels of label-flipped samples, with the orange curve rising well above chance. Penultimate convolutional layer , D) : with the full Fisher, decoupling becomes weaker, highlighting the role of dataset choice in computing the Fisher.
Figure 3: Memorization and generalization across layers. Results of using a block-diagonal approximation to estimate the Fisher information across all model parameters in the noisy-label setting. A) The Fisher is computed on training data with clean labels only. The top eigenspace selectively captures information about the true classification task. We also see non-monotonic behavior in the performance on the true labels of memorized samples (orange dashed line). B) The Fisher is computed on a sample from the full training dataset, including samples with corrupted labels. In this case, top eigenspaces mix information across tasks, emphasizing the data-dependence of the Fisher.
Figure 4: From memorization to generalization, in-context. A) Transient ICL training dynamics. The three-block transformer first improves performance on held-out tasks, before overfitting to training tasks. B) Fisher projections for the attention layer, second transformer block. All projections decrease performance. Shaded region shows standard deviation across 16 models. C) Fisher projections for the MLP layer, second transformer block. Top Fisher subspaces improve task generalization (green) at the cost of performance on memorized tasks (blue).
Figure 5: Single-sample contributions and projections. Participation ratios for A) input-noise case and B) label-noise case. A) suggests that structured CIFAR images primarily contribute to the top of the eigenspectrum, while noisy samples primarily contribute to the bottom of the eigenspectrum. This separation does not occur in B) . C) We plot the self-hit rate of correct sample classification after projecting out the top-k subspace of the corresponding single-sample Fisher information (‘sample projection’) in the input-noise case, and D) shows the same statistic for the label-noise case. Noisy samples are less likely to be recovered than their counterparts.
| Gaussian model | Noisy-label model (10% noise) | ||||||||||
| Fisher source CIFAR-10 (%) | Fisher source uncorrupted (%) | ||||||||||
| CIFAR-10 train (5k samples) | 100.0 | 99.9 | 99.9 | 99.9 | 99.8 | CIFAR-10 train (5k samples) | 100.0 | 100.0 | 100.0 | 99.9 | 99.9 |
| CIFAR-10 test | 84.3 | 83.9 | 83.7 | 83.4 | 83.1 | CIFAR-10 test | 73.4 | 73.1 | 73.0 | 72.7 | 72.3 |
| Fisher source Gaussian (%) | Fisher source corrupted (%) | ||||||||||
| Gaussian set | 100.0 | 98.4 | 96.1 | 90.5 | 75.6 | Corrupted CIFAR (assigned labels) | 100.0 | 100.0 | 100.0 | 99.9 | 99.8 |
Table 1: Bulk accuracy after sample projection. The column is the matched unchanged-weight BatchNorm control (see Appendix D for results on other Fisher sources).
Figure 6: Finite-rank linear model illustrates the risk gain from Fisher projection. A) Generative model. A latent sign vector is drawn uniformly from ; anisotropic scaling by amplitude vectors and produces signal features and clean label , respectively. The observed features result from concatenating the signal and the uninformative barcode features , modeled as an isotropic Gaussian vector in dimensions. The training label is the clean label with random per-dimension sign flips, with a random sign vector . B) Risk gain from removing the barcode block post hoc vs label flip probability , demonstrating quantitative agreement between theory (solid curve, Eq ( 7 )) and simulations (circles; error bars are smaller than the circle size). C) Prediction MSE vs retained Fisher rank for three flip rates (see legend). Two of four signal dimensions have population variance below the spectral separation threshold and therefore mix with the barcode bulk. In both theoretical prediction (solid curve, Eq ( 22 )) and simulations (circles), an interior optimum emerges as the flip rate increases. Shown results are for with replicates.
Appendix figures & tables6 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 7: Layerwise Fisher projection experiments for the input-noise case.
Figure 8: Memorization concentrates in a single MLP. Here, we show all transformer blocks for the model considered in Fig 4 . All layers except the Block 2 MLP perform worse on both train/test tasks when restricted to a top Fisher subspace, but in the Block 2 MLP test performance improves at the cost of performance on the training tasks.
Figure 9: Fisher projections for every layer, in the noisy sample setting. Fisher computed on clean samples.
Figure 10: Fisher projections for every layer, in the noisy sample setting. Fisher computed on a subset of the full training set.
Figure 11: Control experiments for A)-C) SVD compression and D)-F) random basis projections.
| Gaussian model | Noisy-label model (10% noise) | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| Fisher source CIFAR-10 | Fisher source uncorrupted | ||||||||||
| CIFAR-10 train (5k samples) | 100.0 | 99.9 | 99.9 | 99.9 | 99.8 | CIFAR-10 train (5k samples) | 100.0 | 100.0 | 100.0 | 99.9 | 99.9 |
| CIFAR-10 test | 84.3 | 83.9 | 83.7 | 83.4 | 83.1 | CIFAR-10 test | 73.4 | 73.1 | 73.0 | 72.7 | 72.3 |
| Gaussian set | 100.0 | 100.0 | 100.0 | 100.0 | 99.9 | Corrupted CIFAR (assigned labels) | 100.0 | 100.0 | 100.0 | 99.9 | 99.8 |
| Corrupted CIFAR (true labels) | 0.0 | 0.0 | 0.0 | 0.0 | 0.1 | ||||||
Table 2: Bulk accuracy after sample projection. All accuracies are percentages. BatchNorm is reset after each projection and recalibrated by cumulative averaging on equal batches of 250 images. The column is the matched unchanged-weight BatchNorm control.
More in this topic
Neural Network Memorization
Neural network memorization studies how models store and retrieve training examples, how this differs from generalization, and how memory can be measured or controlled.
See all 63 papers