cs.LGMar 20, 2026
SaveOn the Dynamics & Transferability of Latent Generalization during Memorization
Organizations: Anuradha & Prashanth Palakurthi Centre for Artificial Intelligence Research, Department of Computer Science & Information Systems, Birla Institute of Technology & Science Pilani, Hyderabad 500078, India.
Abstract
Deep networks have been known to have extraordinary generalization abilities, via mechanisms that aren't yet well understood. It is also known that upon shuffling labels in the training data to varying degrees, deep networks, trained with standard methods, can still achieve perfect or high accuracy on this corrupted training data. This phenomenon is called memorization, and typically comes at the cost of poorer generalization to true labels. Our recent work has demonstrated, that the internal representations of such models retain significantly better latent generalization abilities than is directly apparent from the model. In particular, it has been shown that such latent generalization can be recovered via simple probes (called MASC probes) on the layer-wise representations of the model. However, the origin and dynamics over training of this latent generalization during memorization is not well understood. Here, we track the training dynamics, empirically, and find that latent generalization abilities largely peak early in training, with model generalization. Next, we investigate to what extent the specific nature of the MASC probe is critical for our ability to extract latent generalization from the model's layerwise outputs. To this end, we first examine the mathematical structure of the MASC probe and show that it is a quadratic classifier, i.e. is non-linear. This brings up the question of the extent to which this latent generalization might be linearly decodable from layerwise outputs. To investigate this, we designed a new linear probe for this setting. Next, we consider the question of whether it is possible to transfer latent generalization to model generalization by directly editing model weights. To this end, we devise a way to transfer the latent generalization present in last-layer representations to the model using the new linear probe.
Figures & tables
Figure 1: Minimum Angle Subspace Classifier (MASC) test accuracy over epochs of training for multiple models/datasets, where test data is projected onto class-specific subspaces constructed at each epoch from corrupted training data with the indicated label corruption degree. The plots display MASC accuracy across different layers of the network. For reference, the evolution of test accuracy of the corresponding model (blue dotted line) over epochs of training is also shown. FC denotes fully connected layers with activation, and Flat refers to the flatten layer without .
Figure 2: Vector Linear Probe Intermediate-layer Classifier (VeLPIC) test accuracy during training of the network, where test data is projected onto class vectors constructed at each epoch from training data with the indicated label corruption degrees. The plots display VeLPIC accuracy across different layers of the network for various model–dataset combinations. For reference, the test accuracy of the models (blue dotted line) over epochs of training is also shown. FC denotes fully connected layers with activation, and Flat refers to the flatten layer without .
Figure 3: Difference in test accuracy (VeLPIC Accuracy - MASC Accuracy) during training of the network, where test data is projected onto class vectors constructed at each epoch from training data with the indicated label corruption degrees. The plots display difference in accuracy across different layers of the network for various model–dataset combinations. For reference, the test accuracy of the models (blue dotted line) over epochs of training is also shown, which would be 0.
Figure 4: Logistic regression probe’s test accuracy over epochs of training for multiple models/datasets. The plots display logistic regression probe’s accuracy across different layers of the network. For reference, the evolution of test accuracy of the corresponding model (blue dotted line) over epochs of training is also shown.
Figure 5: Difference in test accuracy (MASC Accuracy - Logistic regression probe Accuracy) during training of the network, where for MASC test data is projected onto class vectors constructed at each epoch from training data with the indicated label corruption degrees. The plots display difference in accuracy across different layers of the network for various model–dataset combinations. For reference, the test accuracy of the models (blue dotted line) over epochs of training is also shown, which would be 0.
Figure 6: Difference in test accuracy (VELPIC Accuracy - Logistic regression probe Accuracy) during training of the network, where for VELPIC test data is projected onto class vectors constructed at each epoch from training data with the indicated label corruption degrees. The plots display difference in accuracy across different layers of the network for various model–dataset combinations. For reference, the test accuracy of the models (blue dotted line) over epochs of training is also shown, which would be 0.
Figure 7: Model test accuracy when the weight intervention is applied to the epoch in question during training. The test accuracy of the model with standard training without weight intervention (blue dotted line) is overlaid for comparison.
Figure 8: Model test accuracy with true labels during training when a weight intervention that involves replacing pre-softmax weights with the VeLPIC vector, is performed at the 40th epoch and standard training is performed thereafter for 60 epochs. The corresponding test accuracies for the model with standard training (dotted) without intervention is overlaid for comparison. The results correspond to a single run in each case.
Figure 9: Vector Linear Probe Intermediate layer Classifier (VeLPIC) test accuracy during training on models when intervention is performed at 40th epoch and standard training is performed thereafter for 60 epochs. The VeLPIC test accuracy on models trained with standard training (dotted) without intervention is overlaid for comparison. Model test accuracy for models trained with and without intervention is overlaid for comparison. The results correspond to a single run in each case.
Appendix figures & tables15 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 10: Minimum Angle Subspace Classifier (MASC) Test accuracy for 0% and 100% corruption degrees during training of the network, where test data is projected onto class-specific subspaces constructed from training data with the indicated label corruption degrees. The plots display MASC accuracy across different layers of the network for various model–dataset combinations. For reference, the test accuracy of the models (dotted line) is also shown. Each row corresponds to a specific corruption degree, while columns represent different models, as labeled. FC denotes fully connected layers with activation, and Flat refers to the flatten layer without .
Figure 11: Vector Linear Probe Intermediate-layer Classifier (VeLPIC) test accuracy for 0% and 100% corruption degrees during training of the network, where test data is projected onto class vectors constructed at each epoch from training data with the indicated label corruption degrees. The plots display VeLPIC accuracy across different layers of the network for various model–dataset combinations. For reference, the test accuracy of the models (blue dotted line) over epochs of training is also shown. FC denotes fully connected layers with activation, and Flat refers to the flatten layer without .
Figure 12: Difference in test accuracy (VeLPIC Accuracy - MASC Accuracy) during training of the network, where test data is projected onto class vectors constructed at each epoch from training data with the indicated label corruption degrees. The plots display difference in accuracy across different layers of the network for various model–dataset combinations. For reference, the test accuracy of the models (blue dotted line) over epochs of training is also shown, which would be 0.
Figure 13: Logistic regression probe’s test accuracy over epochs of training for multiple models/datasets. The plots display logistic regression probe’s accuracy across different layers of the network. For reference, the evolution of test accuracy of the corresponding model (blue dotted line) over epochs of training is also shown.
Figure 14: Difference in test accuracy (MASC Accuracy - Logistic regression probe Accuracy) during training of the network, where for MASC test data is projected onto class vectors constructed at each epoch from training data with the indicated label corruption degrees. The plots display difference in accuracy across different layers of the network for various model–dataset combinations. For reference, the test accuracy of the models (blue dotted line) over epochs of training is also shown, which would be 0.
Figure 15: Difference in test accuracy (VELPIC Accuracy - Logistic regression probe Accuracy) during training of the network, where for VELPIC test data is projected onto class vectors constructed at each epoch from training data with the indicated label corruption degrees. The plots display difference in accuracy across different layers of the network for various model–dataset combinations. For reference, the test accuracy of the models (blue dotted line) over epochs of training is also shown, which would be 0.
Figure 16: Comparing model test accuracy with VeLPIC transferred accuracy when the weight intervention is applied to the model at the epoch in question during training for corruption degrees 0% and 100%. The test accuracy of the model with standard training without weight intervention (blue dotted line) is overlaid for comparison.
Figure 17: Model train accuracy on corrupted dataset when the VeLPIC weight intervention is applied to pre-softmax weights at the epoch in question during training. The training accuracy on corrupted dataset of the model with standard training without weight intervention (blue dotted line) is overlaid for comparison. Observe that, except for 100% corruption degree, the transferred training accuracy tends to saturate at a level largely consistent with the fraction of true training labels in the corrupted dataset.
Figure 18: Model train accuracy on corrupted dataset when the VeLPIC weight intervention is applied to pre-softmax weights at the epoch in question during training. The training accuracy on corrupted dataset of the model with standard training without weight intervention (blue dotted line) is overlaid for comparison.
Figure 19: Vector Linear Probe Intermediate layer Classifier (VeLPIC) test accuracy during training on models when intervention is performed at 40th epoch and standard training is performed thereafter for 60 epochs for 0.0 corruption degree. The VeLPIC test accuracy on models trained with standard training (dotted) without intervention is overlaid for comparison. The results correspond to a single run in each case.
Figure 20: Model, MASC and VeLPIC test accuracy for randomly initialized and trained models across different corruption degrees and model–dataset pairs. For MASC and VeLPIC, results are reported for the best performing layer, defined as the layer achieving the highest MASC/VeLPIC test accuracy for the corresponding model–dataset pair. The test accuracies are averaged over three runs.
Figure 21: Minimum Angle Subspace Classifier (MASC) test accuracy over epochs of training for ResNet-18 and CNN models having trained with and with dropout, where test data is projected onto class-specific subspaces constructed at each epoch from corrupted training data with the indicated label corruption degree. The plots display MASC accuracy across different layers of the network. For reference, the evolution of test accuracy of the corresponding model (blue dotted line) over epochs of training is also shown. WD corresponding to models trained with dropout.
Figure 22: Minimum Angle Subspace Classifier (MASC) test accuracy for 0% corruption degrees over epochs of training for ResNet-18 and CNN models having with and with dropout, where test data is projected onto class-specific subspaces constructed at each epoch from corrupted training data. The plots display MASC accuracy across different layers of the network. For reference, the evolution of test accuracy of the corresponding model (blue dotted line) over epochs of training is also shown. WD corresponding to models trained with dropout.
Figure 23: Vector Linear Probe Intermediate-layer Classifier (VeLPIC) test accuracy during training of ResNet-18 and CNN models with and with dropout, where test data is projected onto class vectors constructed at each epoch from training data with the indicated label corruption degrees. The plots display VeLPIC accuracy across different layers of the network for various model–dataset combinations. For reference, the test accuracy of the models (blue dotted line) over epochs of training is also shown. WD corresponding to models trained with dropout.
Figure 24: Vector Linear Probe Intermediate-layer Classifier (VeLPIC) test accuracy for 0% corruption degrees during training of ResNet-18 and CNN models with and with dropout, where test data is projected onto class vectors constructed at each epoch from training data. The plots display VeLPIC accuracy across different layers of the network for various model–dataset combinations. For reference, the test accuracy of the models (blue dotted line) over epochs of training is also shown. WD corresponding to models trained with dropout.