Feature Evolution and Migration during Vision Transformer Training

Joonas Järve; Halil Ibrahim Aysel; Tarun Khajuria; Meelis Kull

cs.CVAug 20, 2026

Feature Evolution and Migration during Vision Transformer Training

Authors: Joonas Järve, Halil Ibrahim Aysel, Tarun Khajuria, Meelis Kull

Organizations: Institute of Computer Science University of Tartu Tartu, Estonia

Abstract

We present a novel view on feature evolution in Vision Transformers (ViTs) by visualizing the training process over two dimensions -- network depth (layer) and training time (epochs). We employ Sparse Autoencoders (SAEs) to extract candidate sparse features from CLS-token representations and compare their activation profiles across epoch--layer pairs. This allows us to study feature-level dynamics that are not directly visible from representation-level similarity measures. Furthermore, we demonstrate how this framework of feature evolution allows us to describe feature migration, the change in the layer where a feature is most detectable during training. Our experiments show that migration is concentrated early in training, occurs more often toward earlier layers than toward deeper layers, and declines as feature organization stabilizes. We further find that deeper layers stabilize earlier and more strongly than shallow layers. The results show that our approach can be employed as a tool for understanding how ViTs learn and evolve.

Figures & tables

Figure 1. Examples of distinct feature-evolution types observed during ViT training, shown in an epoch-by-layer view. $\star$ represents the reference feature.

Figure 2. Overview of the methodology: (A) extraction of image representations from every ViT checkpoint–layer pair; (B) SAE training at each pair; and (C) computation of feature-similarity scores for one example.

Figure 3. Feature-types based on ImageNet classes. Human-understandable SAE features are observed under 3 main categories: class-specific, superclass-specific and inter-class (see maximally activating images ( Zeiler and Fergus, 2014 ) for each feature in Appendix).

Figure 4. Percentage of features migrating towards deeper layers.

Panel of Figure 5 — Figure 5. Percentage of features migrating towards earlier layers.

Figure 6. Mean signed active drift. For each reference epoch–layer pair, feature-level quantities are averaged across features. Average signed displacement of the layer center, in layer-index units; negative values indicate movement toward earlier layers.

Figure 7. Percentage of stable features.

Figure 8. Mean active lifetime of features, given in number of epochs.

Figure 9. Layer center-of-mass (COM) $\pm$ standard deviation trajectories averaged over all the features of selected reference checkpoints (0,100,200 and 300). The high-frequency epochs are depicted as a solid line around the reference checkpoints (marked as x) and the low-frequency epochs as dots. On the figure below, the average layer width $\pm$ standard deviation is depicted. The averaging is carried out only on active epochs per feature and therefore color fading signifies the lack of active features in that area.

Figure 10. Percentage of features that are widely spread across layers.

Figure 11. Feature migration analysis by layer center-of-mass (COM). Most of the features migrate towards earlier layers.

Reference epoch	class	superclass	other	total (active)
1	70	0	755	825
100	259	22	678	959
200	401	35	872	1308
300	461	42	903	1406

Table 1. Feature-type counts by reference epochs presented in Figure 11 .

Panel of Figure 12 — Figure 12. The difference between two datasets in label entropy. Low label entropy indicates label-selective features, whereas high label entropy indicates features that activate across many labels.

Appendix figures & tables13 assets

Supplementary material from the paper’s appendix.

Appendix

Panel of Figure 13 — Figure 13. Number of features found by SAE.

Figure 14. Percentage of features migrating towards deeper layers on Mixed 10.

Figure 15. Percentage of stable features on Mixed 10.

Figure 16. Mean active lifetime of features, given in number of epochs on Mixed 10.

Figure 17. Percentage of features that are widely spread across layers on Mixed 10.

Figure 18. Mean signed active drift. For each reference epoch–layer pair, feature-level quantities are averaged across features. Average signed displacement of the layer center, in layer-index units; negative values indicate movement toward earlier layers on Mixed 10.

Figure 19. Layer center-of-mass (COM) $\pm$ standard deviation trajectories averaged over all the features of selected reference checkpoints (0,100,200 and 300). The high-frequency epochs are depicted as a solid line around the reference checkpoints (marked as x) and the low-frequency epochs as dots. On the figure below, the average layer width $\pm$ standard deviation is depicted. The averaging is carried out only on active epochs per feature and therefore color fading signifies the lack of active features in that area.

Figure 20. This figure depicts an SAE feature 881 (marked with $\star$ ) evolution discovered from checkpoint epoch 1, layer 8 (top figure). Figures below show the best-matching target feature at each selected checkpoint (marked with $\star$ ) and the target feature’s top-5 activating images (example image thumbnails are from the ImageNet-1k dataset).

Figure 21. SAE test loss on a log-scale across time and depth on ImageNet-1k trained ViT checkpoints.

Figure 22. Linear CKA between the ImageNet-1k representation at epoch 1, layer 8 and every sampled epoch–layer pair.

Figure 23. This figure depicts a diverse selection of SAE feature evolutions decoded from checkpoint epoch 1, layer 8 (marked with $\star$ ) and the feature’s top-5 activating images (example image thumbnails are from the ImageNet-1k dataset).

Figure 24. Feature-type examples illustrated by maximally activating images ( Zeiler and Fergus, 2014 ) . Human-understandable SAE features are observed under 3 main categories: class-specific, superclass-specific and inter-class (Example image thumbnails are from the ImageNet-1k dataset).

Quantity	Value	Role
$k_{\mathrm{on}}$	$3$	Active checkpoints for emergence
$k_{\mathrm{off}}$	$5$	Inactive checkpoints for disappearance
$\sigma_{\max}$	$4$	Maximum layer width
$f_{\mathrm{se}}$	$0.05$	Fraction of valid active epochs at drift endpoints
$\delta$	$1$	Anchor/reference stability radius
$d_{\min}$	$1$	Strict migration threshold in layers

Appendix

Table 2. Effective feature-evolution hyperparameters.

Neural Network Training Dynamics

Neural network training dynamics studies how parameters and representations change during optimization and how those changes shape learning, stability, and generalization.

See all 516 papers

From Edges to Depth: Probing the Spatial Hierarchy in Vision Transformers
Apr 25, 2026Jainum SanghaviTransformer Interpretability Visual Spatial Reasoning
Northeastern University
Interpreting vision transformers via residual replacement model
Sep 22, 2025Jinyeong Kim, Junhyeok Kim, Yumin Shim +3Transformer Interpretability Vision Transformer
Yonsei University
Can Cross-Layer Transcoders Replace Vision Transformer Activations? An Interpretable Perspective on Vision
Apr 14, 2026Gerasimos Chatzoudis, Konstantinos D. Polyzos, Zhuowei Li +4Transformer Interpretability Vision Transformer
Rutgers University · University of California San Diego

Apr 25, 2026J/K move · Enter open · S save

From Edges to Depth: Probing the Spatial Hierarchy in Vision Transformers

Jainum Sanghavi

Northeastern University

Transformer Interpretability Visual Spatial Reasoning Visual Representation Learning Vision Transformer

Open paper AskPDF arXiv

Vision Transformers trained only on image classification routinely transfer to tasks that demand spatial understanding, yet they receive no spatial supervision during pretraining. We ask where and how robustly such structure is encoded. Probing a frozen ViT-B/16 layerwise for two complementary properties, local patch boundaries (BSDS500) and per-patch depth (NYU Depth V2), reveals a clear hierarchy: boundary structure becomes linearly decodable at layers 5-6 (AP = 0.833), while depth, which requires integrating global cues, peaks two to three layers later at layer 8 (MAE = 0.0875). Both signals collapse at the final classification layer, and random-weight controls confirm the encodings are learned rather than architectural. Causal interventions add specificity: ablating the single direction a linear depth probe reads degrades depth decoding by up to 165%, while ablating any other direction changes it by less than 1%. Targeted activation patching along that direction shows the depth signal is partially re-derived at each layer rather than passively carried in the residual stream, with mid-layer interventions persisting most strongly downstream. The result is that a classification-trained ViT develops an actively maintained spatial hierarchy that mirrors the early-to-late progression observed in the primate visual cortex.

Abstract

Neural Network Training Dynamics

Explore similar work

From Edges to Depth: Probing the Spatial Hierarchy in Vision Transformers

Interpreting vision transformers via residual replacement model

Can Cross-Layer Transcoders Replace Vision Transformer Activations? An Interpretable Perspective on Vision