cs.LGOct 26, 2026
SaveImproving Signals, Declining Encoder: A Known-Answer Test of Label-Free Monitoring in Self-Supervised Learning
Organizations: Weizmann Institute of Science, Rehovot, Israel
Abstract
Self-supervised training is monitored without labels, through the loss and statistics of the learned features. We report a case where we know the encoders lost part of what they had learned, so we can check whether these signals noticed. We trained LeJEPA encoders on synthetic images with known generating factors. In nine of ten seeds, the readability of those factors fell abruptly and stayed down, nearly always along the same direction in factor space. Meanwhile the loss kept falling, and most of the eleven label-free signals we tracked kept improving and rated the declining runs better than runs that did not decline. Standard change-point detection rarely flagged the decline near its onset, and selecting checkpoints by these signals mostly chose ones after it.
Figures & tables
Figure 1: (a) One training pair at : two renders with the same shape factors and energy that differ only in the sampled elements (log intensity, summed over channels). Grey: label-free signals; blue: readability, the average held-out ridge of the three factors. (b) The nine default runs that declined, aligned at their onsets, which are estimated from readability. Readability drops at onset and stays below its level before onset, while the training loss keeps its downward trend, with spikes in some runs, and RankMe rises. Figure 2 shows all ten runs against training epoch.
Figure 2: The ten default runs over 300 epochs: (a) readability, (b) training loss and (c) RankMe. RankMe increases overall, with local decreases; its small values come from the large feature mean, which RankMe does not subtract. (d) Held-out ridge of each factor for seed 3, the median decline in both drop and onset, highlighted in the other panels; its pre-onset checkpoint comes before the peak.
Figure 3: Label-free signals around a known readability decline. Orange: decline runs from the settings of the alarm audit (Appendix F ) that have both outcomes, aligned at their onsets. Grey: runs without a decline from the same settings, aligned at epoch 100, close to the median onset. Readability falls and stays down, while most label-free signals keep moving in their preferred direction; -ReQ moves away from 1. RankMe and the bottom row are the seven metrics compared by Arputharaj et al. (2026) .
Figure 4: How many runs declined under each change, out of how many runs (the count on each bar). The left group uses the default settings, and each other group changes one of them. A run declined if its readability ended at least 0.07 below its peak. All runs trained for 300 epochs. Appendix D covers the conditions not shown.
Appendix figures & tables4 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 5: One image rendered at , 100, 300 and 1000, with the intensity summed over the 4 depth layers and shown as on one shared scale. The render is view 1 of Figure 1 a.
| Condition | Width | Epochs | Runs | Declines | Drop range |
|---|---|---|---|---|---|
| , | 384 | 1000 | 6 | 0 | 0.00023–0.012 |
| , no weight decay | 384 | 300 | 3 | 1 | 0–0.29 |
| , constant LR | 384 | 300 | 3 | 2 | 0.027–0.27 |
| , SIGReg weight 0.05 | 384 | 300 | 10 | 3 | 0–0.27 |
| , cosine LR | 384 | 1000 | 3 | 3 | 0.21–0.55 |
| , constant LR | 384 | 1000 | 3 | 3 | 0.28–0.31 |
Table 1: Training conditions not shown in Figure 4 , all at . Declines are runs whose drop from peak readability is at least 0.07; drop ranges are rounded to two significant figures. LR is the learning-rate schedule. MAE has no projector.
| Metric | Variants | Preference |
|---|---|---|
| -ReQ | Covariance-eigenvalue power-law fit over one-based indices to : centred OLS, the original authors’ weighted fit, and uncentred OLS. Published code windows vary. | Closest to 1 |
| RankMe | Exponential of the entropy of normalized singular values, as published or centred; also centred, on squared singular values. | Higher |
| NE-Sum | Sum of covariance eigenvalues divided by their maximum, with and without centring. | Higher |
| Largest / smallest retained singular value, with and without centring; cutoff uses ten times estimated float32 storage rounding of the raw matrix. | Higher | |
| Self-Cluster | Printed unsquared and normalization-consistent squared forms, with and without centring. | Lower |
| DSE | Original authors’ code and paper normalizations, matching either the numerical bandwidth parameter or actual kernel width; row-normalized variants. The original paper leaves bandwidth unspecified. | Higher |
Table 2: Metric variants, and the direction in which users read each metric as better. Centred variants subtract the feature mean first.
Figure 6: Selected signal–test pairs at matched horizons: wide-projector runs cut at 150 epochs and default-projector runs at 300, before fitting. Timely first alarms among the 26 decline runs whose onset window lies inside these horizons, and alarms among the 13 runs without a decline, for the initial and control-referenced procedures. The last row uses groups that redraw shape (Section 5 ), an augmentation not used in training. Centred subtracts the feature mean first.
More in this topic
Self-Supervised Learning
Self-supervised learning uses structure within unlabeled data to train representations and models for downstream tasks.
See all 261 papers