Topological Dynamics Under Non-Stationary Supervision: Persistent Homology of Neural Network Weight Evolution
Organizations: Independent Researcher
Abstract
We investigate how the persistent topology of neuron- weight point clouds evolves under stable and non-stationary supervision. An identical multilayer perceptron is trained on Fashion-MNIST with correct labels, a fixed random permutation of the training-label vector, or a fresh permutation every epoch. Five paired seeds share initialisation and minibatch order, yielding 15 models trained for 100 epochs. Raw and row-normalised incoming weight clouds are analysed with H0 and H1 Vietoris Rips persistence at 22 checkpoints. Correct-label models reach 89.39% mean true-label test accuracy; fixed-random models fit 67.94% of their assigned training targets without generalising to the original task. Correct-label models have descriptively greater time-averaged normalised fc3 H1 persistence than either random-label condition, but the configured exact comparisons do not reach conventional significance after Holm correction. Early condition classification achieves 60.0% mean balanced accuracy using topology, 66.7% using conventional diagnostics, and 60.0% using both. Thus, no incremental predictive improvement is demonstrated in this setting. The archive contains all 3,300 configured topology cases. Interpretation is limited by five seeds, a single architecture and dataset, 282 capped H1 distance comparisons, and deferred bottleneck analysis.
Figures & tables
| Item | Setting |
|---|---|
| Fitting / validation / test | 54,000 / 6,000 / 10,000 |
| Optimizer / loss | AdamW / cross-entropy |
| Learning rate / weight decay | Constant 0.001 / 0.01 |
| Batch size / epochs | 256 / 100 |
| Gradient clipping | Global L2 limit 1.0 |
| Paired seeds | 42, 123, 456, 789, 2026 |
| Condition | Assigned train | True validation | True test |
|---|---|---|---|
| Correct | |||
| Fixed random | |||
| Changing random |
| Outcome | Comparison | Mean | Unadjusted 95% interval | Exact | Holm | |
|---|---|---|---|---|---|---|
| Fixed random Changing random | 0.1680 | 0.404 | 0.5000 | 1.0000 | ||
| Correct Fixed random | 1.6156 | 6.538 | 0.0625 | 0.4375 | ||
| Correct Changing random | 1.7836 | 3.173 | 0.0625 | 0.4375 | ||
| Fixed random Changing random | 0.0952 | 0.841 | 0.1875 | 0.9375 | ||
| Correct Fixed random | -0.0407 | -0.762 | 0.1875 | 0.9375 | ||
| Correct Changing random | 0.0546 | 0.624 | 0.2500 | 0.9375 |
| Held-out seed | Topology | Baseline | Combined |
|---|---|---|---|
| 42 | 66.67 | 33.33 | 66.67 |
| 123 | 66.67 | 100.00 | 66.67 |
| 456 | 66.67 | 66.67 | 66.67 |
| 789 | 33.33 | 66.67 | 33.33 |
| 2026 | 66.67 | 66.67 | 66.67 |
| Mean | 60.00 | 66.67 | 60.00 |
Appendix figures & tables24 assets
Supplementary material from the paper’s appendix.
Appendix
| Seed | Condition | Online loss | Assigned train (%) | True train (%) | True val. (%) | True test (%) |
|---|---|---|---|---|---|---|
| 42 | Correct | 0.0214 | 99.50 | 99.50 | 89.85 | 89.60 |
| 42 | Fixed random | 1.0002 | 66.91 | 10.20 | 10.23 | 10.86 |
| 42 | Changing random | 2.3027 | 10.00 | 10.00 | 10.00 | 10.00 |
| 123 | Correct | 0.0210 | 99.29 | 99.29 | 89.52 | 89.23 |
| 123 | Fixed random | 0.8170 | 74.19 | 10.32 | 10.52 | 10.95 |
| 123 | Changing random | 2.3027 | 10.00 | 10.00 | 10.00 | 10.00 |
| Layer | Condition | Raw H0 | Raw H1 | Normalized H0 | Normalized H1 |
|---|---|---|---|---|---|
| fc1 | Correct | ||||
| fc1 | Fixed random | ||||
| fc1 | Changing random | ||||
| fc2 | Correct | ||||
| fc2 | Fixed random | ||||
| fc2 | Changing random |
| Condition | Lag | Loss | Gradient | Update | Val. accuracy | Weight norm |
|---|---|---|---|---|---|---|
| Correct | 0 | -0.096 | -0.099 | +0.242 | -0.063 | -0.042 |
| Correct | 1 | +0.040 | +0.158 | +0.219 | -0.212 | +0.026 |
| Fixed random | 0 | -0.358 | +0.418 | -0.026 | +0.032 | +0.138 |
| Fixed random | 1 | -0.393 | +0.505 | +0.028 | +0.172 | +0.044 |
| Changing random | 0 | +0.325 | +0.209 | +0.167 | -0.055 | -0.570 |
| Changing random | 1 | +0.112 | +0.391 | +0.089 | +0.092 | -0.498 |
| Epoch | Supervision | Online loss | Target acc. (%) | Validation (%) | True-label gap (pp) |
|---|---|---|---|---|---|
| 1 | Correct | ||||
| 1 | Fixed random | ||||
| 1 | Changing random | ||||
| 5 | Correct | ||||
| 5 | Fixed random | ||||
| 5 | Changing random |
| Epoch | Supervision | Gradient norm | Weight norm | Update norm | Epoch duration (s) |
|---|---|---|---|---|---|
| 1 | Correct | ||||
| 1 | Fixed random | ||||
| 1 | Changing random | ||||
| 5 | Correct | ||||
| 5 | Fixed random | ||||
| 5 | Changing random |
| Layer | Supervision | Raw | Normalized |
|---|---|---|---|
| fc1 | Correct | ||
| fc1 | Fixed random | ||
| fc1 | Changing random | ||
| fc2 | Correct | ||
| fc2 | Fixed random | ||
| fc2 | Changing random |
| Supervision | Cloud | Mean distance | Effective rank | Leading singular value | H1 count | Max. lifetime |
|---|---|---|---|---|---|---|
| Correct | raw | |||||
| Correct | normalized | |||||
| Fixed random | raw | |||||
| Fixed random | normalized | |||||
| Changing random | raw | |||||
| Changing random | normalized |
| Layer | Supervision | Median Spearman | Minimum | Maximum |
|---|---|---|---|---|
| fc1 | Correct | 0.630 | 0.421 | 0.759 |
| fc1 | Fixed random | 0.142 | -0.101 | 0.675 |
| fc1 | Changing random | 0.570 | 0.526 | 0.936 |
| fc2 | Correct | 0.173 | -0.185 | 0.311 |
| fc2 | Fixed random | 0.508 | 0.436 | 0.599 |
| fc2 | Changing random | 0.632 | -0.123 | 0.850 |
| Layer | Temporal | Conditions | Seeds | Total |
|---|---|---|---|---|
| fc1 | 15 | 15 | 30 | 60 |
| fc2 | 15 | 15 | 30 | 60 |
| fc3 | 16 | 17 | 34 | 67 |
| fc4 | 21 | 26 | 48 | 95 |
| Total | 67 | 73 | 142 | 282 |
| Epoch | Temporal | Conditions | Seeds | Total |
|---|---|---|---|---|
| 0 | 0 | 60 | 120 | 180 |
| 1 | 60 | 13 | 22 | 95 |
| 2 | 7 | 0 | 0 | 7 |
| Outcome | Comparison | 42 | 123 | 456 | 789 | 2026 |
|---|---|---|---|---|---|---|
| Fixed random Changing random | -0.0781 | 0.2515 | 0.2400 | 0.7685 | -0.3417 | |
| Correct Fixed random | 1.3069 | 1.9862 | 1.5417 | 1.6765 | 1.5666 | |
| Correct Changing random | 1.2287 | 2.2378 | 1.7817 | 2.4449 | 1.2249 | |
| Fixed random Changing random | 0.2444 | 0.0586 | -0.0648 | 0.1408 | 0.0973 | |
| Correct Fixed random | -0.0473 | -0.0007 | 0.0246 | -0.1077 | -0.0722 | |
| Correct Changing random | 0.1970 | 0.0579 | -0.0402 | 0.0331 | 0.0251 |
| Held-out seed | Topology | Baseline | Combined |
|---|---|---|---|
| 42 | 0.1 | 1 | 0.01 |
| 123 | 0.01 | 0.1 | 0.01 |
| 456 | 0.01 | 0.01 | 0.01 |
| 789 | 0.01 | 0.01 | 0.01 |
| 2026 | 0.01 | 0.01 | 0.01 |
| Feature family | Coordinates | Balanced acc. (%) | Minimum | Maximum | Versus baseline (pp) |
|---|---|---|---|---|---|
| baseline | 94 | 33.33 | 100.00 | ||
| combined | 1184 | 33.33 | 66.67 | ||
| topology | 1090 | 33.33 | 66.67 |
| Component | Recorded quantity |
|---|---|
| Training + monitoring | 2,643.82 s; excludes checkpoint I/O |
| Cloud topology computation | 191.48 s; excludes other stages |
| Diagram matching wall time | Not separately recorded |
| Nested prediction wall time | Not separately recorded |
| Original ZIP size | 2,192,571,623 bytes |
| Compute device | One Tesla T4; CPU PH/matching |
Neural Network Training Dynamics
Neural network training dynamics studies how parameters and representations change during optimization and how those changes shape learning, stability, and generalization.
See all 547 papers