A Topological Signature of Learning under Noisy Labels

Zhongtian Sun; Fan Mo; Prayag Tiwari; Kelin Xia

cs.LGOct 16, 2026

A Topological Signature of Learning under Noisy Labels

Authors: Zhongtian Sun, Fan Mo, Prayag Tiwari, Kelin Xia

Organizations: University of Kent, United Kingdom · University of Oxford, United Kingdom · Halmstad University, Sweden · Nanyang Technological University, Singapore

Abstract

How does noisy supervision change the representations learned by a neural network? Training accuracy measures agreement with training labels but provides little information about representation geometry. We use persistent homology to track loop structure across distance thresholds in neural activations. Experiments span image classification, text classification, and modular arithmetic. On a fixed CIFAR-10/ResNet-18 training probe, equally sized clean-label and corrupted-label subsets show no detectable persistence difference at epoch 10, whereas corrupted-label subsets have higher persistence at epoch 50. The contrast survives uniform-scale controls, even though predictions match the assigned noisy labels for only 7-11% of corrupted probe examples at that checkpoint. Normalisation preserves the crossover between clean and noisy training trajectories on CIFAR-10/ResNet-18, whereas CIFAR-100/ResNet-18 shows no crossover under any of the five evaluated constructions. In a separate noise-rate calibration comparison across five pipelines, mean activation norm achieves a higher mean in-sample R2 than raw persistence (0.936 versus 0.908). These findings show that noisy supervision leaves a measurable geometric signature even when agreement with corrupted labels is low. Comparing persistence under scale controls with label-fitting measurements offers a way to study how representations respond to label corruption beyond what training accuracy reveals.

Figures & tables

Symbol	Meaning
$Z(t)$	Probe activation cloud at epoch $t$ .
$\Phi(t)$	Filtered total finite $H_{1}$ lifetime on raw activations, unless qualified.
$\Phi_{H_{0}}$	Sum of finite $H_{0}$ lifetimes, excluding the infinite bar.
$\mathcal{F}_{1}$ , $L_{\max}$	Retained $H_{1}$ bars and maximum finite $H_{1}$ lifetime, respectively.
$\mathrm{sf}$	Selection fraction for symmetric uniform relabelling
$\Delta\Phi$	Noisy minus clean persistence under the same construction.

Table 1: Notation. Normalised constructions are named explicitly when reporting results.

$\mathrm{sf}$	Raw $\Phi$	$\Phi_{H_{1}}/\Phi_{H_{0}}$	$\Phi/\operatorname{diam}(Z)$	Loop count
0.25	1.29	1.23	1.44	1.24
0.50	1.37	1.28	1.46	1.36

Table 2: Corrupted/clean persistence ratios at epoch 50 on the fixed CIFAR-10/ResNet-18 training probe, with eight seeds per noise setting. Each entry is the median corrupted-subset statistic divided by the median clean-subset statistic across seeds. Corrupted subsets have larger values in all eight seeds except for diameter-normalised persistence at

sf=0.25

, where the direction agrees in seven seeds. Exact two-sided paired Wilcoxon tests give

p=0.0078

for the eight-of-eight comparisons and

p=0.0156

for the seven-of-eight comparison.

Figure 1: Seed-level persistence comparisons. (a) Corrupted/clean ratios of

\Phi_{H_{1}}/\Phi_{H_{0}}

on the fixed CIFAR-10/ResNet-18 training probe, with eight seeds per setting. Dots show within-run ratios and horizontal bars show their medians. These differ from the ratios of seed medians reported in Table 2 . (b) Epoch-50 random/Patrini ratios for mean activation norm and cosine persistence. Each dot shows one seed’s median ratio across three nominal settings. The random arm changes approximately 1.8 times as many labels as the Patrini arm, so the comparison includes differences in both corruption rate and relabelling structure. Both panels use logarithmic vertical axes. Dashed lines mark a ratio of one.

Figure 2: Representation and persistent cycles in the synthetic two-Gaussian task, using seed 0 at epoch 10. The top and bottom rows show 0% and 50% of near-boundary labels flipped, respectively; the boundary region is

|x_{0}|<1

. Left: PCA projections of the 300-point subsamples of 64-dimensional activations, fitted separately for each condition. Colour denotes the observed label and crosses mark flipped labels. Black edges show a representative of the longest finite

H_{1}

bar, computed in the original 64-dimensional Euclidean Rips filtration over

\mathbb{Z}/2\mathbb{Z}

. Middle: the same representative projected onto its own first two principal components, with subsample points lying within its death radius of a cycle vertex. Right: persistence diagrams with the represented bar circled. Total persistence is 8.38 and 11.08, respectively. Neither displayed representative contains flipped points. Across 20 seeds, the longest-bar representatives show no detected enrichment for flipped points.

Pipeline	Seeds	Raw	Unit / cosine	RMS / z-score
CIFAR-10 / ResNet-18	5	30	20	20
MNIST / MLP	5	30	30	None
CIFAR-100 / ResNet-18	3	None	None	None

Table 3: First persistent crossing under the evaluated constructions: the earliest sampled checkpoint from which all seed-paired noisy trajectories remain above their clean references. “None” denotes no persistent crossing over the evaluated checkpoints. The CIFAR-10/ResNet-18 comparison uses

sf=0.5

Figure 3: Raw persistence on ImageNette/ResNet-50 over 100 epochs. Curves show the mean across seeds, with error bars indicating the sample standard deviation (SD, ddof = 1).

Observable / direction	Epoch 10	Epoch 25	Epoch 50
Raw $\Phi$	$\downarrow$ 8/8 (.0078)	$\downarrow$ 8/8 (.0078)	$\uparrow$ 8/8 (.0078)
Cosine $\Phi$ ( $\uparrow$ )	7/8 (.0156)	8/8 (.0078)	8/8 (.0078)
$\Phi_{H_{1}}/\Phi_{H_{0}}$ ( $\uparrow$ )	7/8 (.0156)	6/8 (.0391)	8/8 (.0078)
Mean norm ( $\downarrow$ )	8/8 (.0078)	8/8 (.0078)	8/8 (.0078)

Table 4: Random versus Patrini asymmetric relabelling on ResNet-18. The conditions have unequal realised corruption rates, as stated in the text. Each seed contributes the median of its three per-setting random/Patrini ratios. Entries give the number of seeds agreeing with the indicated direction out of eight, with exact two-sided Wilcoxon

p

-values in parentheses. Arrows indicate the random arm relative to the Patrini arm, not changes over training.

Observable	$R^{2}$ range	Mean $R^{2}$
Mean activation norm	0.890–0.990	0.936
Raw $\Phi$	0.871–0.927	0.908
Point-cloud diameter	0.566–0.972	0.790
Unit-normalised $\Phi$	0.382–0.821	0.606
$\Phi_{H_{1}}/\Phi_{H_{0}}$	0.115–0.757	0.500

Table 5: In-sample noise-rate calibration across five pipelines at epoch 10. Within each pipeline, ordinary least squares with an intercept predicts the nominal noise setting from each statistic over individual runs. Ranges and means summarize

R^{2}

across pipelines; ranges are not confidence intervals.

Appendix figures & tables17 assets

Supplementary material from the paper’s appendix.

Appendix

Architecture	Optimizer	LR	Schedule	Weight Decay	Batch	Epochs
MLP	Adam	0.001	Cosine	0	128	50
ResNet-18/50/101	SGD+mom.	0.1	Cosine	$5\times 10^{-4}$	128	50/100
ViT-Tiny/Large	AdamW	0.001	Cosine	0.01	128	50/100
DistilBERT	AdamW	$2\times 10^{-5}$	Lin. warmup	0.01	32	50

Appendix

Table 6: Training hyperparameters for the architectures used in the main experiments.

$n$	50	100	200	300	500	750	1000	1500
$\rho$	0.908	0.922	0.960	0.973	0.979	0.982	0.986	0.998
Time (s)	—	—	—	0.058	—	—	0.612	—

Appendix

Table 7: Subsampling sensitivity across 433 configurations.

\rho

measures rank agreement with the 2,000-point reference. Dashes indicate unreported runtimes.

Dataset	Architecture	$\rho_{10}$	$\rho_{50}$
CIFAR-10	ResNet-18	$-0.969$	$-0.225$
MNIST	MLP	$-0.931$	$-0.318$
CIFAR-10	ViT-Tiny	$-0.962$	$-0.962$
FashionMNIST	MLP	$-0.954$	$-0.876$
AG News	DistilBERT	$+0.870$	$+0.870$

Appendix

Table 8: Spearman correlations between noise level and raw persistence at epochs 10 and 50. Changes in correlation describe changes in noise ordering, whereas crossover concerns noisy persistence relative to a clean reference. Values use the raw, 300-point reanalysis consistently across pipelines.

Component	GPU-hours	CPU-hours
Original training (900+ runs)	$\sim$ 2,400	—
Grokking experiments (12 jobs)	$\sim$ 48	—
Realistic noise (39 jobs)	$\sim$ 195	—
TGCS evaluation (40 jobs)	$\sim$ 120	—
PH computation	—	$\sim$ 50

Appendix

Table 9: Training used the GPU partition, including A100 and older GPU models; persistent-homology computations and response-period analyses used CPUs. GPU-hours and CPU-hours are reported separately.

Method	$\Phi$ (ep50)	Test accuracy
Baseline	357.0	52.5%
Dropout	351.9	52.3%
Label smoothing	174.0	52.8%
Weight decay (high)	163.6	46.8%
TGT	618.6	50.1%
Val. early stop	—	51.4%

Appendix

Table 10: Regularisation methods on CIFAR-10/ResNet-18 (averaged over noise levels).

Task	Prime	Seed	$\Delta\Phi_{\mathrm{grok}}$	Decreases?
Addition	97	42	$-31.2$	Yes
Addition	97	123	$-46.9$	Yes
Addition	97	456	$-25.7$	Yes
Addition	113	42	$+3.7$	No
Addition	113	123	$-29.4$	Yes
Addition	113	456	$-64.3$	Yes

Appendix

Table 11: Changes in raw persistence across twelve grokking runs. The mean change is

-35.2

across all runs and

-43.1

among the ten decreasing runs.

Figure 4: Changes in raw persistence across the grokking transition for all twelve runs. Persistence decreases in ten runs and increases by

3.7

and

4.5

in the remaining two. The mean change is

-35.2

across all runs and

-43.1

among the ten decreasing runs. The dashed red line marks the latter mean.

sf	Baseline	Fixed (0.1)	Adaptive	Winner
0.00	86.92	86.74	86.77	Baseline
0.10	84.96	85.62	85.25	Fixed
0.25	82.64	83.16	83.47	Adaptive
0.50	77.00	77.50	77.28	Fixed
Mean	82.88	83.26	83.19	Fixed

Appendix

Table 12:

\Phi

-guided adaptive vs. fixed label smoothing (0.1). Bold = best per row.

Dataset	Arch	sf	Seed	Final acc	$\delta$ (percentage points)
CIFAR-100	ResNet	0.5	42	0.3842	$+10.0$
CIFAR-100	ResNet	0.5	123	0.3904	$+11.0$
FashionMNIST	MLP	0.5	42	0.7265	$+5.0$
FashionMNIST	MLP	0.5	123	0.7263	$+3.6$
FashionMNIST	MLP	0.5	456	0.7285	$+2.5$
MNIST	MLP	0.5	42	0.6804	$+3.3$

Appendix

Table 13: TGCS results (representative subset of 40 runs).

\delta=100\max(0,a_{\mathrm{peak}}-a_{\mathrm{final}})

, where

a_{\mathrm{peak}}

and

a_{\mathrm{final}}

are test accuracies on a 0–1 scale at the

\Phi

-peak and final checkpoints, respectively. Reported clean-data and ViT-collapsed entries have

\delta=0

sf	Per-class $\Phi$	Global ( $n{=}300$ )	Global ( $n{=}1000$ )	Epoch
0.00	260.3	32.1	118.8	10
0.25	143.0	18.5	60.3	10
0.50	118.5	13.5	48.9	10
1.00	0.0	0.0	0.0	10
0.00	403.4	66.1	229.0	50
0.25	160.5	25.8	82.3	50

Appendix

Table 14: ViT-Tiny/CIFAR-10: per-class

\Phi

by noise level (mean over 5 seeds). The classwise sum is larger than the global sum and has noise correlation

\rho\leq-0.93

; unequal point budgets preclude interpreting the magnitude ratio as diagnostic improvement.

sf	Global ( $n{=}300$ )	Per-class	Val Acc	Epoch
0.00	84.4	4,035.2	31.0%	10
0.25	82.3	3,311.4	25.4%	10
0.50	54.3	2,554.4	19.2%	10
1.00	0.6	26.5	0.5%	10
0.00	165.6	5,332.6	50.3%	50
0.25	94.5	4,638.8	35.8%	50

Appendix

Table 15: Tiny-ImageNet/ResNet-18: global vs. per-class

\Phi

by noise level (mean over 5 seeds). The larger classwise aggregate discriminates noise with

\rho=-0.969

(

p<10^{-12}

) at epochs 10 and 25.

Epoch	$\mathrm{sf}=0$ (clean)	$\mathrm{sf}=0.25$	$\mathrm{sf}=0.5$
1	7.4	2.6	0.3
5	28.2	16.7	6.0
10	28.2	16.8	10.7
25	29.5	16.5	12.8
35	32.9	17.1	12.5
50	31.7	18.6	16.4

Appendix

Table 16: Raw persistence on CIFAR-10/ResNet-34, averaged over three seeds per noise setting. The clean trajectory motivated a no-crossover prediction before noisy training began. All six noisy runs remained below the clean reference at every sampled checkpoint.

Epoch	Rate	Asymmetric $\Phi$	Instance $\Phi$	Ratio	$t$ -test $p$
10	0.2	$821.1\pm 42.3$	$796.7\pm 43.7$	0.97	0.395
10	0.4	$777.9\pm 32.9$	$774.8\pm 31.1$	1.00	0.885
25	0.2	$660.5\pm 58.3$	$610.9\pm 24.5$	0.92	0.117
25	0.4	$662.0\pm 28.9$	$711.1\pm 37.1$	1.07	0.048
50	0.2	$426.6\pm 29.5$	$505.4\pm 27.1$	1.18	0.002
50	0.4	$485.8\pm 29.5$	$717.3\pm 49.0$	1.48	$\mathbf{<0.001}$

Appendix

Table 17: Per-class

\Phi

on ViT-Tiny under asymmetric vs. instance-dependent noise (mean

\pm

std over 5 seeds). The two noise types produce no statistically detectable difference in raw persistence at epoch 10 (

p>0.3

) but different raw persistence at epoch 50 (

p\leq 0.002

Epoch	sf	Global ( $n{=}300$ )	Per-class	Global ( $n{=}1000$ )	Noisy/Clean Ratio
10	0.00	18.1	103.7	56.1	—
10	0.50	53.1	371.4	224.5	$2.9\times$ / $3.6\times$ / $4.0\times$
25	0.00	15.7	83.0	43.9	—
25	0.50	56.7	354.2	178.3	$3.6\times$ / $4.3\times$ / $4.1\times$
50	0.00	12.2	77.6	40.2	—
50	0.50	45.3	285.4	143.8	$3.7\times$ / $3.7\times$ / $3.6\times$

Appendix

Table 18: DistilBERT/AG News:

\Phi

at clean vs. sf

=

0.5 (mean over 5 seeds). All three

\Phi

variants show

\rho=+0.870

(

p<0.002

) with a consistent sign reversal compared to vision: noise raises raw total persistence; scale and loop count are not separated by this table.

Epoch	sf	Global $\Phi$	Per-class $\Phi$	Val Acc	$n$
10	0.00	130.5	982.3	59.4%	5
10	0.25	79.6	731.2	52.6%	5
10	0.50	62.5	606.7	44.8%	5
25	0.00	129.4	911.2	68.3%	5
25	0.25	73.6	599.9	63.0%	5
25	0.50	71.8	663.4	55.5%	5

Appendix

Table 19: Small Transformer on CIFAR-10:

\Phi

by noise level and epoch (mean over 5 seeds,

n=15

total). Noise decreases

\Phi

at all epochs tested, matching the vision pattern on ResNet/ViT, opposite to DistilBERT/text.

Figure 5: Exploratory class-pair comparison on CIFAR-10/ResNet-18. Set A was selected using a high noisy/clean persistence ratio; Set B contains comparison pairs. Bars show means, error bars show sample standard deviations, and dots show individual runs across ten seeds per condition. Reported

p

-values use Welch’s two-sample

t

-test. The nominal pairwise flip parameter is 0.5; sequential flips over overlapping pairs change approximately 42.5% of labels, unevenly across classes. These conditions are distinct from the disjoint, centroid-selected NEAR and FAR comparison.

Pipeline	Pairs	Epoch 10	Epoch 25	Epoch 50
CIFAR-10 / ResNet-18	45	$-0.471$	0.823	0.867
CIFAR-10 / ViT-Tiny	45	0.104	0.664	0.614
CIFAR-100 / ResNet-18	60	$-0.172$	$-0.119$	0.413

Appendix

Table 20: Pair-distance correlations reported in the response-period analysis. Both CIFAR-10 configurations have

p<0.0001

at epochs 25 and 50. Correlations are computed across pairs after seed averaging at

sf=0.5

Noisy-Label Learning

Noisy-label learning develops ways to train and evaluate models when some supervision is incorrect, ambiguous, or unreliable.

See all 93 papers

May 7, 2026

Topological Signatures of Grokking

We study the grokking phenomenon through the lens of topology. Using persistent homology on point clouds derived from the embedding matrices of a range of models trained on modular arithmetic with varying primes, we identify a clear and consistent topological signature of grokking: a sharp increase in both the maximum and total persistence of first homology (

H_1

). Persistence diagrams reveal the emergence of a dominant long-lived topological feature together with increasingly structured secondary features, reflecting the underlying cyclic structure of the task. Compared to existing spectral and geometric diagnostics -- specifically, Fourier analysis and local intrinsic dimension -- persistent homology provides a unified geometric and topological characterization of representation learning, capturing both local and global multi-scale structure. Ablations across data regimes and control settings show that these topological transitions are tied to generalization rather than memorization. Our results suggest that persistent homology offers a principled and interpretable framework for analyzing how neural networks internalize latent structure during training.

Yifan Tang, Qiquan Wang, Inés García-Redondo +1

Imperial College London · Queen Mary University of London · University of Fribourg

Neural Network Generalization Neural Representation Geometry Grokking

Ask PDF

Jul 20, 2026

Leveraging Dissimilarity Invariance as a Robust Anchor for Learning with Noisy Labels

Deep learning models excel in visual recognition but suffer severe performance drops when training labels are corrupted by noise. Under label noise prior work cannot learn accurate similarities and thus misguide the learning process. In this paper, we uncover a complementary and novel phenomenon, Dissimilarity Invariance, whereby semantic dissimilarity between unrelated samples remains stable despite label noise. Leveraging this insight, we propose NegScale, a plug-and-play framework that shifts focus from fragile similarity to robust dissimilarity. NegScale integrates: (1) Structured Negative Orthogonality Penalty (SNOP), enforcing subspace orthogonality for unrelated samples; and (2) Dissimilarity-Calibrated Similarity Adjustment (DCSA), suppressing spurious similarity using dissimilarity anchors. We also give theoretical analysis that proves Dissimilarity Invariance and the effectiveness of NegScale. Empirical results demonstrate that NegScale consistently outperforms state-of-the-art baselines, establishing new benchmarks on CIFAR with synthetic noise and real-world datasets.

Wenxiao Fan, Kan Li

School of Computer Science, Beijing Institute of Technology

Metric Learning Learning with Noisy Labels

Ask PDF

Jun 9, 2026

Neural networks become centroid classifiers when trained on heavily corrupted inputs

Learning from imperfect data is a central theme in machine learning, connecting practical questions of robustness to fundamental questions of learnability. Here we examine attribute noise: learning from corrupted inputs while keeping the labels intact, a setting that has received considerably less analytical attention than its label-noise counterpart. We consider two types of corruption models: additive noise and replacement noise. Through experiments with multi-layer perceptrons (MLPs) on corrupted classification datasets, we find that neural networks remain robust, maintaining well-above-chance accuracy even when inputs are >90% corrupted -- far beyond human recognition. To understand this robustness, we analyze infinite-width networks in the heavy-corruption regime using a mean-field-inspired approach and derive a leading-order decision rule for the classification outcome: the network implements a prototype rule, the nearest-class-mean, assigning each test point to the class whose training-set average it most closely resembles. This leading-order decision rule is universal across a broad range of MLP architectures, holding for any depth, as well as a wide class of activation functions and noise distributions. The same centroid mechanism closely matches finite-width network behavior in our experiments and provides an interpretable and analytically tractable account of why learning can succeed even when individual training examples carry almost no signal.

Justin Tahmassebpur, Asadullah Bhuiyan, Hyejin Kim +1

Department of Applied and Engineering Physics, Cornell University, Ithaca, NY 14853, USA · Department of Physics, Cornell University, Ithaca, NY 14853, USA

Neural Network Generalization Multilayer Perceptrons Prototype-Based Classification Neural Network Robustness Corruption Robustness Classification+1

Ask PDF

Abstract

Noisy-Label Learning

Explore similar work

Topological Signatures of Grokking

Leveraging Dissimilarity Invariance as a Robust Anchor for Learning with Noisy Labels

Neural networks become centroid classifiers when trained on heavily corrupted inputs