cs.LGOct 24, 2026

Stable Dimensionality Reduction via Isometric Centroid-Encoder

Authors: Tomojit Ghosh, Siva Rama Krishna Prasad Changala, Prathyusha Kanakamalla, Harigovind Harikumar

Organizations: Colonel Glenn Hwy, Dayton, OH 45435-0001, USA Department of Computer Science and Engineering Wright State University

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Many supervised dimensionality reduction methods promote intra-class compactness and inter-class separation but do not explicitly constrain class geometry, leading to variability in the relative arrangement of classes across independent runs. Such variability limits the reproducibility and interpretability of low-dimensional embeddings. We propose the Isometric Centroid-Encoder (ICE), a supervised dimensionality reduction framework in which class centroids are learned directly in the latent space while preserving a prescribed pairwise class-centroid geometry. Unlike existing centroid- or prototype-based methods that primarily encourage intra-class compactness, ICE explicitly regularizes the relative arrangement of latent class centroids. Unlike multidimensional scaling and many metric-learning approaches that preserve sample-level relationships, ICE operates on class centroids, preserving inter-class geometry without requiring large numbers of sample-pair or triplet constraints. We further show that the intrinsic dimensionality required to represent the centroid-distance configuration of MM classes is at most M−1M-1, explaining the observed saturation of distance-preservation measures as the latent dimensionality increases. To quantify embedding reproducibility, we introduce the Centroid Neighborhood Stability (CNS) metric, which measures the consistency of class-level neighborhood topology across independent runs and complements existing embedding quality measures. Experiments on image and biological datasets demonstrate that ICE produces stable embeddings with improved preservation of class-centroid geometry while remaining computationally efficient. Additional analyses of optimization dynamics, sensitivity, and preprocessing further provide practical insights into latent dimensionality and explicit geometry modeling for supervised dimensionality reduction.

Figures & tables

Figure 1: Illustration of Isometric Centroid-Encoder (ICE). Samples are dispersed around class centroids in 3D, with pairwise distances encoding class geometry. ICE maps data to a 2D space by collapsing samples to centroids while preserving these distances, producing compact classes with retained geometric structure.
PropertyCentroid-Encoder (CE)ICE
Centroid locationFixed in ambient spaceLearned in latent space
Target mappingxi→cj∈Rdx_{i}\rightarrow c_{j}\in\mathbb{R}^{d}xi→μj∈Rpx_{i}\rightarrow\mu_{j}\in\mathbb{R}^{p}
Centroid updatePre-computedUpdated during training
Embedding constraintNot enforcedIntra-class compactness + centroid isometry
Global class geometryNot enforcedExplicitly preserved
Objective perspectiveCentroid reconstructionCentroid encoding + geometry preservation
Table 2: Comparison between Centroid-Encoder (CE) and the proposed ICE framework.
Figure 2: Optimization dynamics of ICE on the MNIST dataset. (a) Latent centroid trajectories in the two-dimensional embedding space. Circles and squares denote the initial and final centroid locations, respectively. Intermediate points illustrate the centroid evolution during training. (b) Mean epoch-to-epoch centroid drift averaged across all classes. The shaded region indicates one standard deviation across classes.
Figure 3: Optimization dynamics of ICE on the FMNIST dataset. (a) Evolution of the latent class centroids in the two-dimensional embedding space throughout training. (b) Mean epoch-to-epoch centroid drift averaged across all classes. The shaded region denotes one standard deviation across classes.
Figure 4: Evolution of the centroid-consistency loss, centroid-isometry loss, and total objective during ICE optimization on FMNIST and MNIST using direct two-dimensional embeddings and class-dimensional embeddings ( p=Mp=M ). The total objective is computed as Lcons+λLiso\mathcal{L}_{\mathrm{cons}}+\lambda\mathcal{L}_{\mathrm{iso}} with λ=0.01\lambda=0.01 , while the isometry loss is shown in its unweighted form. The plots show that the isometry loss quickly plateaus in the two-dimensional embedding but continues to decrease in the higher-dimensional latent space, consistent with the additional representational capacity available for preserving centroid geometry.
Figure 5: Representative comparison from the controlled synthetic experiment designed to distinguish Procrustes Disparity and CNS. Two configurations with identical embedding coordinates but different class-label assignments are compared. (a) Reference configuration. (b) Configuration obtained by permuting the labels assigned to entire clusters while keeping all sample coordinates fixed. (c) Corresponding Procrustes Disparity and CNS values. (d) k=5k=5 nearest-centroid neighborhood graphs illustrating how class-label permutations alter class-neighborhood relationships despite identical embedding geometry. Procrustes Disparity remains zero because the geometric configuration is unchanged, whereas CNS captures the resulting change in class-level neighborhood topology.
Dataset#Features#Classes#SamplesDomain
USPS256109298Image
MNIST7841060000Image
FMNIST7841060000Image
10X PBMC1665384271RNA-Seq
Mouse Bladder Cells20670162746RNA-Seq
Worm Neuron Cells13488104186RNA-Seq
Table 3: Descriptions of datasets used for benchmarking.
Figure 6: Kruskal Stress and Pearson Correlation as functions of latent dimensionality for USPS, Mouse Bladder, Worm Neuron, and 10xPBMC using the deep ICE architecture (1000, 500, and 125 hidden units). Results are averaged over five independent runs. The red vertical line denotes the number of classes MM .
Figure 7: Kruskal Stress and Pearson Correlation of class-centroid distance preservation for USPS, Mouse Bladder, Worm Neuron, and 10xPBMC using linear ICE (no hidden layers). The latent dimension is varied from 2 to 26, and results are averaged over five independent runs. The red vertical line indicates the number of classes MM .
Figure 8: Kruskal Stress and Pearson Correlation of class-centroid distance preservation for USPS, Mouse Bladder, Worm Neuron, and 10xPBMC using shallow ICE (one hidden layer with 500 neurons). The latent dimension ranges from 2 to 26, and results are averaged over five independent runs. The red vertical line denotes the number of classes MM .
Figure 9: Normalized eigenspectra of the centered class-centroid matrix for (a) Mouse Bladder (16 cell types) and (b) Worm Neuron (10 cell types). The eigenvalues are shown in descending order after normalization by their total sum. Both datasets exhibit a rapidly decaying eigenspectrum, indicating that the dominant centroid geometry is captured by only a small number of principal directions. This suggests that the effective latent dimensionality required to preserve the class-centroid geometry can be substantially smaller than the theoretical upper bound of M−1M-1 . The eigenspectra were computed using Dual PCA to efficiently perform the eigendecomposition in the sample space, avoiding the construction of the high-dimensional feature covariance matrix.
λ\lambdaCNSKruskal Stress5NN Accuracy (%)
0.010.80800.2243±0.00110.2243\pm 0.001197.27±0.0597.27\pm 0.05
0.10.89330.2258±0.00050.2258\pm 0.000596.94±0.0996.94\pm 0.09
1.00.84000.2253±0.00060.2253\pm 0.000696.39±0.2096.39\pm 0.20
10.00.77870.2268±0.00350.2268\pm 0.003596.48±0.0896.48\pm 0.08
100.00.75730.2378±0.01000.2378\pm 0.010095.93±0.1295.93\pm 0.12
Table 6: Effect of the regularization parameter λ\lambda on centroid neighborhood stability (CNS), Kruskal Stress, and 5NN accuracy for MNIST in a two-dimensional embedding. Results are averaged over five independent runs.
λ\lambdaCNSKruskal Stress5NN Accuracy (%)
0.010.87470.2086±0.00350.2086\pm 0.003596.24±0.2296.24\pm 0.22
0.10.81600.2164±0.01180.2164\pm 0.011896.15±0.3196.15\pm 0.31
1.00.89330.2085±0.0036\mathbf{0.2085\pm 0.0036}96.09±0.2596.09\pm 0.25
10.00.78930.2147±0.00750.2147\pm 0.007596.80±0.1496.80\pm 0.14
100.00.80270.2137±0.00880.2137\pm 0.008895.28±0.4095.28\pm 0.40
Table 7: Effect of the regularization parameter λ\lambda on USPS. Results are averaged over five independent runs using an 80:2080{:}20 train-test split.
λ\lambdaCNSKruskal Stress5NN Accuracy (%)
0.010.87000.1693±0.01280.1693\pm 0.012882.61±1.4282.61\pm 1.42
0.10.89670.1621±0.00650.1621\pm 0.006581.87±2.7181.87\pm 2.71
1.00.91000.1655±0.00330.1655\pm 0.003380.98±1.2780.98\pm 1.27
10.00.91330.1580±0.00650.1580\pm 0.006583.24±1.4283.24\pm 1.42
100.00.90000.1515±0.0044\mathbf{0.1515\pm 0.0044}78.79±1.8778.79\pm 1.87
Table 8: Effect of the regularization parameter λ\lambda on the 10X PBMC dataset. Results are averaged over five independent runs using an 80:2080{:}20 train-test split.
DatasetLatent Dim.StressCorrelationCNS5NN Accuracy (%)
MNIST221.00±0.001.00\pm 0.00−0.00±0.19-0.00\pm 0.190.46±0.010.46\pm 0.0114.12±3.6814.12\pm 3.68
MNIST10101.00±0.001.00\pm 0.000.00±0.120.00\pm 0.120.48±0.010.48\pm 0.019.91±0.399.91\pm 0.39
FMNIST221.00±0.001.00\pm 0.000.13±0.320.13\pm 0.320.50±0.020.50\pm 0.0220.65±6.1320.65\pm 6.13
FMNIST10101.00±0.001.00\pm 0.00NA0.77±0.040.77\pm 0.0410.00±0.0110.00\pm 0.01
Table 9: Ablation results obtained by removing the centroid-isometry term ( λ=0\lambda=0 ). Lower Kruskal Stress is better, whereas higher Correlation, CNS, and 5NN accuracy are better.
DatasetLatent Dim.Kruskal StressCorrelationCNS5NN Accuracy (%)
MNIST220.23±0.010.23\pm 0.010.83±0.030.83\pm 0.030.72±0.020.72\pm 0.0294.26±0.3294.26\pm 0.32
FMNIST220.15±0.020.15\pm 0.020.94±0.030.94\pm 0.030.95±0.020.95\pm 0.0284.69±0.4184.69\pm 0.41
10X PBMC220.16±0.010.16\pm 0.010.94±0.010.94\pm 0.010.91±0.030.91\pm 0.0385.55±2.1085.55\pm 2.10
Mouse Bladder220.20±0.010.20\pm 0.010.94±0.000.94\pm 0.000.77±0.020.77\pm 0.0265.71±1.6965.71\pm 1.69
MNISTMM0.01±0.000.01\pm 0.001.00±0.001.00\pm 0.000.95±0.020.95\pm 0.0297.39±0.1097.39\pm 0.10
FMNISTMM0.02±0.010.02\pm 0.011.00±0.001.00\pm 0.000.99±0.010.99\pm 0.0187.83±0.1787.83\pm 0.17
Table 10: Performance of isometry-only ICE after removing the centroid-consistency term. Results are averaged over 15 independent runs.
kkRRCNS ranking (higher is better)
13IVIS (0.8667) >> ICE (0.7867) >> SE (0.7778) >> CE (0.6444) >> SUMAP (0.6044)
15IVIS (0.8507) >> ICE (0.7600) >> SE (0.7387) >> CE (0.5680) >> SUMAP (0.5627)
115IVIS (0.8267) >> ICE (0.6853) >> SE (0.6836) >> SUMAP (0.4996) >> CE (0.4844)
33IVIS (0.9111) >> SE (0.8281) >> ICE (0.7793) >> CE (0.7689) >> SUMAP (0.7437)
35IVIS (0.8969) >> SE (0.7796) >> ICE (0.7422) >> SUMAP (0.7271) >> CE (0.7067)
315IVIS (0.8498) >> SE (0.7390) >> SUMAP (0.6942) >> ICE (0.6833) >> CE (0.6361)
Table 11: Sensitivity of CNS to neighborhood size kk and number of runs RR for the MNIST two-dimensional embedding. Methods are ranked according to their average CNS for each (k,R)(k,R) configuration.
kkRRCNS ranking (higher is better)
13ICE (0.9911) >> SE (0.9422) >> IVIS (0.9289) >> CE (0.7733) >> SUMAP (0.6222)
15ICE (0.9893) >> SE (0.9360) >> IVIS (0.9067) >> CE (0.7173) >> SUMAP (0.5547)
115ICE (0.9852) >> SE (0.9262) >> IVIS (0.8927) >> CE (0.6539) >> SUMAP (0.4793)
33ICE (0.9674) >> IVIS (0.9111) >> SE (0.8993) >> CE (0.8267) >> SUMAP (0.7748)
35ICE (0.9609) >> IVIS (0.8969) >> SE (0.8791) >> CE (0.7813) >> SUMAP (0.7271)
315ICE (0.9495) >> IVIS (0.8498) >> SE (0.8407) >> CE (0.7146) >> SUMAP (0.6942)
Table 12: Sensitivity of CNS to neighborhood size kk and number of runs RR for the MNIST class-dimensional embedding ( p=Mp=M ). Methods are ranked according to their average CNS for each (k,R)(k,R) configuration.
Figure 10: Two-dimensional scatter plots of MNIST test digits across five independent runs for each method.
Figure 11: Two-dimensional scatter plots of MNIST test digits across five independent runs for each method. The 2D embeddings are obtained by applying PCA to the corresponding MM -dimensional representations.
Figure 12: Two-dimensional scatter plots of FMNIST fashion item from test set across five independent runs for each method.
Figure 13: Two-dimensional scatter plots of FMNIST fashion item from test set across five independent runs for each method. The 2D embeddings are obtained by applying PCA to the corresponding MM -dimensional representations.
Figure 14: Two-dimensional scatter plots of 10XPBMC test samples across five independent runs for each method.
Figure 15: Two-dimensional scatter plots of 10XPBMC test set across five independent runs for each method. The 2D embeddings are obtained by applying PCA to the corresponding MM -dimensional representations.
More in this topic

Dimensionality Reduction

Dimensionality reduction maps high-dimensional data to compact representations for visualization, prediction, compression, and scientific modeling.

See all 145 papers

Explore similar work

CardsList

  1. MEDAL: Manifold Embedding Distillation via Autoencoder Learning

    May 22, 2026Irene Chang, Tarek M. Zikry, Genevera I. AllenAutoencodersManifold Learning

    Department of Statistics, Columbia University, New York, NY 10027 · School of Data and Information Sciences, University of North Carolina at Chapel Hill, Chapel Hill, NC 27599 · Irving Institute for Cancer Dynamics, Columbia University, New York, NY 10027 +1

  2. Beyond Correlation: Learning Supervised, Sample-Distinct, and Eigenimage-Interpretable Representations

    Jul 22, 2025Mojtaba MoattariRepresentation LearningDimensionality Reduction

    Independent Researcher

  3. Shape Operator PCA: Curvature-Aware Projections for Geometric Machine Learning

    Aug 15, 2026Alexandre L. M. LevadaDimensionality ReductionPrincipal Component Analysis

    Federal University of São Carlos 13565-905, São Carlos-SP, Brazil