Joint Representation Learning and Clustering via Gradient-Based Manifold Optimization

Sida Liu; Yangzi Guo; Mingyuan Wang

stat.MLApr 15, 2026

Joint Representation Learning and Clustering via Gradient-Based Manifold Optimization

Authors: Sida Liu, Yangzi Guo, Mingyuan Wang

Organizations: Department of Statistics, Florida State University, Tallahassee, Florida, USA · Department of Mathematics, Florida State University, Tallahassee, Florida, USA

Abstract

Clustering and dimensionality reduction have been crucial topics in machine learning and computer vision. Clustering high-dimensional data has been challenging for a long time due to the curse of dimensionality. For that reason, a more promising direction is the joint learning of dimension reduction and clustering. In this work, we propose a Manifold Learning Framework that learns dimensionality reduction and clustering simultaneously. The proposed framework is able to jointly learn the parameters of a dimension reduction technique (e.g. linear projection or a neural network) and cluster the data based on the resulting features (e.g. under a Gaussian Mixture Model framework). The framework searches for the dimension reduction parameters and the optimal clusters by traversing a manifold,using Gradient Manifold Optimization. The obtained The proposed framework is exemplified with a Gaussian Mixture Model as one simple but efficient example, in a process that is somehow similar to unsupervised Linear Discriminant Analysis (LDA). We apply the proposed method to the unsupervised training of simulated data as well as a benchmark image dataset (i.e. MNIST). The experimental results indicate that our algorithm has better performance than popular clustering algorithms from the literature.

Figures & tables

Fig. 1: Illustration of the gradient-based manifold optimization framework.

\Delta_{t}=-\nabla_{\mathbf{u}}g(\mathbf{u}_{t})H(\mathbf{u}_{t},{\boldsymbol{\theta}}_{t})^{-1}\frac{\partial^{2}E}{\partial\mathbf{u}\partial{\boldsymbol{\theta}}}(\mathbf{u}_{t},{\boldsymbol{\theta}}_{t})

(11)

Algorithm 1 Gradient Based Manifold Optimization Algorithm (GMOA)

Fig. 2: Surface and manifold plot of $(\Delta_{\boldsymbol{\mu}},\theta,E(\mathbf{u},{\boldsymbol{\theta}}))$ for 2D Gaussians.

Fig. 3: Trajectory of our algorithm in 2D.

Fig. 4: Example of data with noise in 2D.

Panel of Figure 6 — Fig. 5: Comparison of the proposed algorithm with other popular clustering algorithms on 2D noisy data.

Fig. 6: Surface and manifold plot of $(\Delta_{\boldsymbol{\mu}},{\boldsymbol{\theta}},E(\mathbf{u},{\boldsymbol{\theta}}))$ of 3D Gaussian (trajectory plots from three different ${\boldsymbol{\theta}}$ initializations (3.0,3.0), (2.5,2.5), (1.6,1,6))

	K-means	EM	Spectral Clust.	DBSCAN	Our (GMOA)
3 digits (3, 5, 9)
Training Accuracy (%)	93.55 (29.38)	88.44 (46.17)	97.15	95.04	97.42
Test Accuracy (%)	90.45 (30.64)	88.33 (31.40)	92.23	89.21	96.43
10 digits
Training Accuracy (%)	83.33 (54.32)	75.39 (38,77)	81.99	85.30	91.45
Test Accuracy (%)	71.72 (53.37)	70.00 (42.69)	74.11	78.77	87.83

TABLE I: Accuracy of clustering algorithms on the original MNIST dataset (projection to 2D. The results in bracket are the results on original

784D

which are not applicable to Spectral Clustering and DBSCAN due to limit of memory).

	K-means	EM	Spect. Clust.	DBSCAN	Our (GMOA)
3 digits (3, 5, 9)
Training Accuracy (%)	97.58 (75.35)	97.45 (62.40)	98.88	97.88	99.10
Test Accuracy (%)	96.63 (80.63)	96.55 (83.62)	97.62	97.56	98.69
10 digits
Training Accuracy (%)	96.25 (78.51)	96.30 (62.65)	96.38	97.18	98.24
Test Accuracy (%)	96.00 (83.64)	95.99 (86.13)	88.98	96.10	96.91

TABLE II: Accuracy of clustering algorithms on the MNIST dataset using the SimCLR representation (projection to 10D, the results in bracket are the results on 256D which are not applicable to Spectral Clustering and DBSCAN due to memory limitations).

	GAN	DEC	Deep Cluster	DynAE	Our (GMOA)
10 digits
Test Accuracy (%)	82.8	84.3	65.6	84.5	87.9

TABLE III: Test accuracy of clustering algorithms on the MNIST dataset using a shallow neural network structure.

Unsupervised Clustering

Unsupervised clustering groups unlabeled data by similarity or structure to reveal patterns, subgroups, and anomalies.

See all 117 papers

Shape Operator PCA: Curvature-Aware Projections for Geometric Machine Learning
Aug 15, 2026Alexandre L. M. LevadaDimensionality Reduction Principal Component Analysis
Federal University of São Carlos 13565-905, São Carlos-SP, Brazil
Intrinsic Green's Learning: Supervised Learning on Manifolds via Inverse PDE
Jul 8, 2026Alexandre QuemyManifold Learning
Hother Labs
Fair Model-based Clustering
Feb 25, 2026Jinwon Park, Kunwoong Kim, Jihu Lee +1Clustering Group Fairness
Graduate School of Data Science, Seoul National University · Department of Statistics, Seoul National University

Aug 15, 2026J/K move · Enter open · S save

Shape Operator PCA: Curvature-Aware Projections for Geometric Machine Learning

Alexandre L. M. Levada

Federal University of São Carlos 13565-905, São Carlos-SP, Brazil

Dimensionality Reduction Principal Component Analysis Metric Learning

Open paper AskPDF arXiv

In this paper, we propose SHOPCA (Shape Operator-based Principal Component Analysis), a novel method for unsupervised metric learning and dimensionality reduction that incorporates differential geometric information into the covariance structure of classical PCA. SHOPCA regularizes the global covariance matrix using the mean shape operator, defined as the average of the absolute local shape operators estimated from the data manifold, steering principal components toward directions of both maximum variance and informative curvature. A single trace-normalized mixing coefficient

α

controls the regularization, recovering standard PCA at

α= 0

and a curvature-driven embedding as

α\to \infty

. We further introduce a fully unsupervised criterion for selecting

α

based on the spectral eigengap of the regularized covariance matrix, maximizing the relative separation between the top-

d

and remaining eigenvalues without using class labels. We evaluate SHOPCA on more than 50 real-world benchmark datasets, comparing it with PCA, ISOMAP, and UMAP using Adjusted Rand Index (ARI), Normalized Mutual Information (NMI), Fowlkes-Mallows index (FM), and V-measure. Results show that SHOPCA consistently improves clustering quality over PCA across a broad range of datasets and surpasses UMAP on small-sample settings, where iterative neighborhood-based manifold estimation can degrade. SHOPCA is computationally tractable, parameter-efficient, and applicable to domains requiring fully unsupervised, geometry-aware dimensionality reduction.

Abstract

Unsupervised Clustering

Explore similar work

Shape Operator PCA: Curvature-Aware Projections for Geometric Machine Learning

Intrinsic Green's Learning: Supervised Learning on Manifolds via Inverse PDE

Fair Model-based Clustering