Riemannian Difference-of-Convex Optimization for K-Means Clustering
Organizations: AMSS, Chinese Academy of Sciences, and University of Chinese Academy of Sciences, Beijing, China · Ministry of Education Key Laboratory of NSLSCS, School of Mathematical Sciences, Nanjing Normal University, Nanjing, China · Ministry of Education Key Laboratory of Mathematics and Information Networks, School of Mathematical Sciences, Beijing University of Posts and Telecommunications, Beijing, China · Department of Systems Engineering and Engineering Management, The Chinese University of Hong Kong, HKSAR, China
Abstract
K-means is a widely adopted clustering approach in signal processing and machine learning. In this paper, we study K-means clustering through a cardinality-constrained formulation on a compact embedded submanifold. We replace the cardinality constraint with a difference-of-convex (DC) penalty and establish a global error bound to prove that the penalized and constrained formulations share the same global minimizers whenever the penalty parameter exceeds a finite threshold. To solve the resulting nonsmooth Riemannian DC problem, we reformulate it as a minimax problem and propose RADA-DC, a Riemannian alternating descent ascent method combining dual regularization with DC linearization. Under standard assumptions and suitable parameter choices, RADA-DC finds an -Riemannian critical point within iterations. We conduct experiments on synthetic and real-world datasets to demonstrate that the proposed method outperforms the tested baselines, including K-means++, in solution quality at competitive computational cost when the number of clusters is large.
Figures & tables
| Method | Err. (%) | Time (s) | ||
|---|---|---|---|---|
| 200 (500) | I-AManPG | 22.35 (40.30) | 33.24 (16.14) | 0.83 (3.65) |
| RADA-DC | 22.60 ( 41.18 ) | 31.95 ( 11.77 ) | 0.62 ( 1.49 ) | |
| KM++ | 21.75 (39.97) | 37.84 (15.98) | 2.75 (4.00) | |
| 240 (550) | I-AManPG | 21.87 (40.09) | 33.18 (17.03) | 0.91 (4.97) |
| RADA-DC | 22.00 ( 41.21 ) | 32.48 ( 11.99 ) | 0.67 ( 1.68 ) | |
| KM++ | 21.22 (40.08) | 36.84 (15.29) | 2.85 (4.03) |