Infinity-Wasserstein Distributionally Robust Principal Component Analysis: Exact Reformulation, Finite-Sample Guarantee, and Efficient Algorithm
Authors: Zhouxing Luo, Lei Wang, Xin Liu
Organizations: Institute of Computational Mathematics and Scientific/Engineering Computing, Academy of Mathematics and Systems Science, Chinese Academy of Sciences, Beijing, China · Department of Applied Mathematics, The Hong Kong Polytechnic University, Hung Hom, Kowloon, Hong Kong, China
Principal component analysis (PCA) is a fundamental and ubiquitous technique for dimension reduction and representation learning, yet its empirical formulation remains vulnerable to distributional uncertainty. While distributionally robust optimization (DRO) offers a natural remedy, existing order-2 Wasserstein models of PCA either preserve the nominal PCA subspace or lack an exact reformulation. This paper demonstrates how the infinity-Wasserstein distance can reconcile these two desirable properties. For the resulting DRO model of PCA, we derive an exact finite-dimensional reformulation and explicitly identify a worst-case distribution attaining the inner maximum. Crucially, the reformulated objective retains samplewise residual information beyond the empirical covariance matrix, revealing a mechanism through which distributional robustness can reshape the learned principal subspace. Under the mild assumption that the true distribution has a finite fourth moment, we further establish a uniform finite-sample performance guarantee for the proposed model. An iteratively reweighted majorization-minimization algorithm is also developed to solve the resulting nonconvex and nonsmooth optimization problem, with a rigorous analysis of its convergence properties. Numerical experiments substantiate the computational efficiency of the proposed algorithm and the practical effectiveness of the infinity-Wasserstein model in downstream applications.
Figures & tables
Figure 1: Comparison of IRWPCA, DSGM, and RSG for solving the InfWPCA model.
Figure 2: Reconstruction MSE and classification accuracy on clean test samples. Smaller MSE and larger accuracy indicate better performance.
Model
SR ↑
AMR ↑
AV ↓
MD ↓
InfWPCA
0.835
11.435%
13.700%
38.418%
PCA
0.816
11.336%
13.886%
41.811%
W2PCA-res
0.760
10.473%
13.786%
40.447%
W2PCA-pca
0.783
11.145%
14.233%
38.642%
Table 1: Averaged values of the four performance metrics for the tested models over twelve update frequencies. The best results are highlighted in boldface.
Figure 3: Four performance metrics of the tested models under different update intervals.
Principal component analysis (PCA) finds orthogonal directions of greatest variance for data compression, visualization, denoising, and feature discovery.