cs.LGApr 15, 2026

Metric-Aware Principal Component Analysis (MAPCA):A Unified Framework for Scale-Invariant Representation Learning

Authors: Michael Leznik

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

We introduce Metric-Aware Principal Component Analysis (MAPCA), a unified framework for scale-invariant representation learning based on the generalised eigenproblem max Tr(W^T Sigma W) subject to W^T M W = I, where M is a symmetric positive definite metric matrix. The choice of M determines the representation geometry. The canonical beta-family M(beta) = Sigma^beta, beta in [0,1], provides continuous spectral bias control between standard PCA (beta=0) and output whitening (beta=1), with condition number kappa(beta) = (lambda_1/lambda_p)^(1-beta) decreasing monotonically to isotropy. The diagonal metric M = D = diag(Sigma) recovers Invariant PCA (IPCA), a method rooted in Frisch (1928) diagonal regression, as a distinct member of the broader framework. We prove that scale invariance holds if and only if the metric transforms as M_tilde = CMC under rescaling C, a condition satisfied exactly by IPCA but not by the general beta-family at intermediate values. Beyond its classical interpretation, MAPCA provides a geometric language that unifies several self-supervised learning objectives. Barlow Twins and ZCA whitening correspond to beta=1 (output whitening); VICReg's variance term corresponds to the diagonal metric. A key finding is that W-MSE, despite being described as a whitening-based method, corresponds to M = Sigma^{-1} (beta = -1), outside the spectral compression range entirely and in the opposite spectral direction to Barlow Twins. This distinction between input and output whitening is invisible at the level of loss functions and becomes precise only within the MAPCA framework.

Figures & tables

β\betaM(β)M(\beta)κ(β)\kappa(\beta)
0.00II65.65
0.25Σ0.25\Sigma^{0.25}23.06
0.50Σ0.5\Sigma^{0.5}8.10
0.75Σ0.75\Sigma^{0.75}2.85
1.00Σ\Sigma1.00
Table 1: Condition number κ(β)\kappa(\beta) of the effective operator as a function of β\beta , army cadets dataset.
Figure 1: Geometry of the MAPCA family. Each panel shows a fixed two-dimensional dataset (grey scatter) together with its covariance ellipse (dashed grey), the metric unit ball {w:w⊤Mw=1}\{w:w^{\top}Mw=1\} (solid coloured ellipse), and the two principal component directions (red and blue arrows). The metric unit ball encodes what the method treats as “unit length”: a circle means all directions are treated equally; an ellipse aligned with the data means the metric has absorbed the full covariance structure. (1) Standard PCA ( M=IM=I , β=0\beta=0 ): the ball is a circle, blind to the data geometry; the condition number κ=15.3\kappa=15.3 reflects full spectral bias. (2) IPCA ( M=DM=D ): the ball is an axis-aligned ellipse, correcting for marginal variances without rotating the metric; this is the diagonal approximation to Σ\Sigma made geometric. (3)–(4) MAPCA ( β=0.5\beta=0.5 , β=0.75\beta=0.75 ): the ball continuously rotates and contracts toward the data ellipse as β\beta increases, compressing the spectrum and reducing κ\kappa monotonically (Proposition 3.2 ). (5) Whitening ( M=ΣM=\Sigma , β=1\beta=1 ): the ball matches the data ellipse exactly, enforcing isotropic output; κ=1\kappa=1 by construction, but all variance structure is removed and eigenvectors are non-unique (Remark 4 ). IPCA occupies a geometrically distinct position: its axis-aligned ball corrects for scale without rotating the metric, and it is the unique member of the MAPCA family satisfying strict scale invariance under arbitrary diagonal rescaling (Theorem 4 , Corollary 4 ).
MethodImplicit metric MMβ\betaκ\kappa behaviour
Standard PCAII00λ1/λp\lambda_{1}/\lambda_{p}
IPCAD=diag⁡(Σ)D=\operatorname{diag}(\Sigma)—κD\kappa_{D} ; strict invariance
VICReg (var. term)D=diag⁡(Σ)D=\operatorname{diag}(\Sigma)—Marginal correction only
Barlow TwinsΣ\Sigma1111 (isotropic)
ZCA WhiteningΣ\Sigma1111 (isotropic)
W-MSEΣ−1\Sigma^{-1}−1-1 (outside)(λ1/λp)2(\lambda_{1}/\lambda_{p})^{2} (amplified)
Table 2: Metric correspondence between SSL methods and the MAPCA framework.
Methodλ1\lambda_{1} (metric)λ1\lambda_{1} (imperial)Equal?Ratio ≈C−1\approx C^{-1} ?
Standard PCA75.933193.000NoNo
IPCA ( M=DM=D )1.9111.911Yes✓Yes ( ε≈10−14\varepsilon\approx 10^{-14} )✓
β=0.5\beta=0.58.71413.892No (Cor. 4 )No (Cor. 4 )
Whitening1.0001.000Yes (trivial)Degenerate (Rem. 4 )
Table 3: Scale invariance verification across four MAPCA methods, army cadets dataset.
More in this topic

Principal Component Analysis

Principal component analysis (PCA) finds orthogonal directions of greatest variance for data compression, visualization, denoising, and feature discovery.

See all 66 papers

Explore similar work

CardsList
May 25, 2026

Metric-Aware PCA as a Linear Instance of Geometric Deep Learning

Geometric deep learning organises neural architectures around the symmetries of their data domain, with the choice of symmetry group serving as a geometric prior that determines what representations can be learned. Metric-Aware Principal Component Analysis (MAPCA) parameterises principal component analysis by a positive-definite metric matrix, with a canonical subfamily interpolating between standard PCA and output whitening and a diagonal-metric point recovering Invariant PCA (IPCA). This paper positions MAPCA within the geometric deep learning framework. The metric is read as the geometric prior; the orthogonal group preserving it is the symmetry group it induces; MAPCA solutions are equivariant under this group with the resulting spectrum invariant; and MAPCA's defining constraint is the linear analogue of the Schur-type weight constraints used in equivariant networks. Across six axes - domain, symmetry group, equivariance, invariance, architectural primitive, and geometric prior - we construct a precise dictionary between MAPCA and geometric deep learning. The technical anchor is a uniqueness theorem characterising IPCA as the unique linear data-derived metric in the MAPCA family that is equivariant under arbitrary diagonal rescaling and projects onto the fixed-point set of the action, equivalent under normalisation to the variance-maximisation criterion in its precise form. The paper closes with three bridges: kernel PCA as the nonlinear extension, spectral graph methods as MAPCA on graphs, and a deep MAPCA construction extending the positioning into deep equivariant networks

Michael Leznik

Principal Component AnalysisGeometric Representation Learning
AskPDF
Aug 15, 2026

Shape Operator PCA: Curvature-Aware Projections for Geometric Machine Learning

In this paper, we propose SHOPCA (Shape Operator-based Principal Component Analysis), a novel method for unsupervised metric learning and dimensionality reduction that incorporates differential geometric information into the covariance structure of classical PCA. SHOPCA regularizes the global covariance matrix using the mean shape operator, defined as the average of the absolute local shape operators estimated from the data manifold, steering principal components toward directions of both maximum variance and informative curvature. A single trace-normalized mixing coefficient αα controls the regularization, recovering standard PCA at α=0α= 0 and a curvature-driven embedding as α→∞α\to \infty. We further introduce a fully unsupervised criterion for selecting αα based on the spectral eigengap of the regularized covariance matrix, maximizing the relative separation between the top-dd and remaining eigenvalues without using class labels. We evaluate SHOPCA on more than 50 real-world benchmark datasets, comparing it with PCA, ISOMAP, and UMAP using Adjusted Rand Index (ARI), Normalized Mutual Information (NMI), Fowlkes-Mallows index (FM), and V-measure. Results show that SHOPCA consistently improves clustering quality over PCA across a broad range of datasets and surpasses UMAP on small-sample settings, where iterative neighborhood-based manifold estimation can degrade. SHOPCA is computationally tractable, parameter-efficient, and applicable to domains requiring fully unsupervised, geometry-aware dimensionality reduction.

Alexandre L. M. Levada

Federal University of São Carlos 13565-905, São Carlos-SP, Brazil

Dimensionality ReductionPrincipal Component AnalysisMetric Learning
AskPDF
Mar 17, 2026

Manifold-Matching Autoencoders

We study a simple unsupervised regularization scheme for autoencoders called Manifold-Matching (MMAE): we align the pairwise distances in the latent space to those of the input data space by minimizing mean squared error. Because alignment occurs on pairwise distances rather than coordinates, it can also be extended to a lower-dimensional representation of the data, adding flexibility to the method. We find that this regularization outperforms similar methods on metrics based on preservation of nearest-neighbor distances and persistent homology-based measures. We also observe that MMAE provides a scalable approximation of Multi-Dimensional Scaling (MDS).

Laurent Cheret, Vincent Létourneau, Isar Nejadgholi +3

Department of Computer Science, University of Ottawa, Ottawa, Canada · MILA, Montréal, Canada · National Research Council of Canada, Ottawa, Canada

Multidimensional ScalingAutoencodersLatent-Space RegularizationDimensionality ReductionLatent Space GeometryLatent Space Alignment+2
AskPDF