Complex Interpolation of Matrices with an application to Multi-Manifold Learning
Authors: Adi Arbel, Stefan Steinerberger, Ronen Talmon
Organizations: Viterbi Faculty of Electrical and Computer Engineering, Technion – Israel Institute of Technology, Haifa, Israel · Department of Mathematics and Department of Applied Mathematics, University of Washington, Seattle, WA 98195, USA
Given two symmetric positive-definite matrices A,B∈Rn×n, we study the spectral properties of the interpolation A1−xBx for 0≤x≤1. The presence of `common structures' in A and B, eigenvectors pointing in a similar direction, can be investigated using this interpolation perspective. Generically, exact log-linearity of the operator norm ∥A1−xBx∥ is equivalent to the existence of a shared eigenvector in the original matrices; stability bounds show that approximate log-linearity forces principal singular vectors to align with leading eigenvectors of both matrices. These results give rise to and provide theoretical justification for a multi-manifold learning framework that identifies common and distinct latent structures in multiview data.
Figures & tables
Figure 1: Two 2D projections of a 3D point cloud of a cow on two different view angles: given only the two two-dimensional point sets (discretized via a kernel into two symmetric positive-definite matrices A,B∈Rn×n ), can one automatically discover that the eigenstructure captures the same underlying ground truth? (Details in section 2 and section 4 ).
Figure 2: Two aligned point clouds sampled from two 2D cylinders embedded in R3 . The height (red) axis is shared among the point clouds, whereas the azimuthal angle is distinct.
Figure 3: The SVFD of the point clouds depicted in fig. 2 . The empirical singular values of the interpolated kernels connecting the second common eigenvalues (shown left and right) are highlighted in yellow.
Figure 4: The same as fig. 3 , but highlighting the empirical singular values originating at x=0 from the fourth largest eigenvalue of C1 . The point clouds on the left and right are colored by the corresponding eigenvector. The curve is far from a line, the eigenvectors have little in common.
Figure 5: A sketch of the domain D .
Figure 6: A sketch of the multimanifold learning setup: three hidden manifolds and two observables, where at each observation only one manifold ( M3 ) is common.
We study eigenpair estimation for the tensorized, density-weighted Laplace--Beltrami operator from n independent data points sampled from a multi-manifold mixture distribution where manifolds may intersect. Let d denote the largest intrinsic dimension of the manifold components and s≥2 the smoothness of the sampling densities. For simple, nontrivial eigenpairs, we establish the minimax rate n−s/(d+2s) for approximating the eigenvalue and eigenfunction of the tensorized Laplace--Beltrami operator, and show that a continuum plug-in estimator based on kernel density estimation attains the same rate when the supporting manifolds are known. We further analyze a manifold-agnostic graph Laplacian estimator and prove that its eigenpairs converge to those of the tensorized Laplace--Beltrami operator at rate n−1/(2.5d−1), up to logarithmic factors, improving upon previously known spectral convergence rates for multi-manifold learning. Numerical experiments on synthetic multi-manifold data corroborate the rates guaranteed by our theorems. These results characterize the information-theoretic difficulty of multi-manifold spectral estimation and quantify the additional cost of learning without oracle component information.
Haoyu Chen, Chenghui Li
Department of Statistics University of Wisconsin Madison Madison, WI 53706, USA
We study the multi-task linear regression problem in the presence of contaminated tasks. We address the setting where the unknown parameters of a majority of tasks are close in the ℓ2-norm, while a fraction of tasks are arbitrary outliers. Existing theoretical frameworks for this problem rely heavily on the assumption that the empirical second moment of each task has a minimum eigenvalue bounded away from zero (order Ω(1)). Crucially, this assumption fails in many high-dimensional scenarios, rendering prior guarantees vacuous. To overcome this limitation, we propose an estimator based on matrix-weighted norm regularization. We also introduce a relative balancedness condition, quantified by a balancedness constant, that compares each task's second moment with the average inlier geometry and relaxes the need for taskwise second-moment lower bounds. In favorable regimes with moderate balancedness, our prediction MSE bounds match the rate of Duan and Wang (2023) under substantially weaker spectral assumptions; the resulting task-overall MSE is minimax optimal up to logarithmic factors. Furthermore, we demonstrate that our estimator enjoys a safety guarantee: when the relevant balancedness constant is large or infinite, or when tasks are unrelated, the method performs no worse than independent task learning.
The elementwise Hadamard product of two low-rank matrices provides a parameter-efficient model for data with multiplicative structure, but its modeling is challenging due to the presence of additional symmetries under coupled row/column scalings between the two factors. In order to leverage the geometry of the space, we formulate the learning of such matrices as optimization on a Riemannian quotient manifold. We propose a novel block-diagonal Riemannian metric derived from the pullback of the Frobenius inner product. The metric is shown to be invariant under these symmetries. We develop a Riemannian gradient descent algorithm that uses a tuning-free Gauss--Newton step size and scales linearly in the number of observed entries per iteration. The versatile framework of Riemannian quotient optimization enables both first-order and second-order Riemannian methods, the latter through a closed-form connection and the Riemannian Hessian. Experiments on real and synthetic datasets illustrate the efficacy of our proposed Riemannian approach.