Exact Stiefel Optimization for Probabilistic PLS: Closed-Form Updates, Error Bounds, and Calibrated Uncertainty

Haoran Hu; Xingce Wang

stat.MLMay 12, 2026

Exact Stiefel Optimization for Probabilistic PLS: Closed-Form Updates, Error Bounds, and Calibrated Uncertainty

Authors: Haoran Hu, Xingce Wang

Abstract

Probabilistic partial least squares (PPLS) is a central likelihood-based model for two-view learning when one needs both interpretable latent factors and calibrated uncertainty. Building on the identifiable parameterization of Bouhaddani et al.\ (2018), existing fitting pipelines still face two practical bottlenecks: noise--signal coupling under joint EM/ECM updates and nontrivial handling of orthogonality constraints. Following the fixed-noise scalar-likelihood protocol, we develop an end-to-end framework that combines noise pre-estimation, constrained likelihood optimization, and prediction calibration in one pipeline. We estimate the observation noise from the low-eigenvalue noise subspace and enforce orthogonality through exact Stiefel-manifold optimization. The noise-subspace estimator attains a signal-strength-independent leading finite-sample rate and matches a minimax lower bound, whereas a full-spectrum noise estimator carries a deterministic bias under the same model. We further extend the framework to sub-Gaussian settings via optional Gaussianization and provide closed-form standard errors through a block-structured Fisher analysis. Across synthetic high-noise settings and two multi-omics benchmarks (TCGA-BRCA and PBMC CITE-seq), the method achieves near-nominal coverage without post-hoc recalibration, reaches Ridge-level point accuracy on TCGA-BRCA at rank $r=3$ , matches or exceeds PO2PLS on cross-view prediction while providing native calibrated uncertainty, and improves stability of parameter recovery.

Explore similar work

Jul 17, 2026cs.LG

Improving Improved Kernel PLS

Improved Kernel Partial Least Squares (IKPLS) algorithms 1 and 2 are among the fastest PLS calibration algorithms. This article focuses on two shared steps, the computation of the

\mathbf{X}

rotations,

\mathbf{R}

, and the

\mathbf{Y}

loadings,

\mathbf{Q}

, and accelerates both. For

\mathbf{R}

, term-by-term accumulation is replaced by a direct evaluation strategy that requires the same number of multiplications but parallelizes better on modern hardware. For

\mathbf{Q}

, I identify - to the best of my knowledge, for the first time - equivalences showing that each

\mathbf{Y}

loading is obtainable, up to explicitly derived constants, from quantities already computed earlier in the same iteration, and I exploit them in IKPLS to reduce the cost of each loading from

Θ\left(KM\right)

Θ\left(M\right)

operations whenever

M = 1

2 \leq M < K

, with

K

predictor variables (number of columns in

\mathbf{X}

) and

M

response variables (number of columns in

\mathbf{Y}

). Both improvements provably yield exactly the same

\mathbf{W}

\mathbf{P}

\mathbf{Q}

\mathbf{R}

, and

\mathbf{T}

as the original algorithms. Benchmarks with NumPy (CPU) and JAX (GPU) show speedups of up to two orders of magnitude for the isolated steps and of approximately

2\times

(CPU) and

6\times

(GPU) for entire fits. Both improvements are implemented in the free, open-source Python package \texttt{ikpls}.

Ole-Christian Galbo Engstrøm

Least Squares

Ask PDF

May 8, 2026cs.LG

The Minimax Rate of Second-Order Calibration

We characterize the minimax rate of estimating the second-order calibration error for binary classification, which quantifies whether a higher-order predictor's epistemic-uncertainty estimate matches the conditional variance of the label probability on its level sets. Our key observation is that the sech perturbation kernel, previously used only to enforce smoothness of calibration functions, in fact makes them analytic in a strip of half-width

hπ/2

. Polynomial regression then estimates the calibration error at rate

\tilde{O}(1/\sqrt{n})

, with explicit constants, a qualitative improvement over the

O(n^{-1/4})

rate achievable by bucketing or kernel smoothing. A matching

Ω(1/\sqrt{n})

lower bound establishes minimax optimality up to logarithmic factors. As a corollary, we give the first finite-sample guarantee for second-order Platt scaling, yielding a post-hoc procedure that recalibrates both the mean prediction and the epistemic-variance estimate of any higher-order predictor. Along the way, we provide a bucket-free definition of second-order calibration and relate it quantitatively to the bucketed formulation of Ahdritz et al. [2025]. Our experiments confirm the predicted rate and the quality of the recalibrated uncertainties.

Kamil Ciosek, Banafsheh Rafiee, Sina Ghiassian +1

Calibrated Uncertainty Multicalibration and Calibration Minimax Binary Classification Perturbation Propagation

Ask PDF

May 26, 2025stat.ML

No Free Lunch: Non-Asymptotic Analysis of Prediction-Powered Inference

Prediction-Powered Inference (PPI) is a popular strategy for combining gold-standard and possibly noisy pseudo-labels to perform statistical estimation. Prior work has shown an asymptotic \enquote{free lunch} for PPI++, an adaptive form of PPI, showing that the \textit{asymptotic} variance of PPI++ is always less than or equal to the variance obtained from using gold-standard labels alone. Notably, this result holds \textit{regardless of the quality of the pseudo-labels}. In this work, we demystify this result by conducting an exact finite-sample analysis of the estimation error of PPI++ on the mean estimation problem. We give a \enquote{no free lunch} result, characterizing the settings (and sample sizes) where PPI++ has provably worse estimation error than using gold-standard labels alone. Specifically, PPI++ will outperform if and only if the correlation between pseudo- and gold-standard is above a certain level that depends on the number of labeled samples (

n

). In some cases our results simplify considerably: For Gaussian data, for instance, the correlation must be at least

1/\sqrt{n - 2}

in order to see improvement. More broadly, by providing exact non-asymptotic expressions for the variance of PPI++ under sample splitting, we aim to empower practitioners to transparently reason about the benefits of PPI++ in specific applications. In experiments, we illustrate that our theoretical findings hold on real-world datasets.

Pranav Mani, Peng Xu, Zachary C. Lipton +1

Finite-Sample Noisy Labels Asymptotics Finite Time Analysis

Ask PDF