Cheap and Powerful Tests for Supervised Subspaces: Per-Component Inference for PLS

Paweł Lenartowicz; Hubert Plisiecki

cs.LGSep 28, 2026

Cheap and Powerful Tests for Supervised Subspaces: Per-Component Inference for PLS

Authors: Paweł Lenartowicz, Hubert Plisiecki

Organizations: Society for Open Science (Stowarzyszenie na Rzecz Otwartej Nauki) Centre for Brain Research, Jagiellonian University · Society for Open Science (Stowarzyszenie na Rzecz Otwartej Nauki) IDEAS Research Institute

Abstract

Partial Least Squares (PLS) regression extracts a few outcome-aligned directions in a high-dimensional X and is widely used across applied science, but inference on the resulting fit is either expensive, biased and discouraged, or absent. We reduce inference to held-out OLS refits of the supervised subspace, a primitive shared by PLS, supervised PCA, and linear probes, and supply two tests using held-out correlations: a Nadeau-Bengio corrected asymptotic t-test as a fast approximation, and a permutation test with comparable power, finite-sample valid under outcome-predictor independence and iid rows. Held-out predictions are unchanged under any orthogonal rebasing of the supervised span, so an interpretable basis such as varimax inherits the joint claim but not a per-axis p-value; per-component claims come from a fixed-sequence test on the PLS extraction order. We validate on synthetic geometries, two NIR chemometric datasets, and cross-lingual word-embedding regressions; the exact test also transfers to supervised PCA and a ridge probe. The proposed tests have more power than CV-permutation-Q^2, at a fraction of its cost. A pre-run check on n and the spectrum of X says when the approximation is safe. We release a Rust library with Python, R, and Julia bindings, plus a Python text pipeline.

Figures & tables

Method	Target	Cost	Calibration	Rotation
Jackknife $t$	Regression coefficients	$F$ CV fits	Approximate; biased variance estimates	Tests coefficients, not latent axes
CV-perm.- $Q^{2}$	Outcome–predictor independence	$(B+1)F$ fits	Exact under null permutation invariance	Joint predictions unchanged
plsdof	Model complexity; selection and CIs	$O(n^{3})$ exact DoF pass	No supplied omnibus significance test	Fitted-model target
NB (ours)	Held-out predictive association	$J$ fits	Empirical; no proved component-chain FWER	Joint OLS refit invariant
Permutation (ours)	Outcome–predictor independence	$J$ cached maps at $K=1$ ; $B$ draws	Exact under iid rows and independence	Joint OLS refit invariant
Single split (ours)	Train-conditional residual association or independence	One training chain; held-out tests	Strong FWER: Gaussian correlation or permutation independence nulls	Claims follow training order

Table 1: Inference procedures reviewed in § 1.2 . Costs describe the tested fit or component:

F

CV folds,

J

repeated splits,

B

permutations. Rotation entries concern rebasing a fixed fitted span; none supplies significance for an individual varimax axis.

Figure 1: Inference pipeline for § 3.3 . (A) Per-axis NB

t

-statistic (

t_{\mathrm{NB},h}

, Eq. ( 3 )):

J

random 50/50 splits feed Fisher-

z

aggregation with the corrected resampled variance. (B) Fixed-sequence FWER over NIPALS-deflated pairs

(\mathbf{X}^{(h)},\mathbf{y}^{(h)})

h=1,\dots,K^{*}

, stopping at the first non-rejection; the joint summary

\alpha^{*}_{\mathrm{FWER}}

is the minimum level at which all preceding nulls reject. (C) Varimax acts on loadings only after inference; held-out predictions and

t_{\mathrm{NB}}

are invariant under any

\mathbf{Q}\in O(K)

Object	Null tested	Guarantee
Joint subspace	rank- $K$ fit predicts $\mathbf{y}$ no better than chance	omnibus NB test; exact under NB-permutation
Unrotated component $h$	no positive residual association after $h\!-\!1$ deflations	fixed-sequence; FWER requires per-step validity (§ 4.4 )
Varimax axis	none — no separate test	inherits joint significance only (Prop. 1 )

Table 2: What is tested before and after rotation.

Figure 2: Krylov saturation of supervised PLS to OLS, and the calibration-axis fingerprint. (A) Per-axis principal cosine against the planted subspace at

r_{\mathrm{plant}}\!=\!3

, across planted

R^{2}

. (B) Cumulative

\hat{\beta}_{\mathrm{PLS}}(K)

versus ridge-stabilized

\hat{\beta}_{\mathrm{OLS}}

K\!=\!1\dots 10

. Toy 1 setup:

N\!=\!4000

D\!=\!300

, Warriner GloVe, 50 seeds,

\boldsymbol{\alpha}\!=\![1.0,0.7,0.5]

. iid Gaussian replication and

r_{\mathrm{plant}}\!=\!2

in App. A.8 . (C) Toy 2: at

K\!=\!2

with a planted suppressor at angle

\rho

to the main predictor, the varimax-rotated Dim-2 does not recover the suppressor (dashed, near the random baseline) yet carries held-out

\mathbf{y}

-signal (solid), on GloVe and iid geometry alike; 50 seeds per cell, bootstrap 95% bands.

Design	$n$	st. rank	rule	$.05$	$.10$	KS (NB)	$.05$	$.10$
				NB-asymptotic			exact
iid Gaussian ( $p\!=\!300$ )	40	22.0	pass	0.038	0.083	$1.3{\times}10^{-3}$	0.048	0.096
iid Gaussian	160	55.3	pass	0.050	0.100	0.72	0.052	0.106
block-collinear (5 blocks, $r\!=\!.9$ )	40	3.80	pass	0.050	0.095	$7.6{\times}10^{-6}$	0.050	0.103
Warriner GloVe-300 (real)	40	17.1	pass	0.035	0.085	0.18	0.042	0.100
dominant factor ( $r\!=\!.9$ , all columns)	40	1.10	flag	0.041	0.061	$2{\times}10^{-77}$	0.050	0.100

Table 3: The validity rule on eight null cells (full grid of 25 plus Tecator in App. A.13 ). Stable rank

\|\mathbf{X}\|_{F}^{2}/\|\mathbf{X}\|_{2}^{2}

of the column-standardized design; flag if

n\!<\!25

or stable rank

<\!3

. Level: rejection rate under

H_{0}

\alpha\!=\!.05

.10

; KS: KS-uniform

p

of the NB-asymptotic null

p

-values. 5,000 replicates per cell (Wilson half-width

\pm 0.006

.05

\pm 0.008

.10

) except Warriner (2,000) and Tecator (NB-asymptotic 400, exact 2,000).

Figure 3: Empirical power vs.

n

\alpha\!=\!0.05

on synthetic Warriner geometry. NB (blue) vs. perm-

Q^{2}

(orange) at

r_{\mathrm{plant}}\!=\!1

(solid) and

r_{\mathrm{plant}}\!=\!3

(dashed);

R^{2}\!=\!0.05

(left) and

R^{2}\!=\!0.40

(right). Matched-power sample sizes in App. A.12.4 .

Design	Arm	$m\!=\!1$	$m\!=\!2$	$m\!=\!3$
iid Gaussian ( $40\!\times\!20$ )	full-data, NB	0.000 / 0.000	0.000 † / 0.000 †	0.000 † / 0.000 †
dominant factor ( $40\!\times\!20$ )	full-data, NB	0.049 / 0.095	0.000 / 0.000	0.000 † / 0.000 †
	train-only, NB	0.038 / 0.081	—	—
	full-data, perm.	0.059 / 0.118	0.000 / 0.000	0.000 † / 0.000 †
decaying spectrum ( $40\!\times\!60$ )	full-data, NB	0.007 / 0.018	0.004 / 0.012	0.002 / 0.007
NIR gasoline ( $40\!\times\!401$ , real)	full-data, NB	0.017 / 0.035	0.011 / 0.024	0.001 / 0.002

Table 4: Probability that the sequence rejects through step

m\!+\!1

after

m

planted signal axes,

n\!=\!40

J\!=\!50

, total planted

R^{2}\!=\!0.5

spread over the

m

signal axes, 5,000 replicates per cell, at

\alpha\!=\!.05

.10

. full-data, NB : every step deflated on all rows; train-only, NB : each split deflated using its training half. full-data, perm. : permutation reference at the deflated step, where it is not exact. Step

m\!+\!1

need not be a true conditional null (see text).

\dagger

: the chain never reaches that step (the signal axes below it are not detected), so the zero carries no information. Setup and the

\alpha\!=\!.01

column in App. A.15 .

Appendix figures & tables30 assets

Supplementary material from the paper’s appendix.

Appendix

Config	Language	Embedding	Outcome	$n$
1	English	GloVe-300 (Common Crawl 42B, normalized)	Warriner V/A/D	13,365
2	Polish	GloVe-800 (Dadas, normalized)	ANPW V/A/D	4,526
3	Spanish	GloVe-300 (SBWCE, normalized)	Stadthagen-González V/A	12,639

Appendix

Table 5: Three (language

\times

embedding) configurations. Config 1 uses Warriner V/A/D, Config 2 uses ANPW (Polish counterpart), and Config 3 uses the Stadthagen-González [ Stadthagen-Gonzalez et al., 2017 ] valence/arousal norms (Spanish has no rated dominance dimension). All fits use

K\!=\!3

with varimax on the loading matrix; the rotation target is the full embedding vocabulary intersected with the rated lexicon (§ 3.4 , App. A.17 ).

Config	Valence $r^{2}$ (CV)	Arousal $r^{2}$ (CV)	Dominance $r^{2}$ (CV)
EN GloVe-300	0.631 (0.619)	0.370 (0.350)	0.486 (0.472)
PL GloVe-800	0.692 (0.652)	0.617 (0.567)	0.611 (0.557)
ES GloVe-300	0.551 (0.539)	0.447 (0.433)	—

Appendix

Table 6: Aggregate fit on the three configurations,

K\!=\!3

, varimax on loadings. For each rated dimension the first figure is the in-sample (resubstitution)

r^{2}

of the full-data fit and the second, in parentheses, the split-half cross-validated

R^{2}

K\!=\!3

(mean over the

J\!=\!50

NB splits; kselect/selection_sequence.csv ). The largest gap is PL dominance, 0.611 in-sample against 0.557 cross-validated.

\alpha^{*}_{\mathrm{FWER}}\!<\!.001

in every cell (per-axis NB on the unrotated NIPALS extraction,

J\!=\!50

; Prop. 1 ). FWER control for this NB summary remains conditional on per-step validity (§ 3.3 ); the single-split guarantee in App. A.14 does not cover this repeated-split analysis. Spanish dominance is not reported because the Stadthagen-González norms cover valence and arousal only.

		Valence			Arousal			Dominance
Rank	Side	EN	PL	ES	EN	PL	ES	EN	PL
1	pos	wonderful	terrific (wspaniały)	enjoy (disfrutar)	violent	brutal / violent (brutalny)	aggression (agresión)	strive	terrific (wspaniały)
2	pos	enjoy	admire (podziwiać)	pleasant (agradable)	deadly	lust (żądza)	murders (asesinatos)	appreciate	bestow (obdarzyć)
3	pos	appreciation	wonderful (cudowny)	creativity (creatividad)	brutal	frenzy (szał)	murder (asesinato)	appreciated	admire (podziwiać)
4	pos	delightful	excellent (wspaniale)	gift (regalo)	insane	terrible (straszliwy)	deadly (mortal)	exceptional	excellent (wspaniale)
5	pos	fantastic	delight (zachwycić)	beautiful (hermosa)	explosion	cruelty (okrucieństwo)	violent (violento)	dedication	excellent (znakomity)

Appendix

Table 7: Top 10 words per side at the Combined loadings for every (rated dimension

\times

language) cell of Configs 1, 2, 3 (varimax-rotated PLS,

k=3

). Polish and Spanish cells are formatted uniformly as english (native) ; Polish glosses come from the word_en field of Imbir [2016] when available and from Google Translate otherwise, Spanish glosses from Google Translate (caches under applied/_*_en_translations.json ). The Stadthagen-González ES norms do not include a dominance dimension, so Dominance has EN/PL only.

		Valence			Arousal			Dominance
Rank	Side	EN	PL	ES	EN	PL	ES	EN	PL
1	pos	elegant	happily (radośnie)	happiness (felicidad)	violent	drug (narkotyk)	aggression (agresión)	elegant	excellent (wspaniale)
2	pos	lovely	wonderfully (cudownie)	baby (bebé)	brutal	sexual (seksualny)	explosion (explosión)	wonderful	bestow (obdarzyć)
3	pos	fabulous	excellent (wspaniale)	happy (feliz)	deadly	sex/ intercourse (seks)	attack (atentado)	fabulous	mighty / powerful (potężny)
4	pos	delightful	wonderful (cudowny)	smile (sonrisa)	revenge	explosive (wybuchowy)	murders (asesinatos)	lovely	terrific (wspaniały)
5	pos	exquisite	to squirt (tryskać)	enjoy (disfrutar)	rage	mass (masowy)	assaults (agresiones)	delightful	capable (zdolny)

Appendix

Table 8: Top 10 words per side at the Dim-1 loadings, same conventions as Tab. 7 .

		Valence			Arousal			Dominance
Rank	Side	EN	PL	ES	EN	PL	ES	EN	PL
1	pos	provide	modest (skromny)	decor (decoración)	ignorant	show (okazywać)	terrible (terrible)	monitoring	decorate (ozdobić)
2	pos	provided	beautiful (piękny)	design (diseño)	jealous	justify (usprawiedliwić)	deadly (mortal)	component	silver (srebrny)
3	pos	ensure	beauty (uroda)	ceramic (cerámica)	forgive	willing (skłonny)	guilt (culpa)	components	slender (smukły)
4	pos	provides	to like (cenić)	architecture (arquitectura)	cried	feelings (uczucia)	kill (matar)	evaluation	gold (złocisty)
5	pos	providing	admire (podziwiać)	designs (diseños)	rude	cruelty (okrucieństwo)	revenge (venganza)	provides	adorn (zdobić)

Appendix

Table 9: Top 10 words per side at the Dim-2 loadings, same conventions as Tab. 7 .

		Valence			Arousal			Dominance
Rank	Side	EN	PL	ES	EN	PL	ES	EN	PL
1	pos	industrial	central (centralny)	support (respaldar)	yummy	monstrous (potworny)	reject (rechazar)	wedding	social (społeczny)
2	pos	containing	in parallel (równolegle)	consolidate (consolidar)	gourmet	shrill (przeraźliwy)	support (respaldar)	features	range (zakres)
3	pos	e.g.	marked (oznaczony)	promote (impulsar)	tasty	rage (wściekłość)	deny (negar)	feature	institution (instytucja)
4	pos	multiple	millimeter (milimetr)	integrate (integrar)	delicious	terrible (straszliwy)	openly (abiertamente)	large	concern (dotyczyć)
5	pos	storage	diameter (średnica)	promote (promover)	juicy	mad (oszalały)	admit (admitir)	featuring	activity (działalność)

Appendix

Table 10: Top 10 words per side at the Dim-3 loadings, same conventions as Tab. 7 .

lang	pair	J(+)	J( $-$ )	signed-J
EN	Dim-3(V) $\leftrightarrow$ Dim-3(A)	0.00	0.00	$-$ 0.00
EN	Dim-3(V) $\leftrightarrow$ Dim-3(D)	0.00	0.25	$-$ 0.11
EN	Dim-3(A) $\leftrightarrow$ Dim-3(D)	0.00	0.00	$-$ 0.00
PL	Dim-3(V) $\leftrightarrow$ Dim-3(A)	0.00	0.00	$-$ 0.00
PL	Dim-3(V) $\leftrightarrow$ Dim-3(D)	0.00	0.00	$-$ 0.00
PL	Dim-3(A) $\leftrightarrow$ Dim-3(D)	0.00	0.00	$-$ 0.07

Appendix

Table 11: Top-20 Jaccard between supervised Dim-3 vocabularies fit on the same

\mathbf{X}

across pairs of

\mathbf{y}

’s. J(+) and J(

-

) are unsigned Jaccards on the positive- and negative-side word sets; signed-J

\in\![-1,1]

measures whether two

\mathbf{y}

’s split words the same way (+1), opposite way (

-1

, sign-flipped axis), or are disjoint (

\approx\!0

). ES has only the (V, A) pair because the Stadthagen-González norms omit dominance.

lang	$\mathbf{y}$	$n_{\text{seeds}}$	median $r^{2}$	median $p$
EN	valence	20	0.022	0.567
EN	arousal	20	0.023	0.551
EN	dominance	20	0.023	0.519

Appendix

Table 12: Empirical Type-I rate at

\alpha\!=\!0.05

for the NB split-half global

p

-value over 20 i.i.d. random-embedding draws per cell (

\text{seed}=42+1000k+\text{lang\_offset}

k\!=\!0\dots 19

). Real

\mathbf{y}

, same pipeline,

K\!=\!3

+ varimax. Calibrated NB should land near 0.05; substantially higher flags an inflation under

K\!=\!3

+ varimax.

lang	pair	J(+)	signed-J
EN	PC1 $\leftrightarrow$ Dim-3(V)	0.00	$-$ 0.08
EN	PC2 $\leftrightarrow$ Dim-3(V)	0.03	$-$ 0.01
EN	PC3 $\leftrightarrow$ Dim-3(V)	0.00	$-$ 0.00
EN	PC1 $\leftrightarrow$ Dim-3(A)	0.00	$-$ 0.00
EN	PC2 $\leftrightarrow$ Dim-3(A)	0.03	$-$ 0.01
EN	PC3 $\leftrightarrow$ Dim-3(A)	0.03	$-$ 0.01

Appendix

Table 13: Top-20 Jaccard between each unsupervised PC of

\mathbf{X}

and each supervised Dim-3(

\mathbf{y}

), per language. Same J(+), J(

-

), signed-J convention as Tab. 11 . CH3’s prediction is a near-one entry in at least one

(\mathrm{PC},\mathbf{y})

cell per language. The largest observed value is

0.29

(ES PC1 vs Dim-3(V/A)); EN and PL stay at

|\mathrm{signed\text{-}J}|\!\leq\!0.14

throughout.

#	Name	Planted truth	Measured	Headline
Toy 1	Identifiability ceiling	$r_{\text{plant}}\!\in\!\{2,3\}$ directions in $\mathbf{X}\!\to\!\mathbf{y}$	Mean principal cosine vs. PLS subspace	Plateau at $\approx\!1/r_{\text{plant}}$ ; only leading axis recovered ( $\mathrm{pcos}_{1}\!\to\!0.86$ – $0.98$ at $R^{2}\!=\!0.7$ )
Toy 2	Suppressor fingerprint at $K\!=\!2$	Suppressor at $\cos\!=\!\rho$ with main predictor, $\rho\!\in\!\{0,0.3,0.6,0.9\}$	$\|\!\cos(\mathbf{W}_{2}^{\text{rot}},\mathbf{w}_{\text{supp}})\|$ ; $\mathrm{cor}(\mathbf{t}_{2}^{\text{rot}},\mathbf{y})$ ; NB reject rate $\beta_{\text{Dim-2}}\!=\!0$	NB rejects at mean rate 0.91 even though $\|\!\cos\|\!\in\![0.04,0.29]$ — empirical calibration-axis fingerprint
Toy 3	Functional-form invariance	Linear-additive $+$ multiplicative interaction	$R^{2}(K{=}1,K{=}2)$ ; $\cos(\mathbf{W},\mathbf{e}_{D+1})$ ; Arms A (raw) / B (lifted)	Arm A: $K\!>\!1$ does not surface interaction; Arm B: lifted interaction recovered as rank-1 ( $\cos\!\to\!0.88$ )
Toy 4	Rotation-invariance sanity	— (real Warriner fit)	$\Delta(\text{NB global }p)$ under varimax / random orth. rotation	$\Delta\!=\!0$ (varimax); $\sim\!10^{-78}$ (random orth.) vs. baseline $p\!\sim\!10^{-65}$
Toy 5	$K^{*}$ depends on $\Sigma_{\mathbf{X}}$ , not $r_{\mathrm{plant}}$	Rank-1 sparse signal in $\mathbf{y}$ at $R^{2}\!=\!0.7$	Mode $K^{*}$ from sequential per-step NB	$K^{}\!=\!6$ on GloVe vs. $K^{}\!=\!2$ on iid; predictively justified in both cells; $\Delta R^{2}_{\text{CV}}\!\leq\!0.17$
S1	$H_{0}$ calibration grid	$\mathbf{y}\perp\mathbf{X}$	Empirical FPR; KS-uniform $p$ of $p$ -values	NB and perm- $Q^{2}$ calibrated; naive_t and jack.test inflated at $n\!=\!40$

Appendix

Table 14: Overview of synthetic experiments. Toys 1–5 (this section) target mechanism and identifiability claims; S1 and S2 (§ A.9 ) target inference calibration and power. Per-experiment tables and full sweeps in the subsections below (Toys 1–5) and in § A.9 (S1, S2).

$r_{\text{plant}}$	Geometry	$\overline{\mathrm{pcos}}$ at $R^{2}{=}0.7$	$\overline{\mathrm{pcos}_{1}}$ at $R^{2}{=}0.7$	$\overline{\mathrm{pcos}_{\geq 2}}$ range	$1/r_{\text{plant}}$ ceiling
2	GloVe	0.449	0.856	$0.037$ – $0.049$	0.500
2	iid Gaussian	0.511	0.981	$0.040$ – $0.051$	0.500
3	GloVe	0.350	0.909	$0.025$ – $0.106$	0.333
3	iid Gaussian	0.373	0.983	$0.021$ – $0.116$	0.333

Appendix

Table 15: Toy 1 — identifiability ceiling at

R^{2}\!=\!0.7

(top of sweep). Mean principal cosine plateaus at

\approx\!1/r_{\text{plant}}

; only the leading axis is recovered (

\mathrm{pcos}_{1}\to 0.86

–

0.98

), with subdominant directions hovering at

\mathrm{pcos}_{\geq 2}\approx 0.02

–

0.12

. Krylov saturation under univariate

\mathbf{y}

realized empirically. 50 seeds per

(r_{\text{plant}},\text{geom},R^{2})

cell across

R^{2}\in\{0.10,\ldots,0.70\}

Geometry	$\rho$	$\overline{\|\cos(\mathbf{W}_{2}^{\text{rot}},\mathbf{w}_{\text{supp}})\|}$	$\overline{\mathrm{cor}(\mathbf{t}_{2}^{\text{rot}},\mathbf{y})}$	NB reject (Dim-2)
GloVe	0.0	0.122	0.232	0.92
GloVe	0.3	0.049	0.194	0.84
GloVe	0.6	0.116	0.204	0.80
GloVe	0.9	0.243	0.226	0.92
iid Gaussian	0.0	0.138	0.331	0.94
iid Gaussian	0.3	0.044	0.282	0.92

Appendix

Table 16: Toy 2 — suppressor fingerprint at

K\!=\!2

. Eight cells (

\rho\in\{0.0,0.3,0.6,0.9\}\times

geometry), 50 seeds per cell.

|\cos(\mathbf{W}_{2}^{\text{rot}},\mathbf{w}_{\text{supp}})|

shows no monotone signal in

\rho

(range

[0.044,0.293]

); rotated Dim-2 nonetheless carries small but nonzero correlation with

\mathbf{y}

(range

[0.194,0.351]

) and NB rejects

\beta_{\text{Dim-2}}\!=\!0

at mean rate

0.91

across cells.

Arm	Quantity	$K{=}1$	$K{=}2$	$\Delta R^{2}$	NB rej. (Dim-2)
A (unaugmented)	$R^{2}$	0.131	0.191	$+0.060$	—
A	$\mathrm{cor}(\mathbf{t},\mathbf{y})$	0.361	0.105	—	0.75
A	$\cos(\mathbf{W}_{2},\mathbf{w}_{2})$	—	0.049	—	—
B (augmented)	$R^{2}$	0.402	0.572	$+0.170$	—
B	$\mathrm{cor}(\mathbf{t},\mathbf{y})$	0.633	—	—	—
B	$\cos(\mathbf{W},\mathbf{e}_{D+1})$	0.790	0.876	—	—

Appendix

Table 17: Toy 3 — functional-form invariance. Arm A: linear design with planted interaction

y=\beta_{1}\mathbf{X}\mathbf{w}_{1}+\beta_{3}\mathbf{X}\mathbf{w}_{3}-\beta_{2}(\mathbf{X}\mathbf{w}_{1})(\mathbf{X}\mathbf{w}_{2})+\varepsilon

;

\beta_{1}{=}\beta_{3}{=}0.5

\beta_{2}{=}1.5

. Arm B:

\tilde{\mathbf{X}}=[\mathbf{X}\mid(\mathbf{X}\mathbf{w}_{1})(\mathbf{X}\mathbf{w}_{2})]

. 20 seeds, paired across arms and

K

. Arm A:

K{>}1

cannot surface the interaction as a separate axis (Krylov saturation); Arm B: PLS recovers the lifted interaction as a rank-1 direction (

\cos\!\to\!0.88

K\!=\!2

Comparison	$p_{\text{unrotated}}$	$p_{\text{rotated}}$	$\|p_{\text{unrot}}-p_{\text{rot}}\|$
varimax vs raw	$8.886{\times}10^{-65}$	$8.886{\times}10^{-65}$	$0$
random-orth vs raw	$4.860{\times}10^{-65}$	$4.860{\times}10^{-65}$	$8.282{\times}10^{-78}$

Appendix

Table 18: Toy 4 — rotation-invariance sanity check. Single fit on real Warriner; global NB

p

-value computed before and after applying (i) varimax to the loadings and (ii) a uniform random orthogonal rotation. Numerical equality for varimax; machine-precision agreement (delta

\sim 10^{-78}

against a baseline of

\sim 10^{-65}

) for the random rotation.

Geometry	$n_{\text{features}}$	mode $K^{*}$	mean $K^{*}$	$\mathrm{frac}(K^{*}{=}2)$	$\Delta R^{2}_{\text{CV}}(K{=}2-K{=}1)$
iid Gaussian	10	2	2.2	0.8	0.066
iid Gaussian	30	2	2.0	1.0	0.066
iid Gaussian	100	2	2.0	1.0	0.066
iid Gaussian	300	2	2.0	1.0	0.065
Real GloVe	10	5	5.4	0.0	0.145
Real GloVe	30	6	6.0	0.0	0.160

Appendix

Table 19: Toy 5 —

K^{*}

depends on

\Sigma_{\mathbf{X}}

at fixed planted rank,

R^{2}\!=\!0.70

N\!=\!4000

D\!=\!300

, 5 seeds per cell. The 1-SE selector returns mode

K^{*}\!=\!6

on real GloVe vs. mode

K^{*}\!=\!2

on iid Gaussian, both at

r_{\mathrm{plant}}\!=\!1

K^{*}

is the predictively warranted

K

in both cells (smaller

K

are

>

1 SE worse out-of-sample on GloVe);

K^{*}

is not an estimator of

r_{\mathrm{plant}}

— under the Krylov geometry of PLS1, predictive

K

generically exceeds the planted rank in correlated

\mathbf{X}

$r_{\text{plant}}$	Geometry	Regime	NB	perm-Q 2	$\Delta$
1	Warriner	$n\!=\!40$ (valence)	0.81	0.73	$+0.08$
1	Warriner	$n\!=\!500$ (valence)	0.72	0.69	$+0.03$
1	iid	small- $n$ / hi-eff	0.96	0.71	$+0.25$
1	iid	big- $n$ / noisy	0.70	0.63	$+0.07$
3	Warriner	small- $n$ / hi-eff	0.89	0.76	$+0.13$
3	Warriner	big- $n$ / noisy	0.94	0.93	$+0.01$

Appendix

Table 20: S2 coverage matrix — detection rate at

\alpha\!=\!0.05

across

r_{\text{plant}}\in\{1,3\}

\times

geometry

\in\{\text{Warriner},\text{iid}\}

\times

regime

\in\{\text{small-}n/\text{hi-eff},\text{big-}n/\text{noisy}\}

K_{\text{fit}}\!=\!1

throughout. Top two rows: 400 reps (subset of S2). Other six: 200 reps each. NB

\geq

permutation-Q 2 in every cell; gap widest (0.13–0.25) when both methods are away from ceiling, narrowest (0.01) when both saturate.

Geometry	Regime	$n$	NB power	perm- $Q^{2}$ power	NB med. $t$	perm med. $t$
EN GloVe-300	small- $n$ / hi-eff ( $R^{2}\!=\!0.40$ )	40	0.965 [.930, .983]	0.815 [.755, .863]	0.005 s	0.223 s
EN GloVe-300	big- $n$ / noisy ( $R^{2}\!=\!0.05$ )	500	0.945 [.904, .969]	0.950 [.910, .973]	0.142 s	3.730 s
PL GloVe-800	small- $n$ / hi-eff ( $R^{2}\!=\!0.40$ )	40	0.865 [.811, .906]	0.620 [.551, .684]	0.014 s	0.757 s
PL GloVe-800	big- $n$ / noisy ( $R^{2}\!=\!0.05$ )	500	0.870 [.816, .910]	0.820 [.761, .867]	0.517 s	28.27 s
EN XLM-R	small- $n$ / hi-eff ( $R^{2}\!=\!0.40$ )	40	0.965 [.930, .983]	0.800 [.739, .850]	0.014 s	0.779 s
EN XLM-R	big- $n$ / noisy ( $R^{2}\!=\!0.05$ )	500	0.980 [.950, .992]	0.975 [.943, .989]	0.501 s	26.97 s

Appendix

Table 21: Detection power on real embedding geometries with planted rank-1 SVD signal in real

\mathbf{X}

, 200 reps per cell,

K_{\text{fit}}\!=\!1

\alpha\!=\!0.05

. Bracketed numbers are Wilson 95% CIs. NB strictly beats perm-

Q^{2}

in 4/6 cells; the two big-

n

/ noisy cells where both methods saturate near the ceiling are within Wilson overlap. The EN-XLM-R cell tracks EN-GloVe to within sampling.

Test	Cell A power	Cell B power	Median runtime A / B (s)
NB	0.8125	0.715	0.005 / 0.12
Permutation- $Q^{2}$	0.7325	0.685	0.42 / 7.48
jack.test	0.57	0.16	0.003 / 0.04
Naive $t$	0.99	0.96	0.018 / 0.12

Appendix

Table 22: Detection rate at

\alpha\!=\!0.05

(400 reps

\times

2 cells). Naive-

t

(both cells) and jack.test cell A are non-comparable: those tests are uncalibrated under

H_{0}

in those regimes (FPR 0.3875/0.4075 for naive-

t

, 0.260 for jack-A on the S1 runs), so apparent power is inflated by Type-I leakage; reported for sanity only.

Figure 4: Empirical power vs.

n

on Tecator NIR; same style and conventions as Fig. 3 . The

R^{2}\!=\!0.05

curve plateaus below 80% at

n\!=\!200

for both NB and perm-

Q^{2}

; this ceiling is set by the planted signal’s information content (a deliberately weak

R^{2}\!=\!0.05

effect), not by the chemometric design — both tests track the same ceiling, which is what this comparison is meant to read.

Figure 5: PP plot of NB and perm-

Q^{2}

p

-values under

H_{0}

on Synthetic A (iid Gaussian,

n\!=\!40

K\!=\!1

; left) and Tecator TA (NIR,

n\!=\!40

K\!=\!1

; right). 400 reps per panel. Diagonal = uniform-

H_{0}

reference; vertical gray line marks the

\alpha\!=\!0.05

rejection threshold. NB-asymptotic (blue solid) tracks uniform on synthetic but deviates strongly on Tecator (KS

p\!\sim\!10^{-6}

); perm-

Q^{2}

(orange dashed) tracks uniform on both.

				NB-asymptotic			Exact NB-permutation
Design	$n$	st. rank	rule	$\alpha\!=\!.05$	$\alpha\!=\!.10$	KS	$\alpha\!=\!.05$	$\alpha\!=\!.10$	KS
iid Gaussian	10	7.08	flag ( $n$ )	0.004	0.020	$10^{-70}$	0.056	0.108	0.08
	20	12.98	flag ( $n$ )	0.020	0.059	$2{\times}10^{-13}$	0.049	0.102	0.60
	40	22.04	pass	0.038	0.083	$1.3{\times}10^{-3}$	0.048	0.096	0.81
	80	36.07	pass	0.044	0.091	0.23	0.051	0.099	0.77
	160	55.28	pass	0.050	0.100	0.72	0.052	0.106	0.97

Appendix

Table 23: All null cells behind the validity rule. Stable rank

\|\mathbf{X}\|_{F}^{2}/\|\mathbf{X}\|_{2}^{2}

of the column-standardized design; flag if

n\!<\!25

or stable rank

<\!3

(the

n

column marks a flag raised by the sample-size gate alone). Level is the empirical rejection rate at

\alpha\!=\!.05

and

.10

; KS is the Kolmogorov–Smirnov

p

of the null

p

-values against Uniform

[0,1]

. 5,000 replicates per cell unless marked ( w 2,000; t NB-asymptotic 400, exact 2,000). Wilson 95% half-width at 5,000 replicates:

\pm 0.006

\alpha\!=\!.05

\pm 0.008

.10

$\|a\|$	share	$\bar{z}$	$s^{2}/\sigma_{0}^{2}$	splits $r_{j}\!>\!0$	$\hat{\rho}$	rej. .01	rej. .05	rej. .10
$[0,0.10)$	0.47	$-0.107$	0.38	0.25	0.63	0.000	0.000	0.000
$[0.10,0.25)$	0.41	$0.036$	0.91	0.66	0.19	0.001	0.003	0.015
$[0.25,0.40)$	0.11	$0.271$	0.92	0.93	0.26	0.101	0.275	0.450
$[0.40,1]$	0.01	$0.485$	0.68	0.99	0.41	0.711	0.842	0.974

Appendix

Table 24: One-factor design ( col1 construction, pairwise

r\!=\!0.9

n\!=\!40

J\!=\!50

, 4,000 null replicates with the splits held fixed, binned by

|a|\!=\!|\mathrm{cor}(\mathbf{u}_{1},\mathbf{y})|

s^{2}/\sigma_{0}^{2}

is the across-split variance of

z

relative to the single-split null variance

1/(n_{2}\!-\!3)

;

\hat{\rho}

is the jeff estimate

\max\{0,1-s^{2}/\sigma_{0}^{2}\}

quantile	0.50	0.80	0.90	0.95	0.99	0.999
$t_{\mathrm{NB}}$	$-0.25$	0.47	0.95	1.50	2.74	3.62
$t_{49}$	0.00	0.85	1.30	1.68	2.41	3.27

Appendix

Table 25: Null quantiles of

t_{\mathrm{NB}}

on the same 4,000 replicates against the

t_{49}

reference the asymptotic test uses.

$r$	st. rank	$\rho$	$\hat{\rho}$	$\|\cos\|$	$\cos$	sign	lvl .01	lvl .05	lvl .10
0.9	1.1	0.33	0.38	0.67	0.41	0.99	0.017	0.044	0.067
0.7	1.4	0.35	0.40	0.55	0.44	0.97	0.021	0.044	0.071
0.5	2.0	0.35	0.39	0.50	0.46	0.94	0.017	0.045	0.077
0.3	3.3	0.36	0.39	0.48	0.47	0.89	0.017	0.047	0.077
0.1	8.4	0.43	0.43	0.47	0.47	0.77	0.019	0.054	0.084
0 (iid)	22.2	0.43	0.44	0.47	0.47	0.58	0.003	0.029	0.078

Appendix

Table 26: Sweep over the pairwise correlation

r

inside the factor (

n\!=\!40

p\!=\!300

J\!=\!50

, 2,000 null replicates per row, splits held fixed).

\rho

is the mean pairwise correlation between split estimates across replicates;

\hat{\rho}

the mean jeff estimate;

|\cos|

and

\cos

the mean absolute and signed cosine between the weight vectors of two splits; sign the share of splits whose held-out sign equals

\mathrm{sign}(\mathbf{u}_{R}^{\top}\mathbf{y}_{R}\cdot\mathbf{u}_{T}^{\top}\mathbf{y}_{T})

$p$	NB	exact	ratio	$n$	NB	exact	ratio	perm- $Q^{2}$	ratio
75	0.0042	0.088	20.8	400	0.033	0.28	8.6	3.8	117
150	0.0082	0.094	11.4	800	0.068	0.96	14.1	7.9	116
300	0.0166	0.106	6.4	1,600	0.144	2.14	14.8	17.3	120
600	0.0339	0.131	3.9	3,200	0.344	4.32	12.6	43.0	125
1,200	0.0705	0.180	2.5	6,400	0.779	10.8	13.8	89.9	115
				13,365	3.22	24.0	7.5	387.7	120

Appendix

Table 27: Wall-clock seconds per test. Left:

p

sweep at

n\!=\!200

(synthetic pool). Right: Warriner GloVe-300 ladder at

p\!=\!300

, with perm-

Q^{2}

as a fourth method. Ratios are relative to NB-asymptotic.

cell	$R^{2}$	$\alpha$	step $m+1$ rate (95% CI)	KS-uniform $p$	chain rate (95% CI)	min prior rate
M0R10	0.10	0.01	0.003 [.000, .014]	0.82	0.003 [.000, .014]	—
M0R10	0.10	0.05	0.035 [.021, .058]	0.82	0.035 [.021, .058]	—
M0R10	0.10	0.10	0.085 [.061, .116]	0.82	0.085 [.061, .116]	—
M0R25	0.25	0.01	0.015 [.007, .032]	0.36	0.015 [.007, .032]	—
M0R25	0.25	0.05	0.055 [.037, .082]	0.36	0.055 [.037, .082]	—
M0R25	0.25	0.10	0.128 [.098, .164]	0.36	0.128 [.098, .164]	—

Appendix

Table 28: S5 — marginal and chain NB rejection rates beyond

m

planted axes on Warriner-EN GloVe-300 valence, 400 reps per cell, Wilson 95% CIs. M0 cells (full null) report all three

\alpha

levels; m

\geq

1 cells report

\alpha\!=\!0.05

only (full

\alpha

-grid in s5_summary.csv ).

cell	$R^{2}$	$\alpha$	step $m+1$ rate (95% CI)	KS-uniform $p$	chain rate (95% CI)	min prior rate
TM0R10	0.10	0.01	0.015 [.007, .032]	$\sim\!10^{-9}$	0.015 [.007, .032]	—
TM0R10	0.10	0.05	0.033 [.019, .055]	$\sim\!10^{-9}$	0.033 [.019, .055]	—
TM0R10	0.10	0.10	0.060 [.041, .088]	$\sim\!10^{-9}$	0.060 [.041, .088]	—
TM0R25	0.25	0.01	0.013 [.005, .029]	$\sim\!10^{-4}$	0.013 [.005, .029]	—
TM0R25	0.25	0.05	0.030 [.017, .052]	$\sim\!10^{-4}$	0.030 [.017, .052]	—
TM0R25	0.25	0.10	0.068 [.047, .096]	$\sim\!10^{-4}$	0.068 [.047, .096]	—

Appendix

Table 29: T4 — marginal and chain NB rejection rates beyond

m

planted axes on Tecator NIR, 400 reps per cell, Wilson 95% CIs (full

\alpha

-grid in t4_summary.csv ).

Data	Pipeline	$n$	exact	perm- $Q^{2}$	NB	exact	perm- $Q^{2}$	NB	KS (exact)
			power, $\theta\!=\!0.7$			level, $\alpha\!=\!.05$
Warriner	sPCA	40	0.41	0.28	0.19	0.050	0.049	0.010	0.85
Warriner	ridge	40	0.43	0.28	0.37	0.051	0.058	0.040	0.53
Warriner	PLS1	40	0.46	0.34	0.41	0.048	0.054	0.044	0.31
Warriner	sPCA	200	1.00	0.98	1.00	0.055	0.049	0.015	0.40
Warriner	ridge	200	1.00	0.70	0.98	0.051	0.052	0.016	0.53

Appendix

Table 30: R6 — the held-out-refit machinery outside PLS. Power at

\theta\!=\!0.7

(400 replicates) and rejection rate under

H_{0}

\alpha\!=\!.05

(2,000 replicates; Wilson half-width

\pm 0.010

) for the exact NB-permutation test, perm-

Q^{2}

, and NB-asymptotic; KS is the KS-uniform

p

of the exact test’s null

p

-values. NB-asymptotic is Warriner-only (see text). At

n\!=\!200/172

the exact test and NB reach

\approx\!1.00

power everywhere and only perm-

Q^{2}

trails.

Cell	$K$	$s_{k}$	Total $\mathbf{r}^{2}$	Unique features used
dense ( $s\!=\!D$ , $\equiv$ NIPALS PLS1)	3	300	0.631	300
cardinality sweep, $K\!=\!3$	3	200	0.630	287
	3	100	0.621	218
	3	50	0.605	133
	3	25	0.575	71
more axes, sparser each	4	100	0.634	250

Appendix

Table 31: Hard top-

s

sparse-PLS sweep on Warriner-EN valence (Config 1, GloVe-300). Each row is an independent sparse-PLS fit;

r^{2}

values are in-sample on the standardized response and represent the intrinsic fit at the given cardinality. The

s\!=\!D\!=\!300

row coincides with NIPALS PLS1 by construction (it is the natural endpoint of the cardinality grid, not a comparison baseline). Per-cell top-15 word leaderboards on

\mathbf{w}_{1}

in Tab. 32 .

		dim-1	dim-2	dim-3	dim-4	dim-5
Rank	Side	R²=0.506	R²=0.064	R²=0.036	R²=0.018	R²=0.010
1	pos	wonderful	easygoing	significant	madden	localized
2	pos	showcase	affectionate	growth	shovel	eminence
3	pos	superb	togetherness	stimulation	sequel	reinstate
4	pos	fabulous	goofball	facilitate	donkey	loveliness
5	pos	fantastic	spunky	employment	jolly	sovereign

Appendix

Table 32: Top-5 and bottom-5 words on each of five sparse axes for the

K\!=\!5,s\!=\!50

cell on Warriner-EN valence (see Tab. 31 ). Each axis is supported on

\leq 50

features; the columns are read directly off

\mathbf{w}_{k}

, not off a varimax-rotated loading. Dim-5 negative-end <slur> placeholders mask racial, homophobic, or strong-expletive vocabulary present in the Warriner ratings.

Permutation Testing

See all 12 papers

Improving Improved Kernel PLS
Jul 17, 2026Ole-Christian Galbo EngstrømHigh-Performance Computing Kernel Methods
FOSS Analytical A/S, Nils Foss All´e 1, Hillerød, 3400, Denmark
Exact Stiefel Optimization for Probabilistic PLS: Closed-Form Updates, Error Bounds, and Calibrated Uncertainty
May 12, 2026Haoran Hu, Xingce WangParameter Estimation Latent Variable Models
School of Artificial Intelligence Beijing Normal University Beijing 100875, China
Principal Component Regression Dominates all Monotone Spectral Filters for Linear Regression
Sep 30, 2026Juno Kim, Hengyu Fu, Peter Bartlett +2Spectral Regularization Linear Regression
University of California, Berkeley · Google DeepMind

Jul 17, 2026J/K move · Enter open · S save

Improving Improved Kernel PLS

Ole-Christian Galbo Engstrøm

FOSS Analytical A/S, Nils Foss All´e 1, Hillerød, 3400, Denmark

High-Performance Computing Kernel Methods

Open paper AskPDF arXiv

Improved Kernel Partial Least Squares (IKPLS) algorithms 1 and 2 are among the fastest PLS calibration algorithms. This article focuses on two shared steps, the computation of the

\mathbf{X}

rotations,

\mathbf{R}

, and the

\mathbf{Y}

loadings,

\mathbf{Q}

, and accelerates both. For

\mathbf{R}

, term-by-term accumulation is replaced by a direct evaluation strategy that requires the same number of multiplications but parallelizes better on modern hardware. For

\mathbf{Q}

, I identify - to the best of my knowledge, for the first time - equivalences showing that each

\mathbf{Y}

loading is obtainable, up to explicitly derived constants, from quantities already computed earlier in the same iteration, and I exploit them in IKPLS to reduce the cost of each loading from

Θ\left(KM\right)

Θ\left(M\right)

operations whenever

M = 1

2 \leq M < K

, with

K

predictor variables (number of columns in

\mathbf{X}

) and

M

response variables (number of columns in

\mathbf{Y}

). Both improvements provably yield exactly the same

\mathbf{W}

\mathbf{P}

\mathbf{Q}

\mathbf{R}

, and

\mathbf{T}

as the original algorithms. Benchmarks with NumPy (CPU) and JAX (GPU) show speedups of up to two orders of magnitude for the isolated steps and of approximately

2\times

(CPU) and

6\times

(GPU) for entire fits. Both improvements are implemented in the free, open-source Python package ikpls.

Abstract

Permutation Testing

Explore similar work

Improving Improved Kernel PLS

Exact Stiefel Optimization for Probabilistic PLS: Closed-Form Updates, Error Bounds, and Calibrated Uncertainty

Principal Component Regression Dominates all Monotone Spectral Filters for Linear Regression