Selective Inference for Clustering Under Unknown Covariance

Yiqun T. Chen; Lucy L. Gao

stat.MEOct 12, 2026

Selective Inference for Clustering Under Unknown Covariance

Authors: Yiqun T. Chen, Lucy L. Gao

Organizations: Departments of Biostatistics and Computer Science, Johns Hopkins University · Department of Statistics and Actuarial Science, University of Waterloo

Abstract

Unsupervised clustering is widely used to discover underlying group structure, after which researchers often seek to validate these groups via post-hoc hypothesis testing on the population means. However, using the same data both to form clusters and to test for a difference in means inflates the Type I error rate. Recent work in selective inference has proposed solutions when the underlying data are drawn from a multivariate Gaussian distribution with a known covariance matrix or an unknown isotropic covariance matrix. We propose a new selective test for the difference in means between two clusters that provably controls the Type I error rate under an arbitrary unknown covariance matrix. Our test statistic does not require estimating the feature covariance matrix; instead, it leverages a Monte Carlo sampling procedure over the set of orthogonal matrices. We show that this new selective test can be computed efficiently, and we derive closed-form solutions when clusters are formed using k-means or hierarchical clustering. In simulations, we demonstrate that our proposed p-value controls the selective Type I error rate and achieves substantial power, and we illustrate its application on gene expression and drug review data sets.

Figures & tables

Figure 1: Estimating the covariance trades Type I error control against power. (a) One data set from ( 1 ) with two true groups (shapes), split into

K=3

clusters (colors) by

k

-means, which divides one true group in two. (b) QQ plots of the Uniform

(0,1)

distribution against the

p

-values when all observations come from one Gaussian group: the naive test, the test of Chen and Witten (2023) with three choices of covariance (

p_{\mathrm{CW}}

, Section 5 ), and our proposals

p_{\mathrm{polar}}

in ( 13 ),

p_{\mathrm{polar,1d}}

in ( 19 ), and

p_{\mathrm{polar,1d}}^{\mathrm{traj}}

of Section 4.4 . (c) Conditional power against the separation

\delta

between the true groups, for the tests that control the Type I error in (b).

Figure 2: The split

\mathbf{X}=\mathbf{P}_{0}\mathbf{X}+\mathbf{P}_{1}\mathbf{X}+\mathbf{P}_{2}\mathbf{X}

for

n=36

observations from three bivariate Gaussian groups, clustered into

K=3

clusters by

k

-means; open circles are the cluster that is not tested. (a) The data. (b) The three parts:

\mathbf{P}_{0}\mathbf{X}

joins the two tested cluster means,

\mathbf{P}_{1}\mathbf{X}

covers the deviations within each tested cluster, and

\mathbf{P}_{2}\mathbf{X}

covers the untested observations (it also holds the overall mean of the two tested clusters). (c) A data set with

\mathbf{P}_{1}\mathbf{X}

and

\mathbf{P}_{2}\mathbf{X}

held fixed, as in Gao et al. (2024) : the two tested clusters keep their shapes and move along the line between their means. (d) A data set with

\mathbf{P}_{2}\mathbf{X}

and

\mathbf{H}

held fixed, as in our test: the tested observations are redrawn around the same overall mean with the same sample covariance, and the untested observations do not move. The dashed ellipse in (b)–(d) is the

95\%

ellipse of the sample covariance matrix

\mathbf{H}^{2}/(r-1)

of the tested observations; it is the same in (b) and (d) and smaller in (c).

Figure 3: Two ways to perturb the data, for

n=12

observations from one bivariate Gaussian split into

K=3

clusters by

k

-means; open circles are the cluster that is not tested. (a) The perturbation of Gao et al. (2024) and Chen and Witten (2023) slides the two tested clusters apart along

\bm{\nu}

, so every perturbed data set lies on a line. (b) Our perturbation

\mathbf{Q}\mapsto\mathbf{G}_{ij}(\omega)\mathbf{Q}

for one pair of rows

(i,j)

, where

\mathbf{G}_{ij}(\omega)

rotates rows

i

and

j

\mathbf{Q}

by the angle

\omega

(Section 4.1 ): each tested observation moves on an ellipse, purple where

k

-means still returns the tested clusters. (c) The angles

\omega

on the unit circle, purple where

k

-means returns the tested clusters, with an accepted proposal (filled triangle) and a rejected proposal (open triangle); the observed data are at

\omega=0

. (d) The data at the accepted angle, colored by the clusters that

k

-means returns on them. (e) The same at the rejected angle; the ringed observations have changed cluster.

Figure 4: $k$ -means with $K=3$ . (a) Quantile-quantile plots of the $p$ -values under ( 1 ) with $\bm{\mu}=\mathbf{0}_{n\times p}$ , stratified by $\bm{\Sigma}$ and pooled over $n$ and $p$ . (b) The conditional power ( 29 ) at $\alpha=0.05$ under ( 1 ) with $\bm{\mu}$ defined in ( 28 ), $n=50$ , and $\bm{\Sigma}=\mathbf{I}_{p}$ , stratified by $p$ , for the tests that control the selective Type I error. (c) The recovery probability ( 30 ) in the same settings.

Figure 5: Average-linkage hierarchical clustering with $K=3$ and $n=200$ . (a) Quantile-quantile plots of the $p$ -values under ( 1 ) with $\bm{\mu}=\mathbf{0}_{n\times p}$ , stratified by $\bm{\Sigma}$ and pooled over $p$ . (b) The recovery probability ( 30 ) under ( 1 ) with $\bm{\mu}$ defined in ( 28 ) and $\bm{\Sigma}=\mathbf{I}_{p}$ . (c) The conditional power ( 29 ) at $\alpha=0.05$ in the same settings, stratified by $p$ , for the tests that control the selective Type I error under every $\bm{\Sigma}$ ; at each $\delta$ the tests are drawn side by side, with $95\%$ Wilson intervals.

Figure 6: The leukemia data of Section 6.1 . Left: a UMAP embedding, colored by cluster, with the known label as the plotting symbol. Right: naive and selective

p

-values for each pair of clusters. The row Expected is “alternative” when the two clusters have different majority labels and “null” when they share one.

Figure 7: The drug reviews of Section 6.2 . Top: a UMAP embedding of their principal components, colored by cluster, with the condition as the plotting symbol; left, the “alternative” data set; right, the “null” data set. Bottom: naive and selective

p

-values for each pair of clusters. The column Expected is “alternative” when the two clusters have different majority labels and “null” when they share one. Majority labels: Dep, depression; BC, birth control; BC-, birth control with a rating of

4

or less.

Figure 8: The mouse brain cells of Section 6.3 . Top: a UMAP embedding, colored by cluster, with the known label as the plotting symbol; left, the “alternative” data set; right, the “null” data set. Bottom: naive and selective

p

-values for each pair of clusters. The column Expected is “alternative” when the two clusters have different majority labels and “null” when they share one. Majority labels: IN, interneurons; CA1, CA1 pyramidal cells; OL, oligodendrocytes. No existing selective inference test applies to Leiden clustering.

Appendix figures & tables12 assets

Supplementary material from the paper’s appendix.

Appendix

	Gao et al. (2024)	Yun and Barber (2023)	$p_{\mathrm{polar,1d}}$
	( $\bm{\Sigma}$ known)	( $\bm{\Sigma}=\sigma^{2}\mathbf{I}_{p}$ )	(any $\bm{\Sigma}$ )
null	( 2 )	( 5 )	( 5 )
nuisance parameters	$(\mathbf{P}_{1}+\mathbf{P}_{2})\bm{\mu}$	means, $\sigma^{2}$	means, $\bm{\Sigma}$
removed by fixing	$\mathbf{P}_{1}\mathbf{X}$ , $\mathbf{P}_{2}\mathbf{X}$	$\mathbf{P}_{2}\mathbf{X}$ , $\\|(\mathbf{P}_{0}+\mathbf{P}_{1})\mathbf{X}\\|_{F}$	$\mathbf{P}_{2}\mathbf{X}$ , $\mathbf{H}$
fixed to reach one dimension	$\mathrm{dir}(\bm{\Sigma}^{-1/2}\mathbf{X}^{\top}\bm{\nu})$	$\mathrm{dir}(\mathbf{P}_{0}\mathbf{X})$ , $\mathrm{dir}(\mathbf{P}_{1}\mathbf{X})$	$\bm{\gamma}$ , $\mathrm{dir}(\mathbf{P}_{1}\mathbf{q})$ , $\mathbf{Q}(\mathbf{I}_{p}-\bm{\gamma}\bm{\gamma}^{\top})$
random scalar	$\\|\bm{\Sigma}^{-1/2}\mathbf{X}^{\top}\bm{\nu}\\|_{2}$	$R$	$T^{2}$

Appendix

Table A.1: Nuisance parameters and conditioning in three one-dimensional selective tests. Each test conditions on the selection event, on statistics that remove the nuisance parameters under its null, and on further statistics that leave one scalar random.

Figure C.1: Path length changes power but not the Type I error (

n=45

p=10

K=3

). Left: selective Type I error against the path length

M

on null data, each replicate evaluated at every

M

. Right: conditional power against

M

for five separations

\delta

(one curve each).

Figure C.2: The random Givens kernel and the Cayley random walk with

M=5{,}000

and

L=2

, on null data with compound-symmetric

\bm{\Sigma}

. (a) Quantile-quantile plots of the

p

-values against the Uniform

(0,1)

distribution. (b) Seconds of computing time per

p

-value. (c) The fraction of steps at which the chain moves.

Method	isotropic	AR(1)	comp. sym.	spiked
$p_{\mathrm{naive}}$	1.000 [1.000]	1.000 [1.000]	1.000 [1.000]	1.000 [1.000]
$p_{\mathrm{polar}}$	0.053 [0.064]	0.048 [0.065]	0.049 [0.070]	0.048 [0.064]
$p_{\mathrm{polar,1d}}$	0.052 [0.067]	0.048 [0.063]	0.054 [0.068]	0.043 [0.068]
$p_{\mathrm{polar,1d}}^{\mathrm{traj}}$	0.054 [0.073]	0.053 [0.067]	0.048 [0.072]	0.050 [0.060]
$p_{\mathrm{polar}}(\bm{\Sigma})$	0.050 [0.073]	0.053 [0.073]	0.051 [0.062]	0.049 [0.073]
$p_{\mathrm{CW}}(\hat{\sigma}^{2}_{\mathrm{MED}}\mathbf{I})$	0.058 [0.072]	0.103 [0.170]	0.171 [0.693]	0.122 [0.159]

Appendix

Table D.1: Selective Type I error at

\alpha=0.05

after

k

-means, by covariance: median [maximum] over the cells

(n,p)

of Section 5.1 .

Figure D.1: Selective Type I error at

\alpha=0.05

after

k

-means, by covariance structure, on a square-root scale. Each box summarizes one test over the cells

(n,p)

, which are drawn as points; the naive test is omitted. The gray band is the

95\%

Monte Carlo interval for a test of exact level

0.05

Figure D.2: Selective Type I error at

\alpha=0.05

after

k

-means for the tests that we propose, stratified by

n

(rows) and

\bm{\Sigma}

(columns). Each box summarizes one test over the values of

p

with

p\leq n/2

, which are drawn as points. The gray band is as in Figure D.1 .

Figure D.3: Conditional power ( 29 ) at

\alpha=0.05

after

k

-means with

\bm{\Sigma}=\mathbf{I}_{p}

, stratified by

n

(rows) and

p

(columns), with

95\%

Wilson intervals.

Figure D.4: As Figure D.3 , at

n=100

and

200

Figure D.5: As Figure D.3 , at

n=30

under the AR(1), compound-symmetric, and spiked covariances (rows), for the tests that control the selective Type I error under them.

Figure D.6: Leiden community detection in the design of Figure 1 . (a) Quantile-quantile plots of the

p

-values with one true group (

\delta=0

), stratified by

p

. (b) The conditional power ( 29 ) of

p_{\mathrm{polar}}

\alpha=0.05

against

\delta

, with

95\%

Wilson intervals.

Figure D.7: Selective Type I error at

\alpha=0.05

with rows that are not Gaussian (Section 5.3 ), after

k

-means (left) and average-linkage hierarchical clustering (right), on a square-root scale. Each box summarizes one test over the values of

p

(and of

q

for principal components), which are drawn as points. The gray band is as in Figure D.1 . There is no reference

p_{\mathrm{polar}}(\bm{\Sigma})

for principal components.

Figure D.8: Median seconds for one

p

-value after

k

-means at

n=200

, on one core, by

p

. The vertical axis is on a log scale.

Selective Inference

Selective inference develops valid statistical conclusions after data-dependent choices such as feature selection, clustering, or adaptive evaluation.

See all 20 papers

Sep 23, 2026

Selective Inference for Deep Clustering in Latent Spaces

Deep clustering is a powerful approach for discovering meaningful structures in high-dimensional data by learning a low-dimensional latent representation prior to clustering. Despite its empirical success, assessing the statistical reliability of the resulting clusters remains challenging. Testing discovered clusters on the same data induces selection bias and invalidates classical

p

-values. Selective inference (SI) provides a principled framework for correcting this bias, but existing methods focus on clustering performed directly on the observed features. In this work, we develop an SI framework for deep clustering with a fixed pretrained encoder. The key challenge is that cluster assignments are determined through a nonlinear transformation from the original data space to the latent space, resulting in a substantially more complex selection process than in conventional clustering. Our method provides a computationally tractable way to account for this process and enables valid statistical testing of differences between clusters identified in the latent space. Synthetic experiments demonstrate that the proposed method controls the Type I error rate while achieving higher power than valid but conservative baselines, and genomic applications show that it can identify significant cluster differences while appropriately accounting for selection bias. Our framework provides a principled approach to quantifying the statistical reliability of structures discovered by deep clustering.

Eina Mizui, Tomohiro Shiraishi, Shunichi Nishino +1

Nagoya University · RIKEN

Selective Inference Clustering Unsupervised Clustering Deep Clustering Statistical Hypothesis Testing

Ask PDF

Mar 3, 2026

The elbow statistic: Multiscale clustering statistical significance

Selecting the number of clusters remains a fundamental challenge in unsupervised learning. Existing approaches typically focus on identifying a single "optimal" partition, often overlooking statistically meaningful structure present across multiple resolutions. We introduce ElbowSig, a general inferential framework for assessing clustering structure over a range of resolutions. The method formalizes the elbow heuristic by defining a normalized discrete curvature statistic based on the sequence of within-cluster heterogeneity values, and evaluates its significance relative to a null distribution of unstructured data. This yields hypothesis tests across resolutions, enabling simultaneous inference at multiple clustering scales. We derive the asymptotic behavior of the null statistic in both large-sample and high-dimensional regimes, characterizing its limiting form and variability. Because it depends only on the heterogeneity sequence, ElbowSig is compatible with a wide range of clustering algorithms, including hard, fuzzy, and model-based methods. Experiments on synthetic and real datasets show that the procedure controls Type-I error under unstructured data while providing power to detect multiscale organization, revealing structure that is often missed by single-resolution selection criteria.

Francisco J. Perez-Reche

School of Natural and Computing Sciences, University of Aberdeen, Fraser Noble Building, AB24 3UE, United Kingdom

Cluster Number Estimation Clustering Statistical Hypothesis Testing

Ask PDF

Mar 19, 2026

Statistical Testing Framework for Clustering Pipelines by Selective Inference

A data analysis pipeline is a structured sequence of steps that transforms raw data into meaningful insights by integrating multiple analysis algorithms. In many practical applications, analytical findings are obtained only after data pass through several data-dependent procedures within such pipelines. In this study, we address the problem of quantifying the statistical reliability of results produced by data analysis pipelines. As a proof of concept, we focus on clustering pipelines that identify cluster structures from complex and heterogeneous data through procedures such as outlier detection, feature selection, and clustering. We propose a novel statistical testing framework to assess the significance of clustering results obtained through these pipelines. Our framework, based on selective inference, enables the systematic construction of valid statistical tests for clustering pipelines composed of predefined components. We prove that the proposed test controls the type I error rate at any nominal level and demonstrate its validity and effectiveness through experiments on synthetic and real datasets.

Yugo Miyata, Tomohiro Shiraishi, Shuichi Nishino +1

Nagoya University · RIKEN

Statistical Hypothesis Testing Clustering Selective Inference

Ask PDF

Abstract

Selective Inference

Explore similar work

Selective Inference for Deep Clustering in Latent Spaces

The elbow statistic: Multiscale clustering statistical significance

Statistical Testing Framework for Clustering Pipelines by Selective Inference