Statistical Testing Framework for Clustering Pipelines by Selective Inference

Yugo Miyata; Tomohiro Shiraishi; Shuichi Nishino; Ichiro Takeuchi

stat.MLMar 19, 2026

Statistical Testing Framework for Clustering Pipelines by Selective Inference

Authors: Yugo Miyata, Tomohiro Shiraishi, Shuichi Nishino, Ichiro Takeuchi

Organizations: Nagoya University · RIKEN

Abstract

A data analysis pipeline is a structured sequence of steps that transforms raw data into meaningful insights by integrating multiple analysis algorithms. In many practical applications, analytical findings are obtained only after data pass through several data-dependent procedures within such pipelines. In this study, we address the problem of quantifying the statistical reliability of results produced by data analysis pipelines. As a proof of concept, we focus on clustering pipelines that identify cluster structures from complex and heterogeneous data through procedures such as outlier detection, feature selection, and clustering. We propose a novel statistical testing framework to assess the significance of clustering results obtained through these pipelines. Our framework, based on selective inference, enables the systematic construction of valid statistical tests for clustering pipelines composed of predefined components. We prove that the proposed test controls the type I error rate at any nominal level and demonstrate its validity and effectiveness through experiments on synthetic and real datasets.

Figures & tables

Figure 1: Two examples of clustering pipelines composed of outlier detection (OD), feature selection (FS), and clustering components, for which the proposed framework provides statistically valid $p$ -values without additional implementation effort.

Figure 2: Schematic diagram of the proposed line search method to identify the truncated region $\mathcal{Z}$ . The top part shows the DAG representation of the pipeline and its topological sorting (i). The lower left part shows the operations executed sequentially according to the update rules (ii). The lower right part shows how the truncated region $\mathcal{Z}$ is identified by taking the union of several intervals based on parametric-programming (iii).

Panel of Figure 3 — (a) Type I Error Rate of option1 pipeline

Table 1: Results for clustered and non-clustered data using option1 (PBMC 3k).

Table 2: Results for clustered and non-clustered data using option2 (Marketing Campaign).

Appendix figures & tables14 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 4: Algorithm components of the clustering pipeline.

Panel of Figure 7 — (a) Type I Error Rate of option1 pipeline

Table 3: Results of option2 for clustered and non-clustered data (PBMC 3k).

Panel of Figure 6 — Figure 6: Data distributions before and after clustering for feature #6 (JUNB) in the clustered data. Cluster 1: 138 samples, Cluster 2: 55 samples. Although the cluster sizes are somewhat imbalanced, a clear difference in means is observed between the two clusters.

#	feature (gene name)	naive	w/o-pp	proposed (SI)
6	JUNB	0.000	0.478	0.000

Appendix

Table 4:

p

-values for feature #6 (JUNB) in the clustered data

#	feature (gene name)	naive	w/o-pp	proposed (SI)
5	JUN	0.000	0.977	0.981

Appendix

Table 5:

p

-values for feature #5 (JUN) in the non-clustered data

Table 6: Results of option1 for clustered and non-clustered data (Marketing Campaign).

Panel of Figure 8 — Figure 8: Data distributions before and after clustering for feature #4 (MntWines) in the clustered data. Cluster 1: 102 samples, Cluster 2: 90 samples. Two clusters of nearly equal size are formed, and a clear difference in means is observed between them.

#	feature (spending target)	naive	w/o-pp	proposed (SI)
4	MntWines	0.000	0.250	0.003

Appendix

Table 7:

p

-values for feature #4 (MntWines) in the clustered data

Panel of Figure 9 — Figure 9: Data distributions before and after clustering for feature #8 (MntSweetProducts) in the non-clustered data. Cluster 1: 144 samples, Cluster 2: 7 samples. Although the cluster sizes are heavily imbalanced, no meaningful difference in means is observed between the two clusters.

#	feature (spending target)	naive	w/o-pp	proposed (SI)
8	MntSweetProducts	0.000	0.162	0.299

Appendix

Table 8:

p

-values for feature #8 (MntSweetProducts) in the non-clustered data

Panel of Figure 10 — Figure 10: Robustness of Type I error rate control under estimated variance. The proposed method successfully controls the Type I error rate below the significance level $\alpha$ even when the variance is estimated from the same data.

Figure 11: Robustness of Type I error rate control under non-Gaussian noise. The proposed method successfully controls the Type I error rate at the significance level

\alpha

regardless of the Wasserstein distance from the Gaussian distribution.

Statistical Hypothesis Testing

Statistical hypothesis testing develops methods for assessing evidence against a stated null while controlling false-positive and false-negative rates.

See all 98 papers

Selective Inference for Deep Clustering in Latent Spaces
Sep 23, 2026Eina Mizui, Tomohiro Shiraishi, Shunichi Nishino +1Selective Inference Clustering
Nagoya University · RIKEN
The elbow statistic: Multiscale clustering statistical significance
Mar 3, 2026Francisco J. Perez-RecheCluster Number Estimation Clustering
School of Natural and Computing Sciences, University of Aberdeen, Fraser Noble Building, AB24 3UE, United Kingdom
Selective Inference for Clustering Under Unknown Covariance
Oct 12, 2026Yiqun T. Chen, Lucy L. GaoSelective Inference Statistical Hypothesis Testing
Departments of Biostatistics and Computer Science, Johns Hopkins University · Department of Statistics and Actuarial Science, University of Waterloo

Sep 23, 2026J/K move · Enter open · S save

Selective Inference for Deep Clustering in Latent Spaces

Eina Mizui, Tomohiro Shiraishi, Shunichi Nishino, Ichiro Takeuchi

Nagoya University · RIKEN

Selective Inference Clustering Unsupervised Clustering Deep Clustering Statistical Hypothesis Testing

Open paper AskPDF arXiv

Deep clustering is a powerful approach for discovering meaningful structures in high-dimensional data by learning a low-dimensional latent representation prior to clustering. Despite its empirical success, assessing the statistical reliability of the resulting clusters remains challenging. Testing discovered clusters on the same data induces selection bias and invalidates classical

p

-values. Selective inference (SI) provides a principled framework for correcting this bias, but existing methods focus on clustering performed directly on the observed features. In this work, we develop an SI framework for deep clustering with a fixed pretrained encoder. The key challenge is that cluster assignments are determined through a nonlinear transformation from the original data space to the latent space, resulting in a substantially more complex selection process than in conventional clustering. Our method provides a computationally tractable way to account for this process and enables valid statistical testing of differences between clusters identified in the latent space. Synthetic experiments demonstrate that the proposed method controls the Type I error rate while achieving higher power than valid but conservative baselines, and genomic applications show that it can identify significant cluster differences while appropriately accounting for selection bias. Our framework provides a principled approach to quantifying the statistical reliability of structures discovered by deep clustering.

Abstract

Statistical Hypothesis Testing

Explore similar work

Selective Inference for Deep Clustering in Latent Spaces

The elbow statistic: Multiscale clustering statistical significance

Selective Inference for Clustering Under Unknown Covariance