A Data-Informed Variational Clustering Framework for Noisy High-Dimensional Data

Wan Ping Chen

stat.MLApr 8, 2026

A Data-Informed Variational Clustering Framework for Noisy High-Dimensional Data

Authors: Wan Ping Chen

Organizations: Department of Mathematics, Fu Jen Catholic University, Taipei, Taiwan, No. 510 Zhongzheng Rd., Xinzhuang Dist., New Taipei City, 242062, Taiwan

Abstract

Clustering in high-dimensional settings with severe feature noise remains challenging, especially when only a small subset of dimensions is informative and the final number of clusters is not specified in advance. In such regimes, partition recovery, feature relevance learning, and structural adaptation are tightly coupled, and standard likelihood-based methods can become unstable or overly sensitive to noisy dimensions. We propose DIVI, a data-informed variational clustering framework that combines global feature gating with split-based adaptive structure growth. DIVI uses informative prior initialization to stabilize optimization, learns feature relevance in a differentiable manner, and expands model complexity only when local diagnostics indicate underfit. Beyond clustering performance, we also examine runtime scalability and parameter sensitivity in order to clarify the computational and practical behavior of the framework. Empirically, we find that DIVI performs competitively under severe feature noise, remains computationally feasible, and yields interpretable feature-gating behavior, while also exhibiting conservative growth and identifiable failure regimes in challenging settings. Overall, DIVI is best viewed as a practical variational clustering framework for noisy high-dimensional data rather than as a fully Bayesian generative solution.

Figures & tables

	$N=200$ (Small- $n$ )				$N=1000$ (Large- $n$ )
Method	$K$	ARI	NMI	F1	$K$	ARI	NMI	F1
K-means	3.000	.909	.885	–	3.000	.992	.985	–
	(.000)	(.048)	(.053)	–	(.000)	(.006)	(.010)	–
GMM	3.000	.992	.987	–	3.000	.995	.991	–
	(.000)	(.010)	(.016)	–	(.000)	(.004)	(.007)	–
SPKM	3.000	.993	.989	.184	3.000	.996	.993	.190

Table 1: Clustering performance under high-dimensional noise. Results are averaged over 20 independently generated synthetic datasets with

D=100

and 90% irrelevant dimensions. DIVI is evaluated using the current implementation with the sensitivity-selected split interval

T_{\mathrm{split}}=120

. Feature F1 is reported only for methods with explicit feature selection or feature gating.

Scenario	$N$	K-means ARI	GMM ARI	DIVI-Info ARI	DIVI-Info F1
Heavy-tailed signal	200	0.913 (0.034)	0.853 (0.238)	0.989 (0.015)	0.768 (0.095)
Heavy-tailed signal	1000	0.986 (0.005)	0.971 (0.097)	0.964 (0.107)	0.990 (0.020)
Correlated noise	200	0.283 (0.080)	0.307 (0.176)	0.419 (0.153)	0.243 (0.038)
Correlated noise	1000	0.381 (0.037)	0.497 (0.007)	0.712 (0.261)	0.254 (0.031)

Table 2: Main robustness results under model misspecification. DIVI-Info was evaluated with the sensitivity-selected split schedule

(\tau_{\mathrm{mult}},T_{\mathrm{split}})=(1.00,120)

. Values are mean (std) over 20 independently generated datasets.

Figure 1: Interpretability illustration on the UCI Wine dataset. The observations are visualized in the two most relevant dimensions identified by the learned feature-gating mechanism. Although this experiment is small in scale, it shows that the learned relevance profile can produce a meaningful low-dimensional view that aligns reasonably well with known chemical differences among wine categories.

Dataset	Method	ARI	NMI	Time (s)	Final $K$	Active dims
ISOLET	DIVI-Info	0.460 (0.143)	0.585 (0.122)	22.26 (1.14)	4.0 (0.0)	578.0 (1.7)
ISOLET	K-means	0.632 (0.001)	0.716 (0.001)	0.59 (0.04)	5.0 (0.0)	–
ISOLET	GMM	0.580 (0.112)	0.684 (0.068)	2.81 (0.75)	5.0 (0.0)	–
ISOLET	SPKM	0.551 (0.000)	0.640 (0.000)	9.26 (0.66)	5.0 (0.0)	23.0 (0.0)
20NG	DIVI-Info	0.759 (0.133)	0.753 (0.074)	16.49 (1.03)	4.0 (0.0)	351.6 (1.1)
20NG	K-means	0.845 (0.002)	0.803 (0.002)	0.40 (0.02)	4.0 (0.0)	–

Table 3: Real-data clustering and computational summaries on ISOLET and 20NG embeddings, where 20NG denotes the 20 Newsgroups subset. Results are reported as mean (standard deviation) over repeated runs. For K-means, GMM, and SPKM, the true number of clusters is supplied. DIVI starts from a single component and expands via split diagnostics. Active dimensions are reported for feature-selective methods only (DIVI and SPKM).

Figure 2: Runtime scaling with feature dimension

D

on synthetic data under fixed sample size (

N=1000

). DIVI-Info exhibits a monotone increase in wall-clock cost as dimensionality grows, reflecting the increasing cost of feature-wise variational optimization.

Figure 3: Runtime scaling with sample size

N

on synthetic data under fixed dimensionality (

D=1000

). Runtime increases with sample size for all methods, with DIVI-Info incurring the highest computational cost due to joint feature gating and split-based structure adaptation.

Figure 4: Sensitivity of DIVI to the split interval

T_{\mathrm{split}}

on fixed synthetic datasets, measured by ARI. Very small split intervals lead to substantially worse clustering performance, whereas more conservative split intervals yield markedly better ARI across both sample-size regimes.

Figure 5: Sensitivity of DIVI to the KL-scaling multiplier

\beta/N

on fixed synthetic datasets, measured by Feature F1. Larger

\beta/N

improves or stabilizes informative-support recovery, with the gains being most pronounced in the small-

n

setting. The results support

\beta=N

as a practical default for feature-gating regularization.

Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

Panel A: Large- $n$ ( $N=1000$ , $D=100$ )
$T_{\mathrm{split}}$	ARI	NMI	F1	Time (s)	Final $K$	Splits
10	0.414 (0.096)	0.618 (0.048)	1.000 (0.000)	15.21 (2.79)	31.0 (0.0)	30.0 (0.0)
20	0.493 (0.105)	0.665 (0.045)	1.000 (0.000)	6.77 (0.23)	16.0 (0.0)	15.0 (0.0)
40	0.678 (0.103)	0.772 (0.039)	1.000 (0.000)	3.68 (0.25)	8.0 (0.0)	7.0 (0.0)
80	0.879 (0.012)	0.909 (0.006)	1.000 (0.000)	2.43 (0.31)	4.0 (0.0)	3.0 (0.0)
Panel B: Small- $n$ ( $N=200$ , $D=100$ )

Appendix

Table B.1 : Sensitivity of DIVI to the split interval

T_{\mathrm{split}}

on fixed synthetic datasets. Results are mean (standard deviation) over repeated runs.

Panel A: Large- $n$ ( $N=1000$ , $D=100$ )
$\beta/N$	ARI	NMI	F1	Time (s)	Final $K$	Active dims
0.25	0.876 (0.008)	0.908 (0.004)	0.519 (0.017)	2.46 (0.29)	4.0 (0.0)	28.6 (1.3)
0.50	0.877 (0.010)	0.909 (0.005)	0.828 (0.031)	2.59 (0.27)	4.0 (0.0)	14.2 (0.9)
1.00	0.879 (0.012)	0.909 (0.006)	1.000 (0.000)	2.53 (0.20)	4.0 (0.0)	10.0 (0.0)
2.00	0.880 (0.012)	0.910 (0.006)	1.000 (0.000)	2.51 (0.24)	4.0 (0.0)	10.0 (0.0)
4.00	0.880 (0.012)	0.910 (0.006)	1.000 (0.000)	2.53 (0.23)	4.0 (0.0)	10.0 (0.0)

Appendix

Table B.2 : Sensitivity of DIVI to the KL scaling factor

\beta

on fixed synthetic datasets. Results are mean (standard deviation) over repeated runs.

Panel A: Large- $n$ ( $N=1000$ , $D=100$ )
$\tau/\tau_{0}$	ARI	NMI	F1	Time (s)	Final $K$	Active dims
0.9	0.879 (0.012)	0.909 (0.006)	1.000 (0.000)	2.57 (0.20)	4.0 (0.0)	10.0 (0.0)
1.0	0.879 (0.012)	0.909 (0.006)	1.000 (0.000)	2.54 (0.14)	4.0 (0.0)	10.0 (0.0)
1.1	0.879 (0.012)	0.909 (0.006)	1.000 (0.000)	2.74 (0.23)	4.0 (0.0)	10.0 (0.0)
1.2	0.879 (0.012)	0.909 (0.006)	1.000 (0.000)	2.67 (0.18)	4.0 (0.0)	10.0 (0.0)
Panel B: Small- $n$ ( $N=200$ , $D=100$ )

Appendix

Table B.3 : Sensitivity of DIVI to the entropy-threshold multiplier

\tau/\tau_{0}

on fixed synthetic datasets. Results are mean (standard deviation) over repeated runs.

Panel A: Large- $n$ ( $N=1000$ , $D=100$ )
Learning rate	ARI	NMI	F1	Time (s)	Final $K$	Active dims
0.005	0.901 (0.027)	0.915 (0.021)	1.000 (0.000)	2.34 (0.17)	4.0 (0.0)	10.0 (0.0)
0.010	0.879 (0.012)	0.909 (0.006)	1.000 (0.000)	2.58 (0.26)	4.0 (0.0)	10.0 (0.0)
0.020	0.876 (0.008)	0.908 (0.004)	0.995 (0.015)	2.80 (0.33)	4.0 (0.0)	10.1 (0.3)
0.050	0.877 (0.009)	0.909 (0.004)	0.990 (0.020)	3.00 (0.21)	4.0 (0.0)	10.2 (0.4)
Panel B: Small- $n$ ( $N=200$ , $D=100$ )

Appendix

Table B.4 : Sensitivity of DIVI to the optimizer learning rate on fixed synthetic datasets. Results are mean (standard deviation) over repeated runs.

Panel A: Large- $n$ ( $N=1000$ , $D=100$ )
$T_{\min}$	ARI	NMI	F1	Time (s)	Final $K$	Active dims
0.01	0.879 (0.012)	0.910 (0.006)	1.000 (0.000)	2.80 (0.21)	4.0 (0.0)	10.0 (0.0)
0.05	0.879 (0.012)	0.910 (0.006)	1.000 (0.000)	2.61 (0.23)	4.0 (0.0)	10.0 (0.0)
0.10	0.879 (0.012)	0.909 (0.006)	1.000 (0.000)	2.55 (0.21)	4.0 (0.0)	10.0 (0.0)
0.20	0.880 (0.013)	0.910 (0.007)	1.000 (0.000)	2.52 (0.14)	4.0 (0.0)	10.0 (0.0)
Panel B: Small- $n$ ( $N=200$ , $D=100$ )

Appendix

Table B.5 : Sensitivity of DIVI to the temperature annealing endpoint

T_{\min}

on fixed synthetic datasets. Results are mean (standard deviation) over repeated runs.

Method	$N$	Final $K$	ARI	NMI	F1
K-means (oracle $K$ )	200	3.00 (0.00)	0.913 (0.034)	0.886 (0.036)	–
Std. GMM (oracle $K$ )	200	3.00 (0.00)	0.853 (0.238)	0.875 (0.187)	–
DIVI-Info	200	3.00 (0.00)	0.989 (0.015)	0.983 (0.022)	0.768 (0.095)
DIVI-NonInfo	200	3.00 (0.00)	0.906 (0.173)	0.910 (0.124)	0.182 (0.000)
DIVI-Random	200	3.00 (0.00)	0.814 (0.190)	0.819 (0.161)	0.187 (0.041)
K-means (oracle $K$ )	1000	3.00 (0.00)	0.986 (0.005)	0.974 (0.009)	–

Appendix

Table B.6 : Full robustness results under heavy-tailed signal. All DIVI variants were evaluated with the sensitivity-selected split schedule

(\tau_{\mathrm{mult}},T_{\mathrm{split}})=(1.00,120)

. Values are mean (standard deviation) over 20 independently generated datasets.

Method	$N$	Final $K$	ARI	NMI	F1
K-means (oracle $K$ )	200	3.00 (0.00)	0.283 (0.080)	0.307 (0.084)	–
Std. GMM (oracle $K$ )	200	3.00 (0.00)	0.307 (0.176)	0.349 (0.194)	–
DIVI-Info	200	3.00 (0.00)	0.419 (0.153)	0.466 (0.154)	0.243 (0.038)
DIVI-NonInfo	200	3.00 (0.00)	0.300 (0.170)	0.341 (0.189)	0.182 (0.000)
DIVI-Random	200	3.00 (0.00)	0.195 (0.206)	0.220 (0.217)	0.181 (0.044)
K-means (oracle $K$ )	1000	3.00 (0.00)	0.381 (0.037)	0.407 (0.038)	–

Appendix

Table B.7 : Full robustness results under correlated noise. All DIVI variants were evaluated with the sensitivity-selected split schedule

(\tau_{\mathrm{mult}},T_{\mathrm{split}})=(1.00,120)

. Values are mean (standard deviation) over 20 independently generated datasets.

Clustering

Clustering groups data points by shared structure, developing methods to discover, validate, and interpret groups across diverse data and constraints.

See all 345 papers

Oct 9, 2026

Model-free Bayesian Clustering and Multimodal Posterior Summarisation

We develop a model-free, noise-aware Bayesian clustering framework that constructs a martingale posterior over clusterings from a predictive Bayesian perspective. This avoids explicit likelihood-prior model specification, expensive MCMC sampling, and mode-seeking variational approximations, while accommodating irregular cluster shapes and enabling parallel generation of posterior draws. To interpret the posterior over high-dimensional, noise-aware clusterings, we introduce a Wasserstein-type summarization method. Using a novel noise-aware loss, this approach compresses the large collection of posterior draws into a small set of weighted representatives, capturing distinct cluster structures and noise assignments. Further, we marginalise the uncertainty in the density level, avoiding reliance on a specific choice. We establish theoretical properties of the proposed loss and guarantees for the level-marginal posterior approximation. Results demonstrate the framework's effectiveness in capturing clustering and noise uncertainty.

Qing Li, Sara Wade

School of Mathematics University of Edinburgh

Clustering Unsupervised Clustering Bayesian Inference

Ask PDF

Aug 19, 2026

Diffusion Models for High-Dimensional Clustered Data: Intrinsic-Dimension Adaptivity via Bayesian Classification

The empirical success of diffusion models in generative modelling has motivated theoretical work, including quantitative error bounds and qualitative analyses that characterise the different phases of denoising. We bring these two areas together by studying the adaptivity of diffusion models to the structured geometry of multimodal high-dimensional data that consists of multiple clusters in

\mathbb{R}^D

, each with its own low-dimensional structure, and inter-cluster separation depending on

D

. We employ

K

-mixture Gaussian distributions as a canonical framework to capture this geometry and establish two theoretical results. First, we interpret denoising as a dynamical Bayesian classifier: the mixture score is a posterior-weighted average of cluster-wise scores, and we show that, with high probability, the posterior class probabilities concentrate on a single cluster once the signal-to-noise ratio reaches the scale

Θ(\log (KD)/D)

. Second, by separately analysing the denoising process in its mixing and cluster-commitment phases, we prove that the KL error bound depends linearly on the maximum intrinsic dimension of a cluster, up to a logarithmic factor, even when

K

grows polynomially with

D

. This improves on ambient-dimensional bounds and extends existing low-dimensional adaptivity analyses to multimodal distributions with heterogeneous, approximately low-rank covariances.

Yuga Iguchi, Paul Fearnhead

School of Mathematical Sciences, Lancaster University, Lancaster, UK

Diffusion Models Intrinsic Dimensionality

Ask PDF

Apr 20, 2026

Clusterability-Based Assessment of Potentially Noisy Views for Multi-View Clustering

In multi-view clustering, the quality of different views may vary substantially, and low-quality or degraded views can impair overall clustering performance. However, existing studies mainly address this issue within the clustering process through view weighting or noise-robust optimization, while paying limited attention to data-level assessment before clustering. In this paper, we study the problem of pre-clustering noisy-view analysis in multi-view data from a clusterability perspective. To this end, we propose a Multi-View Clusterability Score (MVCS), which quantifies the strength of latent cluster-related structures in multi-view data through three complementary components: per-view structural clusterability, joint-space clusterability, and cross-view neighborhood consistency. To the best of our knowledge, this is the first clusterability score specifically designed for multi-view data. We further use it to perform potentially noisy view analysis and noisy-view detection before clustering. Extensive experiments on real-world datasets demonstrate that noisy views can significantly degrade clustering performance, and that, compared with existing clusterability measures designed for single-view data, the proposed method more effectively supports noisy-view analysis and detection.

Mudi Jiang, Jiahui Zhou, Xinying Liu +2

School of Software, Dalian University of Technology, Dalian, China · School of Software, Dalian University of Technology, Dalian, China, and Key Laboratory for Ubiquitous Network and Service Software of Liaoning Province, Dalian, China

Clustering Multi-View Learning Multi-View Clustering

Ask PDF

Abstract

Clustering

Explore similar work

Model-free Bayesian Clustering and Multimodal Posterior Summarisation

Diffusion Models for High-Dimensional Clustered Data: Intrinsic-Dimension Adaptivity via Bayesian Classification

Clusterability-Based Assessment of Potentially Noisy Views for Multi-View Clustering