A Data-Informed Variational Clustering Framework for Noisy High-Dimensional Data

Wan Ping Chen

stat.MLApr 8, 2026

A Data-Informed Variational Clustering Framework for Noisy High-Dimensional Data

Authors: Wan Ping Chen

Organizations: Department of Mathematics, Fu Jen Catholic University, Taipei, Taiwan, No. 510 Zhongzheng Rd., Xinzhuang Dist., New Taipei City, 242062, Taiwan

Abstract

Clustering in high-dimensional settings with severe feature noise remains challenging, especially when only a small subset of dimensions is informative and the final number of clusters is not specified in advance. In such regimes, partition recovery, feature relevance learning, and structural adaptation are tightly coupled, and standard likelihood-based methods can become unstable or overly sensitive to noisy dimensions. We propose DIVI, a data-informed variational clustering framework that combines global feature gating with split-based adaptive structure growth. DIVI uses informative prior initialization to stabilize optimization, learns feature relevance in a differentiable manner, and expands model complexity only when local diagnostics indicate underfit. Beyond clustering performance, we also examine runtime scalability and parameter sensitivity in order to clarify the computational and practical behavior of the framework. Empirically, we find that DIVI performs competitively under severe feature noise, remains computationally feasible, and yields interpretable feature-gating behavior, while also exhibiting conservative growth and identifiable failure regimes in challenging settings. Overall, DIVI is best viewed as a practical variational clustering framework for noisy high-dimensional data rather than as a fully Bayesian generative solution.

Figures & tables

	$N=200$ (Small- $n$ )				$N=1000$ (Large- $n$ )
Method	$K$	ARI	NMI	F1	$K$	ARI	NMI	F1
K-means	3.000	.909	.885	–	3.000	.992	.985	–
	(.000)	(.048)	(.053)	–	(.000)	(.006)	(.010)	–
GMM	3.000	.992	.987	–	3.000	.995	.991	–
	(.000)	(.010)	(.016)	–	(.000)	(.004)	(.007)	–
SPKM	3.000	.993	.989	.184	3.000	.996	.993	.190

Table 1: Clustering performance under high-dimensional noise. Results are averaged over 20 independently generated synthetic datasets with

D=100

and 90% irrelevant dimensions. DIVI is evaluated using the current implementation with the sensitivity-selected split interval

T_{\mathrm{split}}=120

. Feature F1 is reported only for methods with explicit feature selection or feature gating.

Scenario	$N$	K-means ARI	GMM ARI	DIVI-Info ARI	DIVI-Info F1
Heavy-tailed signal	200	0.913 (0.034)	0.853 (0.238)	0.989 (0.015)	0.768 (0.095)
Heavy-tailed signal	1000	0.986 (0.005)	0.971 (0.097)	0.964 (0.107)	0.990 (0.020)
Correlated noise	200	0.283 (0.080)	0.307 (0.176)	0.419 (0.153)	0.243 (0.038)
Correlated noise	1000	0.381 (0.037)	0.497 (0.007)	0.712 (0.261)	0.254 (0.031)

Table 2: Main robustness results under model misspecification. DIVI-Info was evaluated with the sensitivity-selected split schedule

(\tau_{\mathrm{mult}},T_{\mathrm{split}})=(1.00,120)

. Values are mean (std) over 20 independently generated datasets.

Figure 1: Interpretability illustration on the UCI Wine dataset. The observations are visualized in the two most relevant dimensions identified by the learned feature-gating mechanism. Although this experiment is small in scale, it shows that the learned relevance profile can produce a meaningful low-dimensional view that aligns reasonably well with known chemical differences among wine categories.

Dataset	Method	ARI	NMI	Time (s)	Final $K$	Active dims
ISOLET	DIVI-Info	0.460 (0.143)	0.585 (0.122)	22.26 (1.14)	4.0 (0.0)	578.0 (1.7)
ISOLET	K-means	0.632 (0.001)	0.716 (0.001)	0.59 (0.04)	5.0 (0.0)	–
ISOLET	GMM	0.580 (0.112)	0.684 (0.068)	2.81 (0.75)	5.0 (0.0)	–
ISOLET	SPKM	0.551 (0.000)	0.640 (0.000)	9.26 (0.66)	5.0 (0.0)	23.0 (0.0)
20NG	DIVI-Info	0.759 (0.133)	0.753 (0.074)	16.49 (1.03)	4.0 (0.0)	351.6 (1.1)
20NG	K-means	0.845 (0.002)	0.803 (0.002)	0.40 (0.02)	4.0 (0.0)	–

Table 3: Real-data clustering and computational summaries on ISOLET and 20NG embeddings, where 20NG denotes the 20 Newsgroups subset. Results are reported as mean (standard deviation) over repeated runs. For K-means, GMM, and SPKM, the true number of clusters is supplied. DIVI starts from a single component and expands via split diagnostics. Active dimensions are reported for feature-selective methods only (DIVI and SPKM).

Figure 2: Runtime scaling with feature dimension

D

on synthetic data under fixed sample size (

N=1000

). DIVI-Info exhibits a monotone increase in wall-clock cost as dimensionality grows, reflecting the increasing cost of feature-wise variational optimization.

Figure 3: Runtime scaling with sample size

N

on synthetic data under fixed dimensionality (

D=1000

). Runtime increases with sample size for all methods, with DIVI-Info incurring the highest computational cost due to joint feature gating and split-based structure adaptation.

Figure 4: Sensitivity of DIVI to the split interval

T_{\mathrm{split}}

on fixed synthetic datasets, measured by ARI. Very small split intervals lead to substantially worse clustering performance, whereas more conservative split intervals yield markedly better ARI across both sample-size regimes.

Figure 5: Sensitivity of DIVI to the KL-scaling multiplier

\beta/N

on fixed synthetic datasets, measured by Feature F1. Larger

\beta/N

improves or stabilizes informative-support recovery, with the gains being most pronounced in the small-

n

setting. The results support

\beta=N

as a practical default for feature-gating regularization.

Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

Panel A: Large- $n$ ( $N=1000$ , $D=100$ )
$T_{\mathrm{split}}$	ARI	NMI	F1	Time (s)	Final $K$	Splits
10	0.414 (0.096)	0.618 (0.048)	1.000 (0.000)	15.21 (2.79)	31.0 (0.0)	30.0 (0.0)
20	0.493 (0.105)	0.665 (0.045)	1.000 (0.000)	6.77 (0.23)	16.0 (0.0)	15.0 (0.0)
40	0.678 (0.103)	0.772 (0.039)	1.000 (0.000)	3.68 (0.25)	8.0 (0.0)	7.0 (0.0)
80	0.879 (0.012)	0.909 (0.006)	1.000 (0.000)	2.43 (0.31)	4.0 (0.0)	3.0 (0.0)
Panel B: Small- $n$ ( $N=200$ , $D=100$ )

Appendix

Table B.1 : Sensitivity of DIVI to the split interval

T_{\mathrm{split}}

on fixed synthetic datasets. Results are mean (standard deviation) over repeated runs.

Panel A: Large- $n$ ( $N=1000$ , $D=100$ )
$\beta/N$	ARI	NMI	F1	Time (s)	Final $K$	Active dims
0.25	0.876 (0.008)	0.908 (0.004)	0.519 (0.017)	2.46 (0.29)	4.0 (0.0)	28.6 (1.3)
0.50	0.877 (0.010)	0.909 (0.005)	0.828 (0.031)	2.59 (0.27)	4.0 (0.0)	14.2 (0.9)
1.00	0.879 (0.012)	0.909 (0.006)	1.000 (0.000)	2.53 (0.20)	4.0 (0.0)	10.0 (0.0)
2.00	0.880 (0.012)	0.910 (0.006)	1.000 (0.000)	2.51 (0.24)	4.0 (0.0)	10.0 (0.0)
4.00	0.880 (0.012)	0.910 (0.006)	1.000 (0.000)	2.53 (0.23)	4.0 (0.0)	10.0 (0.0)

Appendix

Table B.2 : Sensitivity of DIVI to the KL scaling factor

\beta

on fixed synthetic datasets. Results are mean (standard deviation) over repeated runs.

Panel A: Large- $n$ ( $N=1000$ , $D=100$ )
$\tau/\tau_{0}$	ARI	NMI	F1	Time (s)	Final $K$	Active dims
0.9	0.879 (0.012)	0.909 (0.006)	1.000 (0.000)	2.57 (0.20)	4.0 (0.0)	10.0 (0.0)
1.0	0.879 (0.012)	0.909 (0.006)	1.000 (0.000)	2.54 (0.14)	4.0 (0.0)	10.0 (0.0)
1.1	0.879 (0.012)	0.909 (0.006)	1.000 (0.000)	2.74 (0.23)	4.0 (0.0)	10.0 (0.0)
1.2	0.879 (0.012)	0.909 (0.006)	1.000 (0.000)	2.67 (0.18)	4.0 (0.0)	10.0 (0.0)
Panel B: Small- $n$ ( $N=200$ , $D=100$ )

Appendix

Table B.3 : Sensitivity of DIVI to the entropy-threshold multiplier

\tau/\tau_{0}

on fixed synthetic datasets. Results are mean (standard deviation) over repeated runs.

Panel A: Large- $n$ ( $N=1000$ , $D=100$ )
Learning rate	ARI	NMI	F1	Time (s)	Final $K$	Active dims
0.005	0.901 (0.027)	0.915 (0.021)	1.000 (0.000)	2.34 (0.17)	4.0 (0.0)	10.0 (0.0)
0.010	0.879 (0.012)	0.909 (0.006)	1.000 (0.000)	2.58 (0.26)	4.0 (0.0)	10.0 (0.0)
0.020	0.876 (0.008)	0.908 (0.004)	0.995 (0.015)	2.80 (0.33)	4.0 (0.0)	10.1 (0.3)
0.050	0.877 (0.009)	0.909 (0.004)	0.990 (0.020)	3.00 (0.21)	4.0 (0.0)	10.2 (0.4)
Panel B: Small- $n$ ( $N=200$ , $D=100$ )

Appendix

Table B.4 : Sensitivity of DIVI to the optimizer learning rate on fixed synthetic datasets. Results are mean (standard deviation) over repeated runs.

Panel A: Large- $n$ ( $N=1000$ , $D=100$ )
$T_{\min}$	ARI	NMI	F1	Time (s)	Final $K$	Active dims
0.01	0.879 (0.012)	0.910 (0.006)	1.000 (0.000)	2.80 (0.21)	4.0 (0.0)	10.0 (0.0)
0.05	0.879 (0.012)	0.910 (0.006)	1.000 (0.000)	2.61 (0.23)	4.0 (0.0)	10.0 (0.0)
0.10	0.879 (0.012)	0.909 (0.006)	1.000 (0.000)	2.55 (0.21)	4.0 (0.0)	10.0 (0.0)
0.20	0.880 (0.013)	0.910 (0.007)	1.000 (0.000)	2.52 (0.14)	4.0 (0.0)	10.0 (0.0)
Panel B: Small- $n$ ( $N=200$ , $D=100$ )

Appendix

Table B.5 : Sensitivity of DIVI to the temperature annealing endpoint

T_{\min}

on fixed synthetic datasets. Results are mean (standard deviation) over repeated runs.

Method	$N$	Final $K$	ARI	NMI	F1
K-means (oracle $K$ )	200	3.00 (0.00)	0.913 (0.034)	0.886 (0.036)	–
Std. GMM (oracle $K$ )	200	3.00 (0.00)	0.853 (0.238)	0.875 (0.187)	–
DIVI-Info	200	3.00 (0.00)	0.989 (0.015)	0.983 (0.022)	0.768 (0.095)
DIVI-NonInfo	200	3.00 (0.00)	0.906 (0.173)	0.910 (0.124)	0.182 (0.000)
DIVI-Random	200	3.00 (0.00)	0.814 (0.190)	0.819 (0.161)	0.187 (0.041)
K-means (oracle $K$ )	1000	3.00 (0.00)	0.986 (0.005)	0.974 (0.009)	–

Appendix

Table B.6 : Full robustness results under heavy-tailed signal. All DIVI variants were evaluated with the sensitivity-selected split schedule

(\tau_{\mathrm{mult}},T_{\mathrm{split}})=(1.00,120)

. Values are mean (standard deviation) over 20 independently generated datasets.

Method	$N$	Final $K$	ARI	NMI	F1
K-means (oracle $K$ )	200	3.00 (0.00)	0.283 (0.080)	0.307 (0.084)	–
Std. GMM (oracle $K$ )	200	3.00 (0.00)	0.307 (0.176)	0.349 (0.194)	–
DIVI-Info	200	3.00 (0.00)	0.419 (0.153)	0.466 (0.154)	0.243 (0.038)
DIVI-NonInfo	200	3.00 (0.00)	0.300 (0.170)	0.341 (0.189)	0.182 (0.000)
DIVI-Random	200	3.00 (0.00)	0.195 (0.206)	0.220 (0.217)	0.181 (0.044)
K-means (oracle $K$ )	1000	3.00 (0.00)	0.381 (0.037)	0.407 (0.038)	–

Appendix

Table B.7 : Full robustness results under correlated noise. All DIVI variants were evaluated with the sensitivity-selected split schedule

(\tau_{\mathrm{mult}},T_{\mathrm{split}})=(1.00,120)

. Values are mean (standard deviation) over 20 independently generated datasets.

Clustering

Clustering groups data points by shared structure, developing methods to discover, validate, and interpret groups across diverse data and constraints.

See all 345 papers

Model-free Bayesian Clustering and Multimodal Posterior Summarisation
Oct 9, 2026Qing Li, Sara WadeClustering Unsupervised Clustering
School of Mathematics University of Edinburgh
Diffusion Models for High-Dimensional Clustered Data: Intrinsic-Dimension Adaptivity via Bayesian Classification
Aug 19, 2026Yuga Iguchi, Paul FearnheadDiffusion Models Intrinsic Dimensionality
School of Mathematical Sciences, Lancaster University, Lancaster, UK
Clusterability-Based Assessment of Potentially Noisy Views for Multi-View Clustering
Apr 20, 2026Mudi Jiang, Jiahui Zhou, Xinying Liu +2Clustering Multi-View Learning
School of Software, Dalian University of Technology, Dalian, China · School of Software, Dalian University of Technology, Dalian, China, and Key Laboratory for Ubiquitous Network and Service Software of Liaoning Province, Dalian, China

Oct 9, 2026J/K move · Enter open · S save

Model-free Bayesian Clustering and Multimodal Posterior Summarisation

Qing Li, Sara Wade

School of Mathematics University of Edinburgh

Clustering Unsupervised Clustering Bayesian Inference

Open paper AskPDF arXiv

We develop a model-free, noise-aware Bayesian clustering framework that constructs a martingale posterior over clusterings from a predictive Bayesian perspective. This avoids explicit likelihood-prior model specification, expensive MCMC sampling, and mode-seeking variational approximations, while accommodating irregular cluster shapes and enabling parallel generation of posterior draws. To interpret the posterior over high-dimensional, noise-aware clusterings, we introduce a Wasserstein-type summarization method. Using a novel noise-aware loss, this approach compresses the large collection of posterior draws into a small set of weighted representatives, capturing distinct cluster structures and noise assignments. Further, we marginalise the uncertainty in the density level, avoiding reliance on a specific choice. We establish theoretical properties of the proposed loss and guarantees for the level-marginal posterior approximation. Results demonstrate the framework's effectiveness in capturing clustering and noise uncertainty.

Abstract

Clustering

Explore similar work

Model-free Bayesian Clustering and Multimodal Posterior Summarisation

Diffusion Models for High-Dimensional Clustered Data: Intrinsic-Dimension Adaptivity via Bayesian Classification

Clusterability-Based Assessment of Potentially Noisy Views for Multi-View Clustering