Beyond Independence: Learning Correlated Views for Variational Incomplete Multi-View Clustering

Zheming Xu; Aiyue Tang; Shidi Chen; Xuechao Zou; Congyan Lang; Rogelio A. Mancisidor; Michael Kampffmeyer

cs.CVAug 16, 2026

Beyond Independence: Learning Correlated Views for Variational Incomplete Multi-View Clustering

Authors: Zheming Xu, Aiyue Tang, Shidi Chen, Xuechao Zou, Congyan Lang, Rogelio A. Mancisidor, Michael Kampffmeyer

Organizations: School of Computer Science & Technology, Beijing Jiaotong University · Key Laboratory of Big Data & Artificial Intelligence in Transportation, Ministry of Education, China · Department of Data Science, BI Norwegian Business School, Oslo, Norway · Department of Physics and Technology, UiT The Arctic University of Norway · Norwegian Computing Center

Abstract

Incomplete multi-view clustering (IMVC) aims to uncover shared cluster structures from data with partially observed views. Although recent imputation-free methods based on variational inference demonstrate robustness to missing views, they commonly rely on a conditional independence assumption across views in the posterior aggregation stage, which fails to capture the inherently structured and potentially correlated nature of multi-view data. In this paper, we propose a variational framework that explicitly goes beyond this assumption by introducing a learnable cross-view correlation structure. Specifically, we explicitly model and learn correlations between views by utilizing the covariance structure of posterior estimation errors during aggregation. To facilitate robust and efficient learning, the correlation matrix is parameterized through a normalized Cholesky decomposition, ensuring positive definiteness and enabling the entire model to be trained jointly through a unified variational objective. Extensive experiments on multiple IMVC benchmarks demonstrate that our method consistently outperforms state-of-the-art approaches across diverse missing-view settings while introducing only a negligible number of learnable parameters. These results highlight the effectiveness of adaptive correlation modeling in variational IMVC, demonstrating the need to go beyond the independence assumption in IMVC. The code is available at https://github.com/zmxu196/ACOVA.

Figures & tables

Panel of Figure 1 — Figure 1: Benefit of modeling inter-view dependencies for different missing rates on Scene15.

Figure 2: Overview of our proposed ACOVA framework. Given multi-view inputs $\{\mathbf{x}_{v}\}_{v=1}^{V}$ , view-specific encoders $\{\bm{\phi}_{v}\}$ produce posterior distribution parameters $\{\bm{\mu}_{v},\bm{\sigma}_{v}^{2}\}$ to characterize view-wise uncertainty. For each dimension $d\in[D]$ , the estimation errors $\epsilon_{v}^{d}$ between the view-specific posteriors and the true latent variable are modeled with learnable cross-view correlations, captured by the correlation matrix $\mathbf{R}$ . This structure, together with view-specific variances $\mathbf{D}$ , forms a structured covariance matrix $\mathbf{\Sigma}^{d}=\mathbf{DRD}$ , which governs the aggregation of posteriors into a consensus distribution with parameter values $\{\tilde{\mu}_{v}^{d},\tilde{\sigma}_{v}^{d\,2}\}$ . A latent representation $\mathbf{z}\in\mathbb{R}^{D}$ is sampled from this aggregated posterior and decoded via view-specific decoders $\{\bm{\theta}_{v}\}$ for reconstruction. Note that conditioning on $\bm{\mu}_{v}$ or on $\mathbf{x}_{v}$ is equivalent in this context.

	Missing Rate	10%				30%				50%				70%
	Metrics	ACC $\uparrow$	NMI $\uparrow$	ARI $\uparrow$	PUR $\uparrow$	ACC $\uparrow$	NMI $\uparrow$	ARI $\uparrow$	PUR $\uparrow$	ACC $\uparrow$	NMI $\uparrow$	ARI $\uparrow$	PUR $\uparrow$	ACC $\uparrow$	NMI $\uparrow$	ARI $\uparrow$	PUR $\uparrow$
Scene15	BSV	0.3595	0.3577	0.1784	0.4002	0.3305	0.3206	0.1253	0.3630	0.2944	0.2905	0.0796	0.3321	0.2593	0.2568	0.0472	0.2903
	Concat	0.3703	0.3820	0.2037	0.4263	0.3360	0.3318	0.1540	0.3825	0.2943	0.2995	0.1201	0.3436	0.2596	0.2665	0.0982	0.3053
	DCP	0.4071	0.4414	0.2488	0.4342	0.3958	0.4264	0.2350	0.4265	0.3879	0.4095	0.2359	0.4166	0.3784	0.3849	0.2113	0.4026
	CPSPAN	0.3574	0.3457	0.1911	0.4103	0.3161	0.2783	0.1541	0.3439	0.2553	0.2070	0.1063	0.2786	0.2025	0.1423	0.0621	0.2198
	DSIMVC	0.2795	0.2976	0.1430	0.3218	0.2734	0.2896	0.1383	0.3131	0.2697	0.2828	0.1335	0.3133	0.2654	0.2741	0.1290	0.3062

Table 1: Clustering performance under various missing-view rates on all datasets. Best results and second best are highlighted in bold and underlined , respectively. Note that all results are averaged over five runs.

	Variant	10%				30%				50%				70%
	Variant	ACC $\uparrow$	NMI $\uparrow$	ARI $\uparrow$	PUR $\uparrow$	ACC $\uparrow$	NMI $\uparrow$	ARI $\uparrow$	PUR $\uparrow$	ACC $\uparrow$	NMI $\uparrow$	ARI $\uparrow$	PUR $\uparrow$	ACC $\uparrow$	NMI $\uparrow$	ARI $\uparrow$	PUR $\uparrow$
Handwritten	Learn $\mathbf{R}$ ( w/ corr.)	0.9363	0.8692	0.8645	0.9363	0.9376	0.8709	0.8669	0.9376	0.9146	0.8489	0.8325	0.9146	0.8999	0.8351	0.8176	0.9021
	w/o corr.	0.9169	0.8604	0.8547	0.9201	0.8954	0.8470	0.8210	0.9020	0.8980	0.8373	0.8175	0.9054	0.7808	0.7905	0.7197	0.8099
	fixed $\rho=0.1$	0.9339	0.8676	0.8595	0.9339	0.9249	0.8534	0.8418	0.9249	0.8860	0.8288	0.8031	0.8898	0.8881	0.8231	0.7986	0.8918
	fixed $\rho=0.3$	0.8874	0.8513	0.8231	0.8945	0.9217	0.8509	0.8355	0.9217	0.8929	0.8335	0.8093	0.8972	0.8859	0.8219	0.7962	0.8903
	fixed $\rho=0.5$	0.9088	0.8590	0.8391	0.9118	0.9301	0.8612	0.8510	0.9301	0.8710	0.8291	0.7957	0.8821	0.8867	0.8239	0.7962	0.8908

Table 2: Ablation results on Handwritten and Scene15 datasets with different missing-view rates. ‘ w/ ’ and ‘ w/o ’ mean ‘ with ’ and ‘ without ’, respectively. ‘corr.’ refers to considering cross-view correlations. The best result in each column is denoted in bold . Note that all results are averaged over five runs.

Figure 3: Learned correlations under different noise and missing rates.

Panel of Figure 4 — Figure 4: t-SNE and learned correlation matrix visualization on the Handwritten dataset.

Panel of Figure 5 — Figure 5: Clustering performance with different parameter $\alpha$ and missing rate settings on Scene15 and Caltech5V dataset.

Appendix figures & tables15 assets

Supplementary material from the paper’s appendix.

Appendix

Dataset	#Views	#View Dimensions	#Instances	#Categories
Handwritten ( LeCun et al., 1989 )	6	76, 216, 64, 47, 240, 6	2,000	10
Caltech5V ( Fei-Fei et al., 2004 )	5	40, 254, 1984, 512, 928	1,400	7
Scene15 ( Fei-Fei & Perona, 2005 )	3	20, 59, 40	4,485	15
Fashion ( Xiao et al., 2017 )	3	784, 784, 784	10,000	10
NoisyMNIST ( Wang et al., 2015 )	2	784, 784	70,000	10
CUB Image-Captions ( Zhang et al., 2019 )	2	1024, 300	600	10

Appendix

Table 3: Detailed statistics for the six multi-view clustering datasets used in our experiments.

Method	Venue	#View	Description
BSV ( Zhao et al., 2016 )	IJCAI	Multiple	Imputes missing views using the view-wise average values and performs clustering using the best-performing single view.
CONCAT ( Zhao et al., 2016 )	IJCAI	Multiple	Similar to BSV that imputes missing views using the view-wise average values, but concatenates features across all views after imputation and then performs K -Means clustering.
DSIMVC ( Tang & Liu, 2022 )	ICML	Multiple	A bi-level optimization-based safe incomplete multi-view clustering framework that dynamically imputes missing views from semantic neighbors and automatically selects reliable imputed samples.
DCP ( Lin et al., 2023 )	PAMI	Two	An information-theoretical framework that unifies cross-view consistency learning and missing-view recovery via dual contrastive learning and dual prediction.
CPSPAN ( Jin et al., 2023 )	CVPR	Multiple	Jointly performs cross-view partial sample alignment and prototype-level alignment to address the challenges of prototype shift under missing-view settings.
DVIMC ( Xu et al., 2024 )	AAAI	Multiple	Employ PoE-based VAEs to construct a shared latent space and mitigate information imbalance via a coherence loss.

Appendix

Table 4: Statistics and brief summary of eight comparison methods.

Hyperparameters	Handwritten	Caltech5V	Scene15	Fashion	NoisyMNIST	CUB
Pretraining Epochs	200	200	200	200	200	200
Training Epochs	300	300	300	300	300	300
Batch Size	256	256	256	256	512	256
Optimizer	Adam	Adam	Adam	Adam	Adam	Adam
Scheduler	ExpLR	ExpLR	ExpLR	ExpLR	ExpLR	ExpLR
Network Learning Rate	$3e^{-4}$	$1e^{-3}$	$1e^{-3}$	$1e^{-3}$	$1e^{-3}$	$5e^{-4}$

Appendix

Table 5: Hyperparameter settings for all datasets. “ExpLR” is short for ExponentialLR.

Panel of Figure 6 — Figure 6: Performance curves vs. missing rates. The proposed approach consistently outperforms prior approaches even in the extreme scenarios.

	Variant	10%				30%
	Variant	ACC $\uparrow$	NMI $\uparrow$	ARI $\uparrow$	PUR $\uparrow$	ACC $\uparrow$	NMI $\uparrow$	ARI $\uparrow$	PUR $\uparrow$
Handwritten	Learn $\mathbf{R}$ ( w/ corr.)	0.9363 ± 0.0067	0.8692 ± 0.0113	0.8645 ± 0.0133	0.9363 ± 0.0067	0.9376 ± 0.0073	0.8709 ± 0.0125	0.8669 ± 0.0144	0.9376 ± 0.0073
	w/o corr.	0.9169 ± 0.0545	0.8604 ± 0.0268	0.8547 ± 0.0514	0.9201 ± 0.0481	0.8954 ± 0.0515	0.8470 ± 0.0336	0.8210 ± 0.0527	0.9020 ± 0.0409
	fixed $\rho=0.1$	0.9339 ± 0.0081	0.8676 ± 0.0097	0.8595 ± 0.0163	0.9339 ± 0.0081	0.9249 ± 0.0165	0.8534 ± 0.0235	0.8418 ± 0.0315	0.9249 ± 0.0165
	fixed $\rho=0.3$	0.8874 ± 0.0614	0.8513 ± 0.0229	0.8231 ± 0.0532	0.8945 ± 0.0527	0.9217 ± 0.0177	0.8509 ± 0.0248	0.8355 ± 0.0340	0.9217 ± 0.0177
	fixed $\rho=0.5$	0.9088 ± 0.0498	0.8590 ± 0.0246	0.8391 ± 0.0454	0.9118 ± 0.0439	0.9301 ± 0.0104	0.8612 ± 0.0168	0.8510 ± 0.0208	0.9301 ± 0.0104

Appendix

Table 7: Ablation results on Handwritten dataset with different missing-view rates. ‘ w/ ’ and ‘ w/o ’ mean ‘ with ’ and ‘ without ’, respectively. ‘corr.’ refers to considering cross-view correlations. The best result in each column is denoted in bold . Note that all results are averaged over five runs, with the mean and standard deviation reported.

Scene15	Variant	10%				30%
	Variant	ACC $\uparrow$	NMI $\uparrow$	ARI $\uparrow$	PUR $\uparrow$	ACC $\uparrow$	NMI $\uparrow$	ARI $\uparrow$	PUR $\uparrow$
	Learn $\mathbf{R}$ ( w/ corr.)	0.5040 $\,\pm$ 0.0153	0.4804 $\,\pm$ 0.0050	0.3273 $\,\pm$ 0.0088	0.5270 $\,\pm$ 0.0202	0.4807 $\,\pm$ 0.0136	0.4560 $\,\pm$ 0.0090	0.3106 $\,\pm$ 0.0079	0.5119 $\,\pm$ 0.0164
	w/o corr.	0.4863 $\,\pm$ 0.0103	0.4780 $\,\pm$ 0.0059	0.3192 $\,\pm$ 0.0085	0.5021 $\,\pm$ 0.0109	0.4678 $\,\pm$ 0.0054	0.4440 $\,\pm$ 0.0033	0.3097 $\,\pm$ 0.0033	0.4823 $\,\pm$ 0.0089
	fixed $\rho=0.1$	0.4963 $\,\pm$ 0.0142	0.4722 $\,\pm$ 0.0069	0.3237 $\,\pm$ 0.0105	0.5220 $\,\pm$ 0.0128	0.4702 $\,\pm$ 0.0077	0.4466 $\,\pm$ 0.0037	0.2977 $\,\pm$ 0.0041	0.4941 $\,\pm$ 0.0095
	fixed $\rho=0.3$	0.5057 $\,\pm$ 0.0062	0.4743 $\,\pm$ 0.0099	0.3272 $\,\pm$ 0.0060	0.5281 $\,\pm$ 0.0075	0.4746 $\,\pm$ 0.0202	0.4458 $\,\pm$ 0.0128	0.3011 $\,\pm$ 0.0136	0.4986 $\,\pm$ 0.0193
	fixed $\rho=0.5$	0.4963 $\,\pm$ 0.0190	0.4673 $\,\pm$ 0.0113	0.3214 $\,\pm$ 0.0099	0.5218 $\,\pm$ 0.0200	0.4689 $\,\pm$ 0.0175	0.4414 $\,\pm$ 0.0111	0.3005 $\,\pm$ 0.0129	0.4891 $\,\pm$ 0.0221

Appendix

Table 8: Ablation results on Scene15 dataset with different missing-view rates. ‘ w/ ’ and ‘ w/o ’ mean ‘ with ’ and ‘ without ’, respectively. ‘corr.’ refers to considering cross-view correlations. The best result in each column is denoted in bold . Note that all results are averaged over five runs, with the mean and standard deviation reported.

Panel of Figure 7 — Figure 7: Clustering performance with different latent dimension $D$ and missing rate settings on Scene15 and Caltech5V dataset.

	Methods
Dataset	Metric	BSV	CONCAT	DSIMVC	DCP	CPSPAN	DVIMC	MVP	PMIMC	ACOVA
Scene15	Time	-	-	$2.35\times 10^{-2}$	$1.47\times 10^{1}$	$6.62\times 10^{0}$	$1.12\times 10^{-1}$	$1.84\times 10^{0}$	$4.63\times 10^{0}$	$1.16\times 10^{-1}$
Scene15	ACC	0.2593	0.2596	0.2654	0.3784	0.2025	0.4136	0.4407	0.3178	0.4428
NoisyMNIST	Time	-	-	$3.06\times 10^{0}$	$1.48\times 10^{1}$	$2.22\times 10^{2}$	$4.98\times 10^{0}$	$7.61\times 10^{1}$	$1.14\times 10^{2}$	$5.59\times 10^{0}$
NoisyMNIST	ACC	0.2672	0.2805	0.6245	0.9225	0.5486	0.8741	0.6660	0.6044	0.9377

Appendix

Table 9: Runtime comparison on the Scene15 and NoisyMNIST datasets.

Panel of Figure 8 — Figure 8: Convergence curves on the Scene15 and Handwritten dataset.

Figure 9: Clustering accuracy (ACC) of different strategies on the Caltech5V dataset under varying missing rates. The proposed shared correlation matrix achieves consistently strong performance compared with other strategies.

Number of Views ( $V$ )	2	3	4	5
Runtime (seconds)	0.18 ± 0.03	0.22 ± 0.03	0.26 ± 0.04	0.27 ± 0.04

Appendix

Table 10: Runtime analysis with varying number of views on Caltech5V dataset.

Panel of Figure 10 — Figure 10: Convergence curves of $\|\mathbf{R}-\mathbf{I}\|_{F}$ on Caltech5V and Handwritten dataset.

Panel of Figure 11 — Figure 11: Correlation matrices on Handwritten: rows are missing rates, columns are epochs.

Panel of Figure 12 — Figure 12: Correlation matrices on Caltech5V: rows are missing rates, columns are epochs.

Panel of Figure 13 — Figure 13: Correlation matrices for the Handwritten dataset averaged across five runs with mean and standard deviation.

Explore similar work

Robust Multi-view Clustering against Imperfect Information
Jun 3, 2026Zhichao Huang, Haochen Zhou, Hao Wang +2Clustering Multi-View Learning
College of Computer Science, Sichuan University, China. · School of Artificial Intelligence, Sichuan University, China. · Tianfu Jincheng Laboratory, Chengdu, China.
Straight-Path Flow Matching for Incomplete Multi-View Clustering
Jul 7, 2026Yiteng Yuan, Junyan Wang, Zheyuan Liu +4Missing-Modality Learning Multi-View Consistency
School of Software Engineering, Huazhong University of Science and Technology, Wuhan, China · Australian Institute for Machine Learning, Adelaide University, Adelaide, Australia · University of Auckland, Auckland, New Zealand +2
SPORT: Structure-Aware Prototype Disentanglement for Incomplete Multi-View Clustering
Jul 11, 2026Yaoyuan Guo, Zhibin Gu, Songhe Feng +2Multi-View Consistency Disentangled Representation Learning
College of Computer and Cyberspace Security, Hebei Normal University, Shijiazhuang 050024, China · Department of Statistics and Data Science, Southern University of Science and Technology, Shenzhen 518055, China · Key Laboratory of Tibetan Information Processing, Ministry of Education, Qinghai Normal University, Xining 810008, China +2

Jun 3, 2026J/K move · Enter open · S save

Robust Multi-view Clustering against Imperfect Information

Zhichao Huang, Haochen Zhou, Hao Wang, Mouxing Yang+1

College of Computer Science, Sichuan University, China. · School of Artificial Intelligence, Sichuan University, China. · Tianfu Jincheng Laboratory, Chengdu, China.

Clustering Multi-View Learning Unsupervised Clustering Multi-View Clustering

Open paper AskPDF arXiv

Real-world multi-view data always suffer from imperfect information problem, where the view-specific observations are absent (i.e., Incomplete Views, IV) and cross-view correspondences are mismatched (i.e., Noisy Correspondences, NC) for certain instances. As a remedy, numerous IV- and NC-oriented multi-view clustering (MvC) methods have been proposed, which however require either reliable correspondences or sufficiently complete instances, thus stopping short of addressing the imperfect information problem. In contrast, we observe that both IV and NC challenges originate from the same issue of imperfect cross-view counterpart information, where the counterpart of an anchor instance in another view might be either unavailable or unreliable. Based on the observation, we propose a novel robust MvC framework, termed Posterior-guided Latent Counterpart Inference (PLCI), which could handle both IV and NC in a unified manner. Specifically, PLCI formulates the desired cross-view counterpart of each anchor instance as a latent variable, and integrates both instance-level reliability and prototype-level semantic transport to infer the posterior distribution of the latent counterpart. Extensive experiments on six widely-used multi-view datasets against 10 state-of-the-art MvC methods demonstrate the effectiveness of PLCI for tackling the imperfect information problem. The code will be released upon acceptance.

Abstract

Explore similar work

Robust Multi-view Clustering against Imperfect Information

Straight-Path Flow Matching for Incomplete Multi-View Clustering

SPORT: Structure-Aware Prototype Disentanglement for Incomplete Multi-View Clustering