AdaMuS: Adaptive Multi-view Sparsity Learning for Dimensionally Unbalanced Data

Cai Xu; Changhao Sun; Ziyu Guan; Wei Zhao

cs.LGMar 18, 2026

AdaMuS: Adaptive Multi-view Sparsity Learning for Dimensionally Unbalanced Data

Authors: Cai Xu, Changhao Sun, Ziyu Guan, Wei Zhao

Organizations: Xidian University, Xi’an 710071, China. · the School of Computer Science and Technology · the School of Cyber Engineering.

Abstract

Multi-view learning primarily aims to fuse multiple features to describe data comprehensively. Most prior studies implicitly assume that different views share similar dimensions. In practice, however, severe dimensional disparities often exist among different views, leading to the unbalanced multi-view learning issue. For example, in emotion recognition tasks, video frames often reach dimensions of $10^6$ , while physiological signals comprise only $10^1$ dimensions. Existing methods typically face two main challenges for this problem: (1) They often bias towards high-dimensional data, overlooking the low-dimensional views. (2) They struggle to effectively align representations under extreme dimensional imbalance, which introduces severe redundancy into the low-dimensional ones. To address these issues, we propose the Adaptive Multi-view Sparsity Learning (AdaMuS) framework. First, to prevent ignoring the information of low-dimensional views, we construct view-specific encoders to map them into a unified dimensional space. Given that mapping low-dimensional data to a high-dimensional space often causes severe overfitting, we design a parameter-free pruning method to adaptively remove redundant parameters in the encoders. Furthermore, we propose a sparse fusion paradigm that flexibly suppresses redundant dimensions and effectively aligns each view. Additionally, to learn representations with stronger generalization, we propose a self-supervised learning paradigm that obtains supervision information by constructing similarity graphs. Extensive evaluations on a synthetic toy dataset and seven real-world benchmarks demonstrate that AdaMuS consistently achieves superior performance and exhibits strong generalization across both classification and semantic segmentation tasks.

Figures & tables

Fig. 1 : Ubiquitous Dimensional Imbalance in Real-world Multi-view Applications. We illustrate four representative scenarios—(a) Medical Diagnosis, (b) Financial Analysis, (c) Emotion Recognition, and (d) Monitoring—where extreme dimensionality disparities (e.g., $\approx 10^{6}$ vs. $\approx 10^{1}$ ) between views are prevalent. This fundamental imbalance challenges conventional fusion methods, which often bias towards high-dimensional features.

Fig. 2 : Illustration of the proposed AdaMuS framework. AdaMuS establishes view-specific encoders $f^{(v)}(\cdot)$ to process unbalanced multi-view data. These encoders are structurally optimized by the Principal Neuron Analysis (PNA) module to learn aligned representations $\{z_{n}^{v}\}_{v=1}^{V}$ with a unified dimension. Specifically, a Multi-view Sparse Batch Normalization (MSBN) layer is proposed to explicitly integrate these features via a sparse fusion paradigm. Finally, we train AdaMuS in a self-supervised manner to guide the representation learning.

Fig. 3 : Learning curves of a DNN classifier on the low-dimensional view of the CUB dataset. The model projects the 30-dimensional input into a high-dimensional space (structure: 30–1024–512–10). The increasing gap between training and validation performance demonstrates that expanding low-dimensional data with deep networks leads to overfitting.

Fig. 4 : PNA workflow: (1) Compute covariance matrix from layer outputs; (2) Extract eigenvalue distribution; (3) Measure dissimilarity to Uniform/Dirac baselines; (4) Adjust pruning rate with view imbalance.

Fig. 5 : Construction process of the multi-view toy example.

Panel of Figure 6 — Fig. 6 : t-SNE visualization of the learned representations on the toy example.

Fig. 7 : Impact of Dimensional Compression. Comparison between appropriate compression (

3000\to 1024

) and over-compression (

3000\to 128

). The gray gap highlights the significant information loss caused by aggressive compression, verifying the need for a high-dimensional subspace.

Panel of Figure 8 — Fig. 8 : Validation of the PNA pruning module. We compare the learning dynamics of (a) PNA-enhanced DNN vs. (b) Standard DNN during dimensional expansion ( $10\to 512\to 1024$ ). The Standard DNN suffers from overfitting, whereas PNA ensures stable convergence by pruning redundant connections.

Fig. 9 : Verification of sparse alignment. (Left) Ground-truth mapping matrix

W_{2}^{T}

, where red dashed boxes indicate irrelevant dimensions. (Right) The learned scaling factor

\bm{\gamma}

effectively suppresses these dimensions while activating the informative ones.

Dataset	Size	Category	Dimensions	$\Lambda$
UCI	1600	10	$6/47/240$	2.88
CUB	480	10	$30/1024$	3.22
ORL	400	40	$30/3304/6750$	2.54
MSRCV1	210	7	$20/100/210/256/512/1302$	2.41
Mfeat	2000	10	$240/216/76/64/47/6$	2.22
100leaves	1600	100	$5/64/64$	2.14

TABLE I : DATASET STATISTICS.

\Lambda

INDICATES THE UNBALANCED DEGREE. NYUV2 IS EMPLOYED FOR SEGMENTATION.

Datasets	Metric	Methods
Datasets	Metric	Concat	DCCAE	CGCN	BMGC	ICMVC	CANDY	GUMRL	PDMF	UIMC	SURER	STCMC-UR	UMDL	AdaMuS
UCI	ACC	71.56±3.98	74.62±1.19	70.00±1.96	73.45±1.68	85.70±1.18	69.18±2.05	86.09±0.08	85.20±1.09	76.88±2.01	77.90±1.12	72.24±1.39	87.51±0.32	91.49±1.08
	NMI	72.39±2.11	79.77±1.11	78.01±0.87	73.33±1.57	83.13±1.05	67.23±2.14	78.66±0.91	75.84±1.44	67.15±2.33	75.57±1.76	73.27±0.60	79.08±1.02	86.93±0.97
	ARI	62.28±3.15	71.97±0.78	57.71±2.96	63.64±2.83	75.14±1.31	57.61±1.75	69.63±0.26	71.51±1.06	60.44±3.11	67.06±2.17	62.46±1.10	74.23±0.14	82.85±0.90
CUB	ACC	64.23±2.91	69.71±1.83	57.08±2.42	68.96±1.95	62.53±2.40	51.48±2.90	61.73±2.16	70.47±3.71	70.11±1.17	69.60±1.02	55.22±2.55	80.12±1.64	81.65±1.74
	NMI	60.85±1.69	67.34±2.54	69.29±1.34	57.46±2.85	54.07±1.76	64.75±2.31	53.05±1.93	66.83±1.71	65.82±1.07	72.29±0.99	63.55±2.63	73.43±2.74	77.21±0.45

TABLE II : Clustering performance comparison on different datasets (mean result

\pm

standard deviation). AdaMuS is our proposed method. The best and the second best results are highlighted by boldface and underlined , respectively.

Datasets	Metric	Methods
Datasets	Metric	PDMF	GUMRL	SURER	RCML	UMDL	AdaMuS (Ours)
UCI	ACC	94.43 $\pm$ 0.15	96.88 $\pm$ 0.09	96.00 $\pm$ 0.25	96.67 $\pm$ 0.72	97.25 $\pm$ 0.17	98.50 $\pm$ 0.03
UCI	F1	96.03 $\pm$ 0.19	96.78 $\pm$ 0.10	95.87 $\pm$ 0.32	96.63 $\pm$ 0.74	97.00 $\pm$ 0.17	98.50 $\pm$ 0.03
CUB	ACC	87.29 $\pm$ 0.69	85.42 $\pm$ 1.16	89.17 $\pm$ 0.24	88.75 $\pm$ 2.64	87.99 $\pm$ 0.40	90.00 $\pm$ 0.31
CUB	F1	87.17 $\pm$ 0.58	85.50 $\pm$ 0.84	89.09 $\pm$ 0.33	88.20 $\pm$ 2.26	87.56 $\pm$ 0.50	89.57 $\pm$ 0.17
ORL	ACC	69.50 $\pm$ 4.03	92.12 $\pm$ 0.27	94.00 $\pm$ 0.21	96.62 $\pm$ 0.72	92.25 $\pm$ 0.43	97.25 $\pm$ 0.02

TABLE III : Classification comparison across benchmarks (Mean

\pm

Std). Bold and underlined values denote the highest and second-highest scores, respectively.

Fig. 10 : Efficiency comparison between UMDL and AdaMuS. The chart displays the ratios of parameters and FLOPs required by AdaMuS relative to UMDL (blue and orange bars, respectively).

Panel of Figure 14 — Fig. 11 : t-SNE visualization of original data and comprehensive representations on CUB dataset. (a) and (b) are original features, respectively. (c) is the result of AdaMuS-RS, which removes the MSBN layer. (d) is the result of AdaMuS.

Datasets	Method			Metrics
Datasets	$\mathcal{L}_{G}$	$\mathcal{L}_{S}$	$\mathcal{L}_{P}$	ACC (%)	NMI (%)	ARI (%)
MSRCV1	✓	-	-	87.67 $\pm$ 1.59	80.72 $\pm$ 1.84	76.24 $\pm$ 1.80
	✓	✓	-	88.71 $\pm$ 1.47	81.19 $\pm$ 1.33	76.66 $\pm$ 1.72
	✓	-	✓	92.86 $\pm$ 0.92	88.28 $\pm$ 1.05	83.21 $\pm$ 1.13
	✓	✓	✓	95.05 $\pm$ 0.77	93.07 $\pm$ 0.76	91.75 $\pm$ 0.60
ORL	✓	-	-	70.27 $\pm$ 2.34	84.21 $\pm$ 1.18	66.04 $\pm$ 1.83

TABLE IV : Ablation study.

\mathcal{L}_{G}

is the foundational graph-based contrastive learning module.

\mathcal{L}_{S}

denotes the sparsity fusion constraint, and

\mathcal{L}_{P}

represents the PNA pruning module. Best results are in bold .

Panel of Figure 16 — Fig. 12 : Segmentation results of our AdaMuS-SEG and baseline UNET on the NYUv2 dataset.

Panel of Figure 17 — Fig. 13 : Hyper-parameter analysis on the MSRCV1 dataset.

Fig. 14 : Convergence analysis on the MSRCV1 dataset.

Multi-View Learning

Multi-view learning combines complementary representations of the same objects to improve prediction, clustering, and robust data integration.

See all 185 papers

Learning Subspace-Preserving Sparse Attention Graphs from Heterogeneous Multiview Data
May 12, 2026Jie Chen, Yuanbiao Gou, Chuanbin Liu +2Graph Structure Learning Multi-View Learning
College of Computer Science Sichuan University Chengdu, China · School of Economics and Management China University of Petroleum (Beijing) Beijing, China · Law School Sichuan University Chengdu, China +1
Rethinking Multi-View Fusion for Urban Representation Learning
Oct 22, 2026Sean Bin Yang, Zongyi Xu, Jilin Hu +5Urban Representation Learning Multi-View Feature Fusion
Aalborg University, Denmark · Chongqing University of Posts and Telecommunications, China · East China Normal University, China
Learning Disentangled Representations for Generalized Multi-view Clustering
May 15, 2026Xin Zou, Ruimeng Liu, Chang Tang +4Disentangled Representation Learning Multi-View Learning
School of Computing and Information Technology, University of Wollongong, NSW, 2500, Australia · School of Software Engineering, Huazhong University of Science and Technology, Wuhan 430074, China

May 12, 2026J/K move · Enter open · S save

Learning Subspace-Preserving Sparse Attention Graphs from Heterogeneous Multiview Data

Jie Chen, Yuanbiao Gou, Chuanbin Liu, Zhu Wang+1

College of Computer Science Sichuan University Chengdu, China · School of Economics and Management China University of Petroleum (Beijing) Beijing, China · Law School Sichuan University Chengdu, China+1

Graph Structure Learning Multi-View Learning Transfer Learning Sparse Attention

Open paper AskPDF arXiv

The high-dimensional features extracted from large-scale unlabeled data via various pretrained models with diverse architectures are referred to as heterogeneous multiview data. Most existing unsupervised transfer learning methods fail to faithfully recover intrinsic subspace structures when exploiting complementary information across multiple views. Therefore, a fundamental challenge involves constructing sparse similarity graphs that preserve these underlying subspace structures for achieving semantic alignment across heterogeneous views. In this paper, we propose a sparse attention graph learning (SAGL) method that learns subspace-preserving sparse attention graphs from heterogeneous multiview data. Specifically, we introduce a bilinear attention factorization scheme to capture asymmetric similarities among the high-dimensional features, which breaks the symmetry bottleneck that is inherent in the traditional representation learning techniques. A dynamic sparsity gating mechanism then predicts a feature-specific compression factor for adaptively controlling the topological contributions of neighbors. Furthermore, we employ a structured sparse projection via

α

-entmax to generate subspace-preserving sparse attention graphs for individual views. SAGL leverages these view-specific graphs to conduct sparse information aggregation, yielding discriminative representations for multiview learning tasks. In addition, we provide a rigorous theoretical analysis that bridges differentiable sparse attention and probability simplex constraints. Extensive experiments conducted on multiple benchmark datasets demonstrate that SAGL consistently outperforms the state-of-the-art unsupervised transfer learning approaches.

Abstract

Multi-View Learning

Explore similar work

Learning Subspace-Preserving Sparse Attention Graphs from Heterogeneous Multiview Data

Rethinking Multi-View Fusion for Urban Representation Learning

Learning Disentangled Representations for Generalized Multi-view Clustering