SEHFS: Structural Entropy-Guided High-Order Correlation Learning for Multi-View Multi-Label Feature Selection

Cheng Peng; Yonghao Li; Wanfu Gao; Jie Wen; Weiping Ding

cs.LGMar 3, 2026

SEHFS: Structural Entropy-Guided High-Order Correlation Learning for Multi-View Multi-Label Feature Selection

Authors: Cheng Peng, Yonghao Li, Wanfu Gao, Jie Wen, Weiping Ding

Organizations: the College of Software, Jilin University, Changchun 130012, China · the Key Laboratory of Symbolic Computation and Knowledge Engineering of the Ministry of Education, Changchun 130012, China · the School of Computing and Artificial Intelligence, Southwestern University of Finance and Economics, Chengdu 611130, China · the Engineering Research Center of Intelligent Finance, Ministry of Education, Chengdu 611130, China · the College of Computer Science and Technology, Jilin University, Changchun 130012, China · the Shenzhen Key Laboratory of Visual Object Detection and Recognition, Harbin Institute of Technology, Shenzhen, Shenzhen 518055, China · the School of Artificial Intelligence and Computer Science, Nantong University, Nantong, 226019, China · the Faculty of Data Science, City University of Macau, Macau 999078, China

Abstract

In recent years, multi-view multi-label learning (MVML) has attracted extensive attention due to its close alignment to real-world scenarios. Information-theoretic methods have gained prominence for learning nonlinear correlations. However, two key challenges persist: first, features in real-world data commonly exhibit high-order structural correlations, but existing information-theoretic methods struggle to learn such correlations; second, commonly relying on heuristic optimization, information-theoretic methods are prone to converging to local optima. To address these two challenges, we propose a novel method called Structural Entropy Guided High-Order Correlation Learning for Multi-View Multi-Label Feature Selection (SEHFS). The core idea of SEHFS is to convert the feature graph into a structural-entropy-minimizing encoding tree, quantifying the information cost of high-order dependencies and thus learning high-order feature correlations beyond pairwise correlations. Specifically, features exhibiting strong high-order redundancy are grouped into a single cluster within the encoding tree, while inter-cluster feaeture correlations are minimized, thereby eliminating redundancy both within and across clusters. Furthermore, a new framework based on the fusion of information theory and matrix methods is adopted, which learns a shared semantic matrix and view-specific contribution matrices to reconstruct a global view matrix, thereby enhancing the information-theoretic method and balancing the global and local optimization. The ability of structural entropy to learn high-order correlations is theoretically established, and and both experiments on eight datasets from various domains and ablation studies demonstrate that SEHFS achieves superior performance in feature selection.

Figures & tables

Fig. 1 : Concept maps of two types of information-theoretic methods. Each ball denotes a feature, and greater color similarity indicates higher correlation. The yellow and red balls show high-order correlation. (a) is existing methods based on mutual information, which are limited to learning low-order correlations; (b) is SEHFS guided by structural entropy, which learns high-order correlations.

Fig. 2 : The framework of SEHFS. Given a multi-view multi-label dataset $[\mathbf{X}_{1},\mathbf{X}_{2},\dots,\mathbf{X}_{V}]$ , SEHFS first learns the semantic matrix $\mathbf{S}_{v}$ and view-specific matrix $\mathbf{H}_{v}$ from the $v$ -th view $\mathbf{X}_{v}$ . Then, semantic matrices from all views are integrated to form a shared semantic matrix $\mathbf{S}$ , whose graph structure is regularized by a graph Laplacian $\mathbf{L_{Y}}$ learned from the label matrix $\mathbf{Y}$ . Here, $\mathbf{S}$ captures cross-view consensus information, whereas $\mathbf{H}_{v}$ compensates for view-specific complementary information that is not fully preserved in the shared space. Their joint effect reconstructs the global view matrix $\mathbf{X^{f}}$ , from which a global feature graph $G^{f}$ is derived. Minimizing the structural entropy of $G^{f}$ yields an optimal encoding tree $\mathcal{T}$ that learns higher-order correlations and removes redundancy. Mapping $\mathbf{Y}$ onto $\mathcal{T}$ produces a more effective, low-redundancy feature selection.

Panel of Figure 3 — Fig. 3 : Structural-entropy–minimizing encoding tree in Scenario 1 and Scenario 2.

Fig. 4 : Subgraphs formed by nodes of the same color are highly densely connected, which is defined as “redundancy”. By minimizing structural entropy, the feature graph $G^{f}$ is converted into an optimal encoding tree $\mathcal{T}$ , which increases intra-layer edge weights while decreasing the weights from nodes to the root of their parent layer.

Datasets	EMOTIONS	YEAST	VOC07	MIRFlickr	SCENE	OBJECT	Corel5K	IAPRTC12
View-1( $d_{1}$ )	Rhythmic(8)	GE(79)	DH(100)	DH(100)	CH(64)	CH(64)	DH(100)	DH(100)
View-2( $d_{2}$ )	Timbre(64)	PP(24)	GIST(512)	GIST(512)	CM(225)	CM(225)	DHV3H1(300)	DHV3H1(300)
View-3( $d_{3}$ )	-	-	HH(100)	HH(100)	CORR(144)	CORR(144)	GIST(512)	GIST(512)
View-4( $d_{4}$ )	-	-	-	-	EDH(73)	EDH(73)	HHV3H1(300)	HHV3H1(300)
View-5( $d_{5}$ )	-	-	-	-	WT(128)	WT(128)	HH(100)	HH(100)
Instances( $n$ )	593	2417	3817	4053	4400	6047	4999	4999

TABLE I : The detailed information for the datasets in our experiment

Datasets	SEHFS	DHLI	GRAFS	MSFS	MIFS	SPLDG	SRFS	MSSL
EMOTIONS	.686 $\pm$ .052	.591 $\pm$ .045	.605 $\pm$ .036	.591 $\pm$ .050	.637 $\pm$ .049	.654 $\pm$ .049	.666 $\pm$ .052	.631 $\pm$ .048
YEAST	.670 $\pm$ .019	.663 $\pm$ .022	.665 $\pm$ .020	.651 $\pm$ .039	.663 $\pm$ .024	.663 $\pm$ .019	.672 $\pm$ .021	.660 $\pm$ .020
VOC07	.592 $\pm$ .019	.585 $\pm$ .017	.586 $\pm$ .020	.593 $\pm$ .016	.576 $\pm$ .021	.583 $\pm$ .015	.598 $\pm$ .016	.577 $\pm$ .016
MIRFlickr	.692 $\pm$ .022	.655 $\pm$ .016	.659 $\pm$ .025	.682 $\pm$ .019	.646 $\pm$ .019	.682 $\pm$ .015	.687 $\pm$ .019	.678 $\pm$ .020
SCENE	.804 $\pm$ .019	.786 $\pm$ .018	.801 $\pm$ .019	.718 $\pm$ .046	.792 $\pm$ .017	.788 $\pm$ .017	.801 $\pm$ .016	.797 $\pm$ .018
OBJECT	.454 $\pm$ .052	.408 $\pm$ .051	.437 $\pm$ .056	.357 $\pm$ .049	.436 $\pm$ .044	.423 $\pm$ .050	.435 $\pm$ .045	.433 $\pm$ .054

TABLE II : Experimental results

\uparrow

(mean

\pm

std) of each method on AP; the best result is shown in bold and the second-best is underlined.

Datasets	SEHFS	DHLI	GRAFS	MSFS	MIFS	SPLDG	SRFS	MSSL
EMOTIONS	.663 $\pm$ .029	.775 $\pm$ .035	.749 $\pm$ .029	.769 $\pm$ .033	.722 $\pm$ .031	.706 $\pm$ .033	.681 $\pm$ .026	.721 $\pm$ .033
YEAST	.679 $\pm$ .034	.690 $\pm$ .037	.682 $\pm$ .034	.729 $\pm$ .062	.679 $\pm$ .042	.688 $\pm$ .039	.675 $\pm$ .035	.689 $\pm$ .034
VOC07	.441 $\pm$ .005	.458 $\pm$ .005	.459 $\pm$ .006	.454 $\pm$ .004	.495 $\pm$ .006	.454 $\pm$ .004	.445 $\pm$ .004	.474 $\pm$ .005
MIRFlickr	.582 $\pm$ .007	.635 $\pm$ .007	.617 $\pm$ .011	.592 $\pm$ .006	.640 $\pm$ .007	.585 $\pm$ .005	.587 $\pm$ .006	.592 $\pm$ .006
SCENE	.430 $\pm$ .009	.456 $\pm$ .008	.439 $\pm$ .008	.499 $\pm$ .010	.448 $\pm$ .008	.457 $\pm$ .009	.435 $\pm$ .008	.446 $\pm$ .009
OBJECT	.311 $\pm$ .010	.338 $\pm$ .008	.319 $\pm$ .009	.375 $\pm$ .008	.320 $\pm$ .009	.331 $\pm$ .008	.323 $\pm$ .007	.334 $\pm$ .009

TABLE III : Experimental results

\downarrow

(mean

\pm

std) of each method on Cov; the best result is shown in bold and the second-best is underlined.

Datasets	SEHFS	DHLI	GRAFS	MSFS	MIFS	SPLDG	SRFS	MSSL
EMOTIONS	.243 $\pm$ .031	.297 $\pm$ .032	.297 $\pm$ .031	.332 $\pm$ .059	.275 $\pm$ .031	.271 $\pm$ .032	.255 $\pm$ .028	.274 $\pm$ .039
YEAST	.223 $\pm$ .007	.225 $\pm$ .009	.230 $\pm$ .007	.254 $\pm$ .013	.224 $\pm$ .009	.230 $\pm$ .008	.223 $\pm$ .009	.230 $\pm$ .008
VOC07	.079 $\pm$ .004	.085 $\pm$ .004	.085 $\pm$ .005	.087 $\pm$ .005	.085 $\pm$ .004	.084 $\pm$ .004	.084 $\pm$ .004	.086 $\pm$ .004
MIRFlickr	.167 $\pm$ .010	.189 $\pm$ .007	.184 $\pm$ .010	.175 $\pm$ .009	.188 $\pm$ .007	.173 $\pm$ .007	.172 $\pm$ .009	.176 $\pm$ .009
SCENE	.092 $\pm$ .007	.103 $\pm$ .008	.098 $\pm$ .007	.136 $\pm$ .019	.102 $\pm$ .006	.100 $\pm$ .006	.097 $\pm$ .007	.099 $\pm$ .006
OBJECT	.052 $\pm$ .005	.059 $\pm$ .004	.057 $\pm$ .004	.069 $\pm$ .004	.060 $\pm$ .005	.057 $\pm$ .004	.056 $\pm$ .004	.057 $\pm$ .004

TABLE IV : Experimental results

\downarrow

(mean

\pm

std) of each method on HL; the best result is shown in bold and the second-best is underlined.

Datasets	SEHFS	DHLI	GRAFS	MSFS	MIFS	SPLDG	SRFS	MSSL
EMOTIONS	.272 $\pm$ .051	.383 $\pm$ .056	.362 $\pm$ .043	.395 $\pm$ .062	.330 $\pm$ .054	.316 $\pm$ .053	.291 $\pm$ .047	.333 $\pm$ .057
YEAST	.249 $\pm$ .015	.255 $\pm$ .018	.254 $\pm$ .015	.278 $\pm$ .038	.254 $\pm$ .019	.256 $\pm$ .015	.247 $\pm$ .016	.259 $\pm$ .016
VOC07	.194 $\pm$ .017	.206 $\pm$ .015	.206 $\pm$ .019	.204 $\pm$ .014	.225 $\pm$ .018	.205 $\pm$ .013	.194 $\pm$ .014	.215 $\pm$ .014
MIRFlickr	.149 $\pm$ .014	.180 $\pm$ .010	.173 $\pm$ .017	.159 $\pm$ .013	.183 $\pm$ .011	.156 $\pm$ .009	.154 $\pm$ .012	.159 $\pm$ .012
SCENE	.085 $\pm$ .011	.099 $\pm$ .011	.091 $\pm$ .011	.136 $\pm$ .025	.097 $\pm$ .011	.100 $\pm$ .011	.090 $\pm$ .010	.093 $\pm$ .011
OBJECT	.173 $\pm$ .027	.199 $\pm$ .025	.182 $\pm$ .027	.228 $\pm$ .025	.183 $\pm$ .024	.192 $\pm$ .024	.186 $\pm$ .022	.185 $\pm$ .026

TABLE V : Experimental results

\downarrow

(mean

\pm

std) of each method on RL; the best result is shown in bold and the second-best is underlined.

Panel of Figure 10 — Fig. 5 : Performance comparison of SEHFS and seven baseline methods on SCENE (first row), VOC07 (second row), and MIRFlickr (third row). The horizontal axis denotes the number of selected features, and the vertical axis reports AP, Cov, HL, and RL.

Panel of Figure 11 — Fig. 6 : Comparison of SEHFS with other methods using the Bonferroni-Dunn test. Methods not connected to SEHFS indicate a significant difference in performance (CD = 3.2946 at the significance level of 0.05).

Evaluation Metric	$F_{F}$	Critical value
Average Precision	6.0485	2.2032
Coverage	5.9945
Hamming Loss	16.5200
Ranking Loss	5.8174

TABLE VI: Friedman statistics

F_{F}

over four evaluation metrics, along with the critical value at a 0.05 significance level for SEHFS.

Fig. 7 : Reordered feature graph heatmap according to the encoding-tree clustering learned by SEHFS on the Emotions dataset.

Panel of Figure 14 — Fig. 8 : Parameter sensitivity analysis of SEHFS on VOC07 (first row), SCENE (second row), and OBJECT (third row), where the horizontal axis indicates the parameter range, the vertical axis shows the number of selected features, and the depth axis represents AP.

Panel of Figure 15 — Fig. 9 : Ablation study results of SEHFS on VOC07, MIRFlickr, SCENE, and OBJECT, where each subfigure lists SEHFS and its four variants on the horizontal axis and reports the evaluation metric on the vertical axis, with HL on the left and RL on the right.

Panel of Figure 16 — Fig. 10 : Convergence curve of SEHFS on VOC07, MIRFlickr, Corel5K, and IAPRTC12.

Feature Selection

Feature selection identifies compact, informative variables or input dimensions to improve prediction, efficiency, interpretability, and statistical inference.

See all 196 papers

Apr 5, 2026

Incomplete Multi-View Multi-Label Classification via Shared Codebook and Fused-Teacher Self-Distillation

Although multi-view multi-label learning has been extensively studied, research on the dual-missing scenario, where both views and labels are incomplete, remains largely unexplored. Existing methods mainly rely on contrastive learning or information bottleneck theory to learn consistent representations under missing-view conditions, but loss-based alignment without explicit structural constraints limits the ability to capture stable and discriminative shared semantics. To address this issue, we introduce a more structured mechanism for consistent representation learning: we learn discrete consistent representations through a multi-view shared codebook and cross-view reconstruction, which naturally align different views within the limited shared codebook embeddings and reduce feature redundancy. At the decision level, we design a weight estimation method that evaluates the ability of each view to preserve label correlation structures, assigning weights accordingly to enhance the quality of the fused prediction. In addition, we introduce a fused-teacher self-distillation framework, where the fused prediction guides the training of view-specific classifiers and feeds the global knowledge back into the single-view branches, thereby enhancing the generalization ability of the model under missing-label conditions. The effectiveness of our proposed method is thoroughly demonstrated through extensive comparative experiments with advanced methods on five benchmark datasets. Code is available at https://github.com/xuy11/SCSD.

Xu Yan, Jun Yin, Shiliang Sun +1

College of Information Engineering, Shanghai Maritime University · School of Automation and Intelligent Sensing, Shanghai Jiao Tong University

Multi-Label Classification Multi-View Learning Learning with Missing Data Multi-View Consistency

Ask PDF

Oct 11, 2026

ProML: Progressive Multi-label Learning

Multi-label learning typically pursues model construction through the joint optimization of label fitting, correlation modeling, and structural regularization. While empirically effective, such a coupled paradigm imposes multiple learning objectives on a single predictive parameter matrix, forcing it to simultaneously fit observed labels and encode structural priors. This coupling may induce ill-conditioned optimization geometry, increase hyper-parameter sensitivity, and complicate capacity control. To address the these issues, this paper proposes a Progressive Multi-Label Learning (ProML) framework, which decouples direct label fitting from predictor learning via the introduction of an auxiliary predictive variable. This auxiliary variable admits a closed-form solution, yielding an equivalent profiled objective in which the supervised fitting term is scaled by a trade-off parameter. This result provides a unified view of ProML and conventional coupled learning, with the latter recovered as a limiting case. We theoretically analyze the optimization geometry of ProML and show that, under quadratic positive semidefinite structural regularization, ProML attenuates the dominant curvature of the fitting term, enlarges the stable learning rate range, and provides a controllable fidelity-regularization trade-off. Further, we instantiate the ProML for multi-label feature selection task by integrating global-local label priors and row-sparse regularization. Extensive experiments and controlled optimization analyses on 12 benchmark datasets empirically corroborate the theoretical analysis and demonstrate the robust and competitive performance of the proposed method.

Yi Kou, Jieting Wang, Yuhua Qian

Institute of Big Data Science and Industry, the School of Artificial Intelligence, and the Key Laboratory of Evolutionary Science Intelligence of Shanxi Province, Shanxi University, Taiyuan 030006, Shanxi, China

Multi-Label Classification

Ask PDF

Aug 16, 2026

Beyond Independence: Learning Correlated Views for Variational Incomplete Multi-View Clustering

Incomplete multi-view clustering (IMVC) aims to uncover shared cluster structures from data with partially observed views. Although recent imputation-free methods based on variational inference demonstrate robustness to missing views, they commonly rely on a conditional independence assumption across views in the posterior aggregation stage, which fails to capture the inherently structured and potentially correlated nature of multi-view data. In this paper, we propose a variational framework that explicitly goes beyond this assumption by introducing a learnable cross-view correlation structure. Specifically, we explicitly model and learn correlations between views by utilizing the covariance structure of posterior estimation errors during aggregation. To facilitate robust and efficient learning, the correlation matrix is parameterized through a normalized Cholesky decomposition, ensuring positive definiteness and enabling the entire model to be trained jointly through a unified variational objective. Extensive experiments on multiple IMVC benchmarks demonstrate that our method consistently outperforms state-of-the-art approaches across diverse missing-view settings while introducing only a negligible number of learnable parameters. These results highlight the effectiveness of adaptive correlation modeling in variational IMVC, demonstrating the need to go beyond the independence assumption in IMVC. The code is available at https://github.com/zmxu196/ACOVA.

Zheming Xu, Aiyue Tang, Shidi Chen +4

School of Computer Science & Technology, Beijing Jiaotong University · Key Laboratory of Big Data & Artificial Intelligence in Transportation, Ministry of Education, China · Department of Data Science, BI Norwegian Business School, Oslo, Norway +2

Multi-View Clustering Multi-View Learning Learning with Missing Data Variational Inference

Ask PDF

Abstract

Feature Selection

Explore similar work

Incomplete Multi-View Multi-Label Classification via Shared Codebook and Fused-Teacher Self-Distillation

ProML: Progressive Multi-label Learning

Beyond Independence: Learning Correlated Views for Variational Incomplete Multi-View Clustering