Multi-label classification (MLC) requires predicting multiple relevant labels for each instance, where a central challenge is modeling complex label dependencies arising from co-occurrence patterns. Existing approaches are limited in capturing high-order label correlations, relying on implicit learning through contrastive objectives or pairwise attention mechanisms without structural guidance. We propose HyperLabel, an encoder-decoder framework that explicitly models label dependencies through hypergraph neural networks. Our contributions are twofold: (i) We construct a label hypergraph where sample-defined hyperedges naturally encode multi-way co-occurrence patterns, providing explicit structural prior knowledge that captures relationships beyond pairwise interactions. (ii) We propose a unified cross-modal learning approach where HGNN+ performs bidirectional message passing to integrate feature information with label structure, and a shared cross-attention decoder processes both modalities through complementary learning objectives. Extensive experiments on seven benchmark datasets demonstrate that HyperLabel achieves state-of-the-art performance, with particularly significant improvements on macro-F1 scores (+10.3% on Delicious, +8.2% on Bibtex), validating that explicit hypergraph structure effectively captures complex label relationships. The code is available at https://github.com/iZHpy/Multi-label_hypergraph .
Figures & tables
Figure 1: Hypergraph construction process. Label nodes V={1,2,…,L} are connected by sample-defined hyperedges. Each sample’s positive label set P(yi) defines a hyperedge (e.g., e1∗={1,5,8} ). Duplicate hyperedges are merged and assigned weights wk based on occurrence frequency. The resulting hypergraph explicitly encodes multi-way label co-occurrence patterns.
Figure 2: Overview of HyperLabel framework. Labels are encoded via HGNN+ with hypergraph message passing, while features pass through a Transformer encoder. Aggregated label embeddings are aligned with feature embeddings and both are processed by a shared cross-attention decoder to produce label predictions y^ij and reconstructed predictions y^ijrecon .
Dataset
#Samples
#Features
#Labels
Card.
Density
Domain
Yeast
2,417
103
14
4.24
0.303
Biology
Scene
2,407
294
6
1.07
0.178
Vision
SIDER
1,427
27
27
15.30
0.567
Medical
Reuters
10,789
23,662
90
1.23
0.014
Text
Bibtex
7,395
1,836
159
2.40
0.015
Bibliography
Bookmarks
87,856
2,150
208
2.03
0.010
Social Tagging
Table 1: Statistics of benchmark datasets. Cardinality denotes the average number of labels per sample, and Density represents the ratio of positive labels to total possible labels.
Dataset
bibtex
sider
bookmarks
Metric
ebF1
miF1
maF1
ebF1
miF1
maF1
ebF1
miF1
maF1
w/o HGNN+
0.426
0.449
0.354
0.767
0.797
0.665
0.366
0.360
0.273
HGNN+ → GNN
0.424
0.458
0.362
0.766
0.799
0.665
0.383
0.376
0.293
Transformer → MLP
0.418
0.444
0.364
0.765
0.796
0.665
0.384
0.377
0.285
Decoupled Decoder
0.424
0.460
0.350
0.768
0.801
0.667
0.374
0.374
0.278
HyperLabel
0.433
0.462
0.370
0.772
0.805
0.668
0.390
0.383
0.302
Table 3: Ablation study results on selected datasets.
Figure 3: Learned correlations in HyperLabel on a Reuters sample. Top : feature-label correlation map. Bottom : label-label correlation map.
Large-scale multi-label text classification assigns a small subset of relevant labels to each document from a vocabulary containing thousands or tens of thousands of candidate labels. Although pretrained language models have improved semantic text representations, most representation-based approaches center their prediction pipelines on a primary encoder or combine auxiliary features within a single ranker. The complementarity between heterogeneous language models therefore remains insufficiently explored. We propose DualMLC, a dual-branch framework that processes the same document through an autoregressive decoder-only language model and a bidirectional encoder. Each branch maintains its own representation pathway and independently estimates relevance scores over the shared label space. DualMLC combines the two score vectors through late logit fusion, allowing shared evidence to reinforce relevant labels and branch-specific evidence to compensate for limitations in the other branch's representation. DualMLC achieves state-of-the-art results on three widely used large-scale multi-label text classification benchmarks. Ablation results further confirm that integrating the heterogeneous predictors produces stronger rankings than either branch alone. The source code is publicly available at https://github.com/huiyegit/DualMLC.
Hui Ye, Jing Zhang, Xiulong Yang +1
Department of Computer Science, Georgia State University, Atlanta, GA 30303, USA · Amazon, San Diego, CA, USA · School of Computer Science, Central China Normal University, Wuhan 430079, China
Graphs are a complex and versatile data structure used across various domains, with possibly multi-label nodes playing a particularly crucial role. Examples include proteins in PPI networks with multiple functions and users in social or e-commerce networks exhibiting diverse interests. Tackling multi-label node classification (MLNC) on graphs has led to the development of various approaches. Some methods leverage graph neural networks (GNNs) to exploit label co-occurrence correlations, while others incorporate label embeddings to capture label proximity. However, these approaches fail to account for the intricate influences between labels in non-Euclidean graph data. To address this issue, we decompose the message passing process in GNNs into two operations: propagation and transformation. We then conduct a comprehensive analysis and quantification of the influence correlations between labels in each operation. Building on these insights, we propose a novel model, Label Influence Propagation (LIP). Specifically, we construct a label influence graph based on the integrated label correlations. Then, we propagate high-order influences through this graph, dynamically adjusting the learning process by amplifying labels with positive contributions and mitigating those with negative influence. Finally, our framework is evaluated on comprehensive benchmark datasets, consistently outperforming SOTA methods across various settings, demonstrating its effectiveness on MLNC tasks.
Yifei Sun, Zemin Liu, Bryan Hooi +4
Zhejiang University · National University of Singapore · Capital One +1
Heterogeneous graphs have attracted increasing attention for modeling multi-typed entities and relations in complex real-world systems. Multi-label node classification on heterogeneous graphs is challenging due to structural heterogeneity and the need to learn shared representations across multiple labels. Existing methods typically adopt either flexible attention mechanisms or meta-path constrained anchoring, but in heterogeneous multi-label prediction they often suffer from semantic dilution or coverage constraint. Both issues are further amplified under multi-label supervision. We present a theoretical analysis showing that as heterogeneous neighborhoods expand, the attention mass allocated to task-critical (primary) neighborhoods diminishes, and that meta-path constrained aggregation exhibits a dilemma: too few meta-paths intensify coverage constraint, while too many re-introduce dilution. To resolve this coverage-anchoring conflict, we propose FOCAL: Fusion Of Coverage and Anchoring Learning, with two components: coverage-oriented attention (COA) for flexible, unconstrained heterogeneous context aggregation, and anchoring-oriented attention (AOA) that restricts aggregation to meta-path-induced primary semantics. Our theoretical analysis and experimental results further indicates that FOCAL has a better performance than other state-of-the-art methods.
Chenghao Zhang, Qingqing Long, Ludi Wang +3
Computer Network Information Center, Chinese Academy of Sciences, Beijing, China · University of Chinese Academy of Sciences, Beijing, China