Bringing Clustering to MLL: Weakly-Supervised Clustering for Partial Multi-Label Learning
Organizations: School of Automation, Guangdong University of Technology
Abstract
Label noise in multi-label learning (MLL) poses significant challenges for model training, particularly in partial multi-label learning (PML) where candidate labels contain both relevant and irrelevant labels. While clustering offers a natural approach to exploit data structure for noise identification, traditional clustering methods cannot be directly applied to multi-label scenarios due to a fundamental incompatibility: clustering produces membership values that sum to one per instance, whereas multi-label assignments require binary values that can sum to any number. We propose a novel weakly-supervised clustering approach for PML (WSC-PML) that bridges clustering and multi-label learning through membership matrix decomposition. Our key innovation decomposes the clustering membership matrix into two components: , where maintains clustering constraints while preserves multi-label characteristics. This decomposition enables seamless integration of unsupervised clustering with multi-label supervision for effective label noise handling. WSC-PML employs a three-stage process: initial prototype learning from noisy labels, adaptive confidence-based weak supervision construction, and joint optimization via iterative clustering refinement. Extensive experiments on 24 datasets demonstrate that our approach outperforms six state-of-the-art methods across all evaluation metrics.
Figures & tables
| Datsets | #Instances | #Dims | #Labels | Avg.#CLs | Avg.#GLs |
|---|---|---|---|---|---|
| Mirflickr | 10433 | 100 | 7 | 3.35 | 1.77 |
| Music emotion | 6833 | 98 | 11 | 5.29 | 2.42 |
| Music style | 6839 | 98 | 10 | 6.04 | 1.44 |
| YeastBP | 6139 | 6139 | 217 | 5.93 | 5.54 |
| YeastCC | 6139 | 6139 | 50 | 1.39 | 1.35 |
| YeastMF | 6139 | 6139 | 39 | 1.04 | 1.01 |
| Data Set | Avg#CLS | WSC-PML | FBD-PML | PML-LENFN | NLR | PAMB | PML-NI | PML-fp |
|---|---|---|---|---|---|---|---|---|
| Mirflickr | 3.35 | 0.821±0.008 | 0.786±0.009 | 0.798±0.007 | 0.786±0.008 | 0.807±0.051 | 0.786±0.009 | 0.792±0.005 |
| Music_emotion | 5.29 | 0.649±0.012 | 0.604±0.013 | 0.610±0.012 | 0.610±0.012 | 0.626±0.011 | 0.608±0.012 | 0.435±0.093 |
| Music_style | 6.04 | 0.746±0.014 | 0.738±0.015 | 0.731±0.016 | 0.739±0.016 | 0.741±0.007 | 0.739±0.015 | 0.472±0.200 |
| YeastBP | 5.93 | 0.436±0.018 | 0.342±0.019 | 0.404±0.015 | 0.401±0.022 | 0.356±0.022 | 0.404±0.022 | 0.286±0.009 |
| YeastCC | 1.39 | 0.608±0.025 | 0.576±0.025 | 0.598±0.013 | 0.569±0.027 | 0.559±0.022 | 0.452±0.026 | 0.360±0.020 |
| YeastMF | 1.04 | 0.490±0.028 | 0.456±0.031 | 0.480±0.012 | 0.470±0.021 | 0.407±0.011 | 0.417±0.013 | 0.359±0.015 |
| Evaluayion metric | Critical value | |
|---|---|---|
| Haming loss | 25.4261 | |
| Ranking loss | 22.0066 | |
| One-error | 16.2616 | 1.8386 |
| Coverage | 23.0114 | |
| Average Precise | 26.7644 |
| Data Set | Avg#CLS | WSC-PML | FBD-PML | PML-LENFN | NLR | PAMB | PML-NI | PML-fp |
|---|---|---|---|---|---|---|---|---|
| Mirflickr | 3.35 | 0.110±0.006 | 0.124±0.005 | 0.118±0.005 | 0.127±0.006 | 0.112±0.038 | 0.126±0.007 | 0.124±0.003 |
| Music_emotion | 5.29 | 0.224±0.009 | 0.247±0.009 | 0.246±0.009 | 0.248±0.008 | 0.234±0.007 | 0.246±0.008 | 0.458±0.114 |
| Music_style | 6.04 | 0.132±0.011 | 0.137±0.011 | 0.138±0.010 | 0.138±0.010 | 0.135±0.005 | 0.137±0.010 | 0.445±0.231 |
| yeastBP | 5.93 | 0.199±0.009 | 0.251±0.009 | 0.226±0.011 | 0.228±0.011 | 0.230±0.011 | 0.220±0.011 | 0.269±0.012 |
| YeastCC | 1.39 | 0.156±0.016 | 0.182±0.020 | 0.178±0.021 | 0.189±0.020 | 0.187±0.026 | 0.193±0.028 | 0.187±0.023 |
| YeastMF | 1.04 | 0.209±0.019 | 0.245±0.022 | 0.235±0.020 | 0.239±0.012 | 0.253±0.025 | 0.247±0.031 | 0.249±0.024 |
| AP | NC | ND | NWS | Full |
|---|---|---|---|---|
| Mirflickr | 0.800±0.012 | 0.811±0.012 | 0.815±0.009 | 0.821±0.008 |
| Music_emotion | 0.624±0.013 | 0.627±0.015 | 0.637±0.019 | 0.649±0.012 |
| birds (5) | 0.584±0.057 | 0.593±0.044 | 0.596±0.051 | 0.597±0.050 |
| image (3) | 0.758±0.018 | 0.781±0.017 | 0.786±0.026 | 0.792±0.019 |
Multi-Label Classification
Multi-label classification assigns several potentially related tags to each example, from medical findings and topics to objects and user interests.
See all 135 papers