Hierarchical multi-label classification (HMLC) is essential for modeling structured label dependencies in remote sensing. Yet existing approaches struggle in multi-path settings, where images may activate multiple taxonomic branches, leading to underuse of hierarchical information. We propose MAPLE (Multi-Path Adaptive Propagation with Level-Aware Embeddings), a framework that integrates (i) hierarchical semantic initialization from graph-aware textual descriptions, (ii) graph-based structure encoding via graph convolutional networks (GCNs), and (iii) adaptive multi-modal fusion that dynamically balances semantic priors and visual evidence. An adaptive level-aware objective automatically selects appropriate losses per hierarchy level. Evaluations on CORINE-aligned remote sensing datasets (AID, DFC-15, and MLRSNet) show consistent improvements of up to +42% in few-shot regimes while adding only 2.6% parameter overhead, demonstrating that MAPLE effectively and efficiently models hierarchical semantics for Earth observation (EO).
Figures & tables
Figure 1: MAPLE architecture overview. The framework processes an input image with a ViT encoder that uses hierarchy-specific class tokens. A GCN refines these token embeddings by propagating information along the label taxonomy. Finally, visual features and refined semantic embeddings are fused via adaptive gating to produce level-aware classifications.
MAPLE
Dataset
l1
l2
l3
Leaf
MLC
Δ (%)
AID
95.31
94.32
84.83
87.25
84.21
+3.6
DFC-15
99.62
98.33
98.37
98.71
98.65
+0.1
MLRSNet
98.23
97.88
96.77
96.71
96.17
+0.6
Table 1: Hierarchical vs. flat classification (AU PRC %). MAPLE outperforms the flat baseline at all hierarchy levels ( l1 – l3 ) and leaf nodes.
Method
4-shot
8-shot
12-shot
16-shot
AID
MLC
0.286±0.018
0.334±0.012
0.341±0.025
0.310±0.009
MAPLE
0.357±0.021
0.371±0.017
0.396±0.039
0.440±0.046
Δ (%)
+25.0
+11.1
+16.1
+41.9
DFC-15
MLC
0.541±0.044
0.541±0.012
0.790±0.008
0.807±0.021
Table 2: Few-shot performance (AU PRC ) with K-shot training per class. μ±σ over 3 runs. Δ shows MAPLE’s improvement over the MLC baseline.
Method
AID
DFC-15
MLRSNet
C-HMCNN [ 11 ]
0.764
0.962
0.792
HiMulConE [ 7 ]
0.770
0.970
0.865
HMI [ 23 ]
0.647
0.923
0.437
MAPLE (Ours)
0.872
0.987
0.967
Table 3: Comparison with state-of-the-art HMLC methods on leaf-level AU PRC ( ↑ ). Best in bold , second-best underlined .
Appendix figures & tables11 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 2: Examples of constructed label hierarchies for (a) the AID dataset, derived from the CORINE Land Cover (CLC) nomenclature, and (b) the MuRed dataset, with abbreviations in leaf labels corresponding to ICD-10 codes of disease names.
N
Ntrain
Ntest
∣L∣
Dataset
1
2
3
4
5
6
ℓ
AID
3,000
2,400
600
4
9
15
7
-
-
17
DFC-15
3,341
2,672
669
3
7
7
-
-
-
8
MLRSNet
109,151
87,336
21,815
7
15
22
60
-
-
60
MuRed
2,208
1,764
444
4
8
17
5
-
-
20
HPA
31,072
24,837
6,235
6
28
-
-
-
-
28
Appendix
Table 4: Summary of the datasets used in this study. N denotes the total number of images in each dataset, while Ntrain and Ntest represent the number of images in the training and test sets, respectively. ∣L∣ indicates the number of unique labels at each hierarchical level, where ℓ corresponds to the deepest (leaf) level.
Figure 3: Representative examples from the nine datasets used in our evaluation, showing sample images alongside their corresponding hierarchical label structures. The datasets span three domains: remote sensing (AID, DFC-15, MLRSNet), medical imaging (MuRed, HPA, PadChest), and fine-grained visual categorization (OxfordPets-37, Stanford Cars, ETHEC). Each hierarchy displays the multi-level taxonomic organization from coarse-grained categories at Level 1 to fine-grained leaf labels.
Dataset
MAPLE*
MAPLE
Flat Baseline
Δ∗ (%)
Δ (%)
Remote Sensing Datasets
AID
l1
95.31
94.11
-
-
-
l2
94.32
93.10
-
-
-
l3
84.83
82.50
-
-
-
Leaf
87.25
86.66
84.21
+3.61
+2.91
DFC-15
l1
99.62
99.51
-
-
-
Appendix
Table 5: Complete performance comparison across hierarchical levels using AU PRC (%). This table extends Table 1 from the main paper with results across all nine datasets and all hierarchical levels. li denotes the i -th hierarchical level. MAPLE* uses semantic initialization while MAPLE uses random initialization. Δ∗ and Δ show relative improvements over flat baseline.
Figure 5: Few-shot learning performance comparison between MAPLE (in green ) and flat MLC baseline (in red ) across representative datasets. Results show AU PRC performance ( μ±σ across three experimental repeats) for varying numbers of shots per category ( K∈{4,8,12,16} ). MAPLE consistently outperforms the baseline, with particularly pronounced benefits in low-data regimes.
Per Image (ms)
Model
8
16
32
64
GFLOPs
Params (M)
Flat Baseline
3.02
2.96
2.62
2.55
33.54
86.57
MAPLE
3.18
3.01
2.65
2.61
33.77
88.84
Overhead (%)
+5.3
+1.7
+1.1
+2.4
+0.7
+2.6
Appendix
Table 6: Computational efficiency comparison on AID dataset.
Figure 6: Evolution of node embeddings throughout MAPLE training stages visualized using UMAP dimensionality reduction on the AID dataset. (a) Initial embeddings show random spatial distribution. (b) Learned embeddings after hierarchical semantic initialization demonstrate clear semantic clustering. (c) GNN embeddings after graph neural network refinement exhibit enhanced separation between semantic groups. (d) Visual-fused embeddings after multimodal fusion achieve sophisticated organization, integrating both visual and semantic information while respecting hierarchical structure.
Dataset
Baseline
MAPLE
Improvement (%)
Absolute Reduction
AID
107
94
12.1
13
MuRed
77
76
1.3
1
Oxford Pets
41
12
70.7
29
Appendix
Table 7: Leaf-level confusion reduction across datasets. MAPLE consistently reduces confusion instances compared to baseline models, with particularly strong improvements on Oxford Pets (70.7% reduction) and AID (12.1% reduction). Lower values indicate better performance.
Figure 7: Leaf-level confusion matrix comparison for AID dataset. Left: baseline model confusions, Center: MAPLE confusions, Right: difference matrix (MAPLE − Baseline). Blue cells indicate reduced confusions (improvements), red cells indicate increased confusions. MAPLE achieves a 12.1% reduction in total leaf-level confusions, with notable improvements in semantically challenging pairs such as trees → water and buildings → bare-soil.
Figure 8: Performance comparison of different GNN architectures across datasets.
Figure 9: Comparison of embedding initialization strategies.
Hierarchical multi-label classification (HMLC) is essential for modeling complex label dependencies in remote sensing. Existing methods, however, struggle with multi-path hierarchies where instances belong to multiple branches, and they rarely exploit unlabeled data. We introduce HELM (Hierarchical and Explicit Label Modeling), a novel framework that overcomes these limitations. HELM: (i) uses hierarchy-specific class tokens within a Vision Transformer to capture nuanced label interactions; (ii) employs graph convolutional networks to explicitly encode the hierarchical structure and generate hierarchy-aware embeddings; and (iii) integrates a self-supervised branch to effectively leverage unlabeled imagery. We perform a comprehensive evaluation on four remote sensing image (RSI) datasets (UCM, AID, DFC-15, MLRSNet). HELM achieves state-of-the-art performance, consistently outperforming strong baselines in both supervised and semi-supervised settings, demonstrating particular strength in low-label scenarios.
Marjan Stoimchev, Boshko Koloski, Jurica Levatić +2
Jožef Stefan Institute, Ljubljana, Slovenia · Jožef Stefan International Postgraduate School, Ljubljana, Slovenia
Deep learning has become increasingly important in remote sensing image classification due to its ability to extract semantic information from complex data. Classification tasks often include predefined label hierarchies that represent the semantic relationships among classes. However, these hierarchies are frequently overlooked, and most approaches focus only on fine-grained classification schemes. In this paper, we present a novel Semantics-Aware Hierarchical Consensus (SAHC) approach that integrates hierarchical-level-specific classification heads within a deep network architecture and combines their output through cross-level probability projectors. Direct and projected predictions are fused into a geometric consensus distribution, which is used for self-consistent training and optional hierarchy-aware inference. This mechanism acts as a geometric ensemble that leverages the inherent structure of the hierarchical classification task. The projectors are initialized from the user-defined taxonomy (i.e., the hierarchical label structure), and can be adaptively refined during optimization. The proposed SAHC method is evaluated on two benchmark datasets with different degrees of hierarchical complexity on different tasks, considering varying spectral and spatial resolutions. Experimental results show both the effectiveness of the proposed approach in guiding network learning and the robustness of the hierarchical consensus for remote sensing image classification tasks. The source code is available at https://github.com/rslab-unitrento/sahc.
Giulio Weikmann, Gianmarco Perantoni, Lorenzo Bruzzone
Department of Information Engineering and Computer Science, University of Trento, 38123 Trento, Italy
Multimodal contrastive learning has enabled zero-shot visual classification by aligning images with textual categories. However, in hierarchically structured label spaces, existing methods often produce predictions that are inconsistent across taxonomic levels. For example, a model may predict a fine-grained category whose parent category contradicts its simultaneously predicted higher-level label. By analysis, the issue originates from false negative labels when contrastive comparison involves multiple taxonomic levels. To this end, we propose to restrict contrastive comparisons to categories within the same taxonomic level. In addition, we adopt a group-balanced design, ensuring each taxonomic level receives adequate optimization. As a result, the proposed framework improves both hierarchical consistency and classification accuracy from coarse to fine granularity. We train our model with TreeOfLife-10M based on BioCLIP and evaluate it across multiple hierarchical classification benchmarks, where the model demonstrates significantly improved hierarchical consistency in both Euclidean and hyperbolic spaces. Notably, on iNaturalist 2021 (iNat21), our method improves average accuracy across levels by 30.47% over the baseline, highlighting its effectiveness for hierarchical zero-shot classification.
Zhiyuan Tao, Srikumar Sastry, Matthew J Thompson +9
The Ohio State University · Washington University in St. Louis · Neuromatch +1