cs.CVNov 19, 2026

Fusing Multimodal CNN Features with Structurally-Masked Transformers for Hierarchical Sub-Pixel Tree Species Mapping

Authors: Mattia Ferrari, Lorenzo Bruzzone

Organizations: Department of Information Engineering and Computer Science, University of Trento, 38123 Trento, Italy

Abstract

Precise forest species mapping using spaceborne hyperspectral data is challenging due to high spectral similarity and the inherent sub-pixel heterogeneity of natural forest stands. While traditional deep learning models exploit spatial context through convolutional architectures, these approaches can introduce noise in fragmented or mixed-species forests. In this paper, we propose a hierarchy-aware, multimodal deep learning framework to resolve sub-pixel forest composition by integrating PRISMA (PRecursore IperSpettrale della Missione Applicativa) hyperspectral imagery, high-resolution panchromatic data, and digital elevation models. The architecture features a multi-branch encoder combining an EfficientNetV2-based one-dimensional convolutional neural network (CNN) for spectral signatures and twin two-dimensional CNNs for spatial and topographic features, coupled with a novel structurally-masked Transformer. By using a binary transition tensor to condition the Transformer slot embeddings and constrain inference, the model enforces taxonomic consistency between broad forest categories and fine-grained species distributions. The framework is validated across three diverse Italian biogeographical regions (the Alps and the Central and Southern Apennines) using field-surveyed plots. The proposed method consistently outperforms state-of-the-art multi-label and ranking-based paradigms, particularly in identifying subordinate species and their relative dominance. The bidirectional learning mechanism allows sub-pixel compositional features to refine macroscopic forest type predictions, while the integration of topographic priors effectively resolves spectral ambiguities in complex terrains. This knowledge-driven method supports regional-scale biodiversity monitoring and forest characterization within the evaluated PRISMA scenarios.

Figures & tables

Explore similar work

CardsList
  1. From Articles to Canopies: Knowledge-Driven Pseudo-Labelling for Tree Species Classification using LLM Experts

    Apr 17, 2026Michał Romaszewski, Dominik Kopeć, Michał Cholewa +6Tree Species ClassificationGraph-Based Semi-Supervised Learning

  2. Reconstructing Multi-Decadal Forest Disturbances: A Spatio-Temporal Transformer Approach

    Jun 5, 2026Linus Scheibenreif, Anton Raichuk, Maxim NeumannRemote Sensing Change DetectionRemote Sensing

  3. Phase-Preserving Trimodal Transformer for Tropical Forest Biomass Estimation Using Optical and PolInSAR Data

    Jul 4, 2026Luiz Felipe Parente Santiago, Felipe Ferrari, Daniel Rodrigues dos Santos +1Forest Aboveground Biomass EstimationRemote Sensing