Comparative analysis of dual-form networks for live land monitoring using multi-modal satellite image time series

Iris Dumeur; Jérémy Anger; Gabriele Facciolo

eess.IVMar 25, 2026

Comparative analysis of dual-form networks for live land monitoring using multi-modal satellite image time series

Authors: Iris Dumeur, Jérémy Anger, Gabriele Facciolo

Organizations: CB · Kayrros SAS · Université Paris-Saclay, ENS Paris-Saclay, CNRS, Centre Borelli, 91190, Gif-sur-Yvette, France · Institut Universitaire de France

Abstract

Multi-modal Satellite Image Time Series (SITS) analysis faces significant computational challenges for live land monitoring applications. While Transformer architectures excel at capturing temporal dependencies and fusing multi-modal data, their quadratic computational complexity and the need to reprocess entire sequences for each new acquisition limit their deployment for regular, large-area monitoring. This paper studies various dual-form attention mechanisms for efficient multi-modal SITS analysis, that enable parallel training while supporting recurrent inference for incremental processing. We compare linear attention and retention mechanisms within a multi-modal spectro-temporal encoder. To address SITS-specific challenges of temporal irregularity and unalignment, we develop temporal adaptations of dual-form mechanisms that compute token distances based on actual acquisition dates rather than sequence indices. Our approach is evaluated on two tasks using Sentinel-1 and Sentinel-2 data: multi-modal SITS forecasting as a proxy task, and real-world solar panel construction monitoring. Experimental results demonstrate that dual-form mechanisms achieve performance comparable to standard Transformers while enabling efficient recurrent inference. The multimodal framework consistently outperforms mono-modal approaches across both tasks, demonstrating the effectiveness of dual mechanisms for sensor fusion. The results presented in this work open new opportunities for operational land monitoring systems requiring regular updates over large geographic areas.

Figures & tables

Attention mechanism	Similarity function $\text{s}(\mathbf{q}_{i},\mathbf{k}_{j})$
Transformer [ Vaswani et al., 2017 ]	$\exp({\nicefrac{{\mathbf{q}_{i}\cdot\mathbf{k}_{j}}}{{\sqrt{d_{K}}}}})$
Linear Transformer [ Katharopoulos et al., 2020 ]	$\left(1+\text{elu}(\mathbf{q}_{i})\right)\cdot\left(1+\text{elu}(\mathbf{k}_{j})\right)^{T}$
CosFormer [ Qin et al., 2022 ]	$\cos(\frac{\pi}{2}\times\frac{i-j}{M})\psi(\mathbf{q}_{i})\cdot\psi(\mathbf{k}_{j})^{T}$
RoPE Linear transformer [ Su et al., 2024 ]	$(\psi(\mathbf{q}_{n})R_{i\boldsymbol{\theta}})(\psi(\mathbf{k}_{m})R_{j\boldsymbol{\theta}})^{T}$

Table 1: Considered attention mechanisms in this study and corresponding similarity functions.

Figure 1: Overview of the proposed multi-modal spectro-spatio temporal architecture, composed of: a modality specific spectro-spatial encoder (noted

g^{S1}

resp.

g^{S2}

), a temporal fusion encoder, an upsampling operation with pixel shuffle (P.S) and a task specific decoder layer.

Figure 2: Multi-modal forecasting framework.

Figure 3: Visualization of the reconstruction in the multi-modal forecasting task using the Time CosFormer mechanism. The top row corresponds to the input and target SITS. The $(n-1)^{\text{th}}$ images are used by the model to predict the $n^{\text{th}}$ one. The bottom row corresponds to the reconstruction by our model.

Figure 4: Boxplot showing the MSE on the forecasting task for various temporal fusion encoders. Top row represents result obtained in the mono-modal S2 forecasting framework, bottom row corresponds to the multi-modal reconstruction loss. On the boxplot a line is drawn at the median value.

Figure 5: Effect of the look-back length in the forecasting task for various dual-form mechanisms. Mono-modal results are displayed on the top-line and multi-modal results on the bottom line.

Figure 6: Multi-temporal solar panel segmentation with S2 and S1 SITS performed with the Time CosFormer. The top row corresponds to the input SITS. The second row corresponds to the Sentinel-2 labels, with black representing non-solar panel pixels and red representing solar panels. Grey is associated with unlabeled areas. The third row corresponds to the predictions of our proposed model after the sigmoid function. The colormap matches the label scheme (1 is associated with red, 0.5 with white, and 0 with black).

Variant	Model	Mod	Accuracy	F1 Score	IoU
-	Transformer (non-causal)	S2	$0.92\pm 0.01$	$0.84\pm 0.02$	$0.73\pm 0.03$
-	Transformer (causal)	S2	$0.92\pm 0.01$	$0.85\pm 0.01$	$0.74\pm 0.02$
-	Linear	S2	$0.92\pm 0.01$	$0.85\pm 0.03$	$0.73\pm 0.04$
-	CosFormer	S2	$0.92\pm 0.01$	$0.85\pm 0.02$	$0.75\pm 0.04$
Time	CosFormer	S2	$0.92\pm 0.01$	$0.85\pm 0.02$	$0.74\pm 0.03$
-	LinRoFormer	S2	$0.91\pm 0.01$	$0.83\pm 0.02$	$0.71\pm 0.02$

Table 2: Multi-temporal solar panel segmentation metrics: mean and standard deviation across two different experiments. The Time variant indicates modified reweighting mechanisms that account for acquisition dates.

Efficient Multimodal Inference

Efficient multimodal inference reduces the compute, memory, latency, and energy needed to process image, video, audio, text, and sensor inputs.

See all 227 papers

Cropland PAtteRNS: Parallel Dimensional Attention Networks and Attention to Dataset Disparity for Crop Segmentation in Satellite Imagery Time Series Data
Sep 29, 2026Joseph Metcalfe, Sara Sharifzadeh, Fabio CaraffiniRemote Sensing Image Segmentation Agricultural Remote Sensing
Department of Computer Science, Swansea University, Swansea, SA1 8EN, United Kingdom
Multi-modal, multi-scale representation learning for satellite imagery analysis just needs a good ALiBi
Apr 11, 2026Patrick Kage, Pavlos AndreadisRepresentation Learning Remote Sensing Image Understanding
Artificial Intelligence and its Applications Institute, (University of Edinburgh, 10 Crichton Street, Newington, Edinburgh EH8 9AB)
UniverSat: Resolution- and Modality-Agnostic Transformers for Earth Observation
Jun 22, 2026Yohann Perron, Guillaume Astruc, Nicolas Gonthier +2Geospatial Representation Learning Vision Transformer
LIGM, Ecole Nationale des Ponts et Chaussées, IP Paris, Univ Gustave Eiffel, CNRS · EFEO · LASTIG, Univ Gustave Eiffel, IGN, ENSG +2

Sep 29, 2026J/K move · Enter open · S save

Cropland PAtteRNS: Parallel Dimensional Attention Networks and Attention to Dataset Disparity for Crop Segmentation in Satellite Imagery Time Series Data

Joseph Metcalfe, Sara Sharifzadeh, Fabio Caraffini

Department of Computer Science, Swansea University, Swansea, SA1 8EN, United Kingdom

Remote Sensing Image Segmentation Agricultural Remote Sensing Hybrid CNN-Transformer Architectures Hybrid Attention

Open paper AskPDF arXiv

The landscape of satellite imagery time series datasets and boundary-pushing architectures for cropland segmentation has never been richer. However, in this gold rush, important truths are being missed on both fronts, as a drive for the most novel concepts or the largest datasets pushes finer details to the side. In this paper, we present our hybrid transformer-convolutional model, Cropland Parallel Attention and Refinement Network for Segmentation (PAtteRNS), the first model to use self-attention mechanisms separately for each of the temporal, spectral, and spatial aspects of Sentinel-2 multispectral SITS data. To achieve fully-factorised attention in our proposed model, we introduce a novel parallel transformer architecture which significantly reduces the computational complexity of triple-factorised self-attention. We validate our architecture with an in-depth ablation study, and analyse the performance of our model against state-of-the-art crop segmentation models on multiple tile-size variants of the popular PASTIS and MTLCC datasets. Our findings show our model to outperform all others in the task of crop class segmentation, verified across multiple important segmentation metrics, with especially strong performance against compared models seen in the often under-reported parcel delineation quality, for which we use the Boundary IoU metric. We also find that flawed class groupings within datasets can have a significant negative impact on model performance, and report that alternate tile-size variants of crop segmentation datasets produce results incomparable to one-another, invalidating fair comparison between model performance when trained on different tile-sizes. Based on these findings, we suggest further work is required to standardise best practices when constructing SITS crop segmentation datasets, and to enable future dynamic-tile-sizing for ideal model performance.

Abstract

Efficient Multimodal Inference

Explore similar work

Cropland PAtteRNS: Parallel Dimensional Attention Networks and Attention to Dataset Disparity for Crop Segmentation in Satellite Imagery Time Series Data

Multi-modal, multi-scale representation learning for satellite imagery analysis just needs a good ALiBi

UniverSat: Resolution- and Modality-Agnostic Transformers for Earth Observation