Cropland PAtteRNS: Parallel Dimensional Attention Networks and Attention to Dataset Disparity for Crop Segmentation in Satellite Imagery Time Series Data

Joseph Metcalfe; Sara Sharifzadeh; Fabio Caraffini

cs.CVSep 29, 2026

Cropland PAtteRNS: Parallel Dimensional Attention Networks and Attention to Dataset Disparity for Crop Segmentation in Satellite Imagery Time Series Data

Authors: Joseph Metcalfe, Sara Sharifzadeh, Fabio Caraffini

Organizations: Department of Computer Science, Swansea University, Swansea, SA1 8EN, United Kingdom

Abstract

The landscape of satellite imagery time series datasets and boundary-pushing architectures for cropland segmentation has never been richer. However, in this gold rush, important truths are being missed on both fronts, as a drive for the most novel concepts or the largest datasets pushes finer details to the side. In this paper, we present our hybrid transformer-convolutional model, Cropland Parallel Attention and Refinement Network for Segmentation (PAtteRNS), the first model to use self-attention mechanisms separately for each of the temporal, spectral, and spatial aspects of Sentinel-2 multispectral SITS data. To achieve fully-factorised attention in our proposed model, we introduce a novel parallel transformer architecture which significantly reduces the computational complexity of triple-factorised self-attention. We validate our architecture with an in-depth ablation study, and analyse the performance of our model against state-of-the-art crop segmentation models on multiple tile-size variants of the popular PASTIS and MTLCC datasets. Our findings show our model to outperform all others in the task of crop class segmentation, verified across multiple important segmentation metrics, with especially strong performance against compared models seen in the often under-reported parcel delineation quality, for which we use the Boundary IoU metric. We also find that flawed class groupings within datasets can have a significant negative impact on model performance, and report that alternate tile-size variants of crop segmentation datasets produce results incomparable to one-another, invalidating fair comparison between model performance when trained on different tile-sizes. Based on these findings, we suggest further work is required to standardise best practices when constructing SITS crop segmentation datasets, and to enable future dynamic-tile-sizing for ideal model performance.

Figures & tables

Figure 1: The structure of a vision transformer encoder block. This structure is repeated to the desired depth of the transformer $L$

Figure 2: The geographical extent of the two datasets selected for this study across France and Germany.

Figure 3: The parallel temporal and spectral patch extraction structure from raw SITS data. The input SITS tile tensor $I\in\mathbb{R}^{T\times S\times H\times W}$ is taken twice, once for the temporal transformer branch (top), and once for the spectral transformer branch (bottom). Each branch begins by partitioning the raw data into the same pattern of non-overlapping spatial patches with size $P_{h}\times P_{w}$ , giving $N_{h}=H/P_{h}$ and $N_{w}=W/P_{w}$ patches. Each branch then constructs a token sequence per patch, with the temporal branch flattening each patch along the spectral and spatial dimensions $S\times P_{h}\times P_{w}$ while preserving the temporal dimension, and the spectral branch flattening each patch along the temporal and spatial dimensions $T\times P_{h}\times P_{w}$ while preserving the spectral dimension. This yields $N=N_{h}\times N_{w}$ token sequences per each of the spectral and temporal branches, which are separately projected through a linear layer into the embedding dimension of size $d$ .

Figure 4: The compared Fuse Block structures for the $cls$ token fusion following our parallel spectral and temporal transformer branches. The outputs of these fuse blocks are passed onwards to the spatial embedding process, and then the spatial transformer.

Figure 5: The overall architecture of our SITS Crop Segmentation Model. We name our model as PAtteRNS: P arallel Atte ntion and R efinement N etwork for S egmentation. While this network is intended for cropland segmentation, the same concepts may apply across other fields in remote sensing.

			MTLCC-48			PASTIS-128
Transformers $temp$ / $spec$ / $spat$	Cross Attention	Refine Head	OA%	mIoU	mBIoU	OA%	mIoU	mBIoU
✓/ ✓/ ✓	$S{\leftarrow}T$	✓	92.0	0.774	0.527	83.3	0.649	0.435
✓/ ✓/ ✓	$S{\leftarrow}T$	–	91.7	0.762	0.513	82.8	0.641	0.428
✓/ ✓/ ✓	$T{\leftarrow}S$	✓	91.6	0.756	0.509	81.9	0.612	0.394
✓/ ✓/ ✓	$T{\leftarrow}S$	–	91.0	0.724	0.491	81.3	0.600	0.386
✓/ ✓/ ✓	$Bi$	✓	92.1	0.774	0.537	83.0	0.640	0.429

Table 2: Our design ablation study on the core components of our PAtteRNS model. The ablation study was performed on both MTLCC-48 and PASTIS-128’s first folds, with the mean of 3 seeded repeats reported. The cross-attention fuse block element is only present where both temporal and spectral transformers are active for the combination of parallel

cls

token sets prior to the spatial transformer, with

S{\leftarrow}T

representing the Spectral Queries Temporal fuse block and so forth as illustrated in Figure 4 . Results are reported as overall percentage accuracy, mean Intersection over Union, and mean Boundary Intersection over Union. Bold = best per metric per dataset.

Patch $p$	Dim $d$	$cls$ tokens	OA%	mIoU	mBIoU	Inference Time (ms)	Params (M)
4	256	$k$	92.3	0.779	0.533	22.95	14.94
4	128	$k$	92.0	0.773	0.537	12.72	4.56
4	256	1	92.1	0.773	0.535	17.01	12.70
4	128	1	92.0	0.774	0.538	9.88	4.00
2	256	$k$	92.3	0.774	0.564	86.66	12.87
2	128	$k$	92.2	0.774	0.563	37.05	3.52

Table 3: Ablation over patch size

p

, hidden dimension

d

, and number of

cls

tokens, reporting the mean of three seeded repeats on just MTLCC-48’s first fold. Inference Time is reported for the mean time per batch of 4 samples over the fold’s test set for all three seeds. Other results are reported as overall percentage accuracy, mean Intersection over Union, and mean Boundary Intersection over Union, and model trainable Parameters. Bold = best per metric.

		MTLCC-48			PASTIS-128
Pad Mask	Date Encoding Method	OA%	mIoU	mBIoU	OA%	mIoU	mBIoU
Y	DSRD	91.9	0.771	0.532	82.9	0.638	0.424
Y	DOY	92.1	0.775	0.530	82.9	0.637	0.423
N	DSRD	92.0	0.773	0.537	83.1	0.643	0.428
N	DOY	91.9	0.772	0.526	82.9	0.640	0.424

Table 4: Ablation over padding mask and temporal date encoding scheme, evaluated on both MTLCC-48 and PASTIS-128’s first folds. Results are reported as overall percentage accuracy, mean Intersection over Union, and mean Boundary Intersection over Union.

Model	Patch Size i	OA%	mIoU	Dice	mBIoU ii
MTLCC 2016		$48\times 48$ pixel tiles / $24\times 24$ pixel tiles
UNet3D	–	90.3/90.0	0.694/0.726	0.799/0.826	0.448/0.463
UTAE	–	92.6 /92.1	0.782/0.790	0.867/0.874	0.550/0.542
TSViT	4	91.0/90.4	0.749/0.755	0.844/0.849	0.538/0.510
TSViT	2	91.5/90.9	0.757/0.769	0.849/0.859	0.566/0.554
PAtteRNS (proposed)	4	92.2/91.5	0.783/0.789	0.869/0.874	0.546/0.520

Table 5: Comparison of state-of-the-art methods in semantic segmentation on dataset tile-size variants. All reported results are the mean of 3 iteratively-seeded repeats of the mean performance across all folds in a dataset. For each fold, the best epoch by test set performance is considered. All results in these tables are our results, as opposed to results stated in other works. Results are reported as overall percentage accuracy, mean Intersection over Union, Dice Score, and mean Boundary Intersection over Union. Bold = best per metric per dataset, and for mBIoU, per tile-size ii .

Figure 6: An illustration of PAtteRNS’ segmentation capabilities against comparison models. Our model is shown here to combine precise boundaries with full recall of present classes in the ground truth labels of a tile from MTLCC-48.

Model	Params (M)	MACs (G)	Inference Time (ms)	Mean Epoch Duration (s)	Mean Best Epoch
UNet3D	1.6	20.2	14.0	42.7	84.0
UTAE	1.1	9.8	24.3	98.0	135.7
TSViT $P4$	1.7	9.4	13.4	59.0	69.7
PAtteRNS $P4$	3.4	11.5	13.5	40.3	140.3
TSViT $P2$	1.7	36.8	56.1	167.0	90.0
PAtteRNS $P2$	3.3	44.5	37.9	143.7	87.7

Table 6: Model Computational Performance Comparison. Model trainable parameters, MACs of the forward pass, mean inference time, and mean training time are reported. All reported results are for 3 seeded repeats using MTLCC 48’s first fold, with inference time reporting the mean time per batch of 4 samples over the test, mean epoch duration reporting the average combined train and validation duration per epoch using our training setup, and best epoch measured as the average index of the best test set mIoU out of 150 maximum epochs from training. MACs are reported for the dimensions of MTLCC 48.

Figure 7: Mean Boundary IoU (mBIoU) achieved by each model across all four dataset and tile-size combinations. PAtteRNS with $p=2$ achieves the leading result in three of the four combinations, with PAtteRNS at $p=4$ leading marginally on PASTIS-128 (see Table 5 ).

Appendix figures & tables17 assets

Supplementary material from the paper’s appendix.

Appendix

Class Name	PASTIS ID	MTLCC ID	Aggregation
Background *	0	0
Meadow	1	3	NH
Winter Soft Wheat *	2	–
Maize *	3	–
Winter Barley	4	14
Winter Rapeseed	5	–

Appendix

Table 7: Correspondence between PASTIS and MTLCC crop classes. Classes are ordered first by PASTIS ID, then by remaining MTLCC-exclusive class ID. Classes are annotated with the issues and flawed groupings introduced in section 2.4 , where NH

=

Non-Homogeneous, TA

=

Taxonomical Aggregate, and SA

=

Seasonal Aggregate.

Figure 8: Class segmentation outputs, and boundary error maps, for all models on an example tile from MTLCC-48.

Figure 9: Class segmentation outputs, and boundary error maps, for all models on an example tile from MTLCC-48.

Figure 10: Class segmentation outputs, and boundary error maps, for all models on an example tile from MTLCC-24.

Figure 11: Class segmentation outputs, and boundary error maps, for all models on an example tile from MTLCC-24.

Figure 12: Class segmentation outputs, and boundary error maps, for all viable models on an example tile from PASTIS-128.

Figure 13: Class segmentation outputs, and boundary error maps, for all viable models on an example tile from PASTIS-128.

Figure 14: Class segmentation outputs, and boundary error maps, for all models on an example tile from PASTIS-24.

Figure 15: Class segmentation outputs, and boundary error maps, for all models on an example tile from PASTIS-24.

Figure 16: The cumulative class-wise confusion matrix of all seeded repeats and folds for PAtteRNS $p=2$ on PASTIS 24

Figure 17: The cumulative class-wise confusion matrix of all seeded repeats and folds for TSViT $p=2$ on PASTIS 24

Figure 18: The cumulative class-wise confusion matrix of all seeded repeats and folds for UTAE on PASTIS 24

Figure 19: The cumulative class-wise confusion matrix of all seeded repeats and folds for PAtteRNS $p=4$ on PASTIS 128

Figure 20: The cumulative class-wise confusion matrix of all seeded repeats and folds for UTAE on PASTIS 128

Figure 21: The cumulative class-wise confusion matrix of all seeded repeats and folds for PAtteRNS $p=2$ on MTLCC 24

Figure 22: The cumulative class-wise confusion matrix of all seeded repeats and folds for TSViT $p=2$ on MTLCC 24

Figure 23: The cumulative class-wise confusion matrix of all seeded repeats and folds for UTAE on MTLCC 24

Explore similar work

May 30, 2026cs.CV

A Modelling and Evaluation Framework for EuroCrops-Driven Sentinel-2 Crop Segmentation

This work presents a configurable pipeline for generating semantic-segmentation-ready agricultural datasets from Sentinel-2 imagery and EuroCrops parcel-level annotations. The workflow transforms heterogeneous vector crop annotations into aligned multispectral image--mask pairs through label harmonization, Sentinel-2 product selection, spatial alignment, rasterization, patch extraction, quality filtering, and class-aware sample selection. The generated dataset contains 67,337 patches from five European countries and uses a reduced taxonomy of ten crop classes plus background. A four-level U-Net with Group Normalization was trained using 10 Sentinel-2 spectral bands and a composite loss combining class-weighted cross-entropy and Dice loss. On the internal EuroCrops-based test split, the model achieved a mean Intersection over Union (mIoU) of 0.7665, a pixel accuracy of 0.8693, and a mean class accuracy of 0.9072. Compared with spectral and spatial-context Random Forest baselines, the U-Net showed the importance of learned multi-scale spatial representations for crop segmentation. External evaluation was performed on unseen Belgian EuroCrops subsets, DACIA5, and PASTIS. The results show a clear performance gap under external and cross-dataset evaluation, especially for benchmarks with different taxonomies, annotation protocols, spatial coverage, or temporal organization. The model transfers more reliably to dominant and taxonomically aligned classes such as maize and wheat, while performance remains limited for several minority classes and for the adapted single-date PASTIS setting. These findings highlight both the potential and the limitations of using EuroCrops-derived supervision for Sentinel-2 crop segmentation under realistic domain shifts.

Alexandra Nicoleta Scarlat, Ioana Cristina Plajer, Alexandra Baicoianu

Transilvania University of Braşov, Faculty of Mathematics and Computer Science, Braşov, Romania

Sentinel-2 Land Cover U-Net Dense Annotation

Ask PDF

Aug 10, 2026cs.CV

SwissCrop25: A National Multi-Year Benchmark for Operational Crop Mapping

Operational crop mapping requires models that generalise across years, resolve fine-grained crop taxonomies, and distinguish cropland from surrounding landscapes. However, existing crop mapping datasets enable evaluation of these requirements only in isolation. We therefore introduce SwissCrop25, a national-scale crop mapping benchmark dataset spanning seven growing seasons (2019-2025). SwissCrop25 combines Sentinel-2 time series, daily temperature observations, a fine-grained 73 crop taxonomy including grassland management types, and 5 explicit non-crop land cover classes. To evaluate realistic deployment conditions, we define a leave-one-year-out protocol with joint cropland delineation and crop classification for benchmarking representative crop mapping architectures. Evaluating U-TAE (convolutional temporal-attention model), TSViT (transformer-based spatio-temporal model), and Galileo (EO foundation model) reveals differences between architectures hidden by conventional benchmarks. In this setting, domain-specific models outperform Galileo, with TSViT achieving the best overall performance and a 12 pp macro-mIoU advantage over U-TAE. SwissCrop25 also exposes substantial interannual distribution shifts and shows that incorporating temperature-derived phenological information improves robustness. Finally, in-season evaluation reveals a trade-off between models, with U-TAE performing better early in the season and TSViT gaining an advantage later through improved rare-class discrimination. SwissCrop25 provides a challenging testbed for evaluating crop mapping systems under realistic operational conditions and is publicly released at https://huggingface.co/datasets/EOA-team/SwissCrop25 .

Thomas Lauber, Mehmet Ozgur Turkoglu, Sélène Ledain +1

Earth Observation of Agroecosystems Team, Agroscope, Switzerland

Agricultural Monitoring Land Cover Season Taxonomy

Ask PDF

Aug 1, 2026cs.CV

PhenoStitch: Training-Free Panoptic Crop Mapping from Satellite Image Time Series

Panoptic crop mapping requires both delineating individual agricultural parcels and assigning a crop type to each parcel from satellite image time series. Existing approaches typically rely on dense parcel-level annotations and task-specific model training, which limits their applicability to new regions and growing seasons. We introduce PhenoStitch, a panoptic crop-mapping pipeline that requires no task-specific gradient-based training. A frozen Segment Anything model first oversegments each patch into class-agnostic regions. For each region, optical NDVI and Sentinel-1 backscatter series are summarized by an analytic double-harmonic phenological signature. Adjacent regions are then merged into parcels by minimizing a Potts graph energy, and each parcel is classified by nearest-prototype matching using only (k) labeled parcels per class. A final topology-closure step produces the panoptic map. Under a matched budget of (k=20) parcels per class, corresponding to less than 1% of the available labels, PhenoStitch achieves 20.0 crop mIoU, 76.2 segmentation quality, and 6.2 panoptic quality on PASTIS-R under a 5-fold, 3-seed evaluation. It outperforms the evaluated frozen foundation-model, few-shot, and matched-budget supervised baselines under the same protocol, with a consistent ranking also observed on ZueriCrop. Ablation studies show that radar observations contribute the largest performance gain, while the graph-energy merge and compact phenological signature provide further improvements. These results demonstrate the effectiveness of combining label-free parcel delineation with few-shot phenological recognition for panoptic crop mapping under limited supervision.

Xuechen Li

University of Minnesota Minneapolis, MN, USA

Agricultural Monitoring Panoptic Segmentation Satellite Imagery Delineation

Ask PDF