The role of spatial context and multitask learning in the detection of organic and conventional farming systems based on Sentinel-2 time series
Authors: Jan Hemmerling, Marcel Schwieder, Philippe Rufin, Leon-Friedrich Thomas, Mirela Tulbure, Patrick Hostert, Stefan Erasmi
Organizations: Thünen Earth Observation (ThEO), Thünen Institute of Farm Economics, Braunschweig, Germany. · Earth Observation Lab, Geography Department, Humboldt-Universität zu Berlin, Unter den Linden 6, 10099 Berlin, Germany. · Earth and Life Institute, UCLouvain, 1348 Louvain-la-Neuve, Belgium. · Department of Agricultural Sciences, University of Helsinki, P.O. Box 28, FI-00014 Helsinki, Finland. · Center for Geospatial Analytics, North Carolina State University, NC 27695 Raleigh, USA.
Organic farming is a key element in achieving more sustainable agriculture. For a better understanding of the development and impact of organic farming, comprehensive, spatially explicit information is needed. This study presents an approach for the discrimination of organic and conventional farming systems using intra-annual Sentinel-2 time series. In addition, it examines two factors influencing this discrimination: the joint learning of crop type information in a concurrent task and the role of spatial context. A Vision Transformer model based on the Temporo-Spatial Vision Transformer (TSViT) architecture was used to construct a classification model for the two farming systems. The model was extended for simultaneous learning of the crop type, creating a multitask learning setting. By varying the patch size presented to the model, we tested the influence of spatial context on the classification accuracy of both tasks. We show that discrimination between organic and conventional farming systems using multispectral remote sensing data is feasible. However, classification performance varies substantially across crop types. For several crops, such as winter rye, winter wheat, and winter oat, F1 scores of 0.8 or higher can be achieved. In contrast, other agricultural land use classes, such as permanent grassland, orchards, grapevines, and hops, cannot be reliably distinguished, with F1 scores for the organic management class of 0.4 or lower. Joint learning of farming system and crop type provides only limited additional benefits over single-task learning. In contrast, incorporating wider spatial context improves the performance of both farming system and crop type classification. Overall, we demonstrate that a classification of agricultural farming systems is possible in a diverse agricultural region using multispectral remote sensing data.
Figures & tables
Figure 1: Location of our study area in Germany, Zoom-in (1): grid system with 30 × 30 km tiles, indicated are training, validation and test tiles; Zoom-in (2): Subdivision of tiles into 300 × 300 m patches (30 × 30 pixel); outlined are patches with more than 15% agricultural land as used for training and validation, further subdivision of the patches into 100 × 100 m (10 × 10 pixel) and 20× 20 m (2 × 2 pixel), Background: Google Earth
Crop type
Area[ha]
Share[%]
Org,[%]
Winter wheat
34336.24
19.05
7.98
Winter barley
17152.44
9.52
1.71
Winter rye
3442.15
1.91
18.48
Triticale
6453.90
3.58
9.35
Spring barley
6110.98
3.39
11.70
Spring oat
1785.68
0.99
46.92
Table 1: Total area, share of agricultural area and area under organic management per crop type in the study area
Tiles [n]
Patches [n]
Training Data
37
218815
Validation Data
4
26079
Test Data
8
80000
Table 2: Data Distribution of Data Sets
Figure 2: Overview of the adapted Temporo-Spatial Vision Transformer (TSViT), Pixel are represented by grey grid lines, colored boxes represent sub-patches (1) Input patches are arranged into n sub-patches (PP) of size S. For each of the 36 time steps (T) 10 bands (B) are embedded into tokens of length d (2) In the temporal transformer stage, tokens interact across T and the concatenated class tokens (3) In the spatial transformer stage, class tokens interact only with their corresponding spatial class token. Final predictions for crop type and management system are produced via separate argmax layers.
Figure 3: Mean F1-score derived from validation data set during model training after each epoch.
30 x 30
10 x 10
2 x 2
1 x 1
Multitask
Singletask
Multitask
Singletask
Multitask
Singletask
RF
Crop Types
F1
Re.
Pr .
F1
Re.
Pr.
F1
Re.
Pr.
F1
Re.
Pr.
F1
Re.
Pr.
F1
Re.
Pr.
F1
Re.
Pr.
Winter wheat
0.94
0.94
0.95
0.95
0.94
0.95
0.94
0.94
0.94
0.94
0.94
0.94
0.93
0.93
0.94
0.92
0.92
0.93
0.80
0.72
0.90
Winter barley
0.95
0.95
0.96
0.96
0.95
0.96
0.95
0.95
0.95
0.95
0.95
0.95
0.95
0.94
0.95
0.94
0.93
0.94
0.79
0.75
0.84
Winter rye
0.83
0.86
0.81
0.84
0.85
0.83
0.81
0.81
0.81
0.82
0.84
0.80
0.80
0.84
0.77
0.78
0.79
0.76
0.20
0.40
0.13
Triticale
0.83
0.82
0.83
0.84
0.83
0.86
0.81
0.82
0.81
0.81
0.83
0.79
0.79
0.80
0.79
0.77
0.79
0.75
0.40
0.58
0.31
Table 3: Classification results, F1-scores, Recall and Precision values for single crop types
30 x 30
10 x 10
2 x 2
1 x 1
Multitask
Singletask
Multitask
Singletask
Multitask
Singletask
RF
Management
F1
Re.
Pr.
F1
Re.
Pr.
F1
Re.
Pr.
F1
Re.
Pr.
F1
Re.
Pr.
F1
Re.
Pr.
F1
Re.
Pr.
Conventional
0.93
0.91
0.95
0.93
0.91
0.95
0.93
0.92
0.95
0.93
0.92
0.94
0.93
0.91
0.95
0.92
0.91
0.94
0.79
0.93
0.69
Organic
0.61
0.69
0.55
0.59
0.64
0.54
0.59
0.67
0.52
0.58
0.65
0.52
0.55
0.70
0.46
0.53
0.63
0.45
0.25
0.15
0.74
Background
0.96
0.97
0.95
0.96
0.97
0.95
0.96
0.97
0.96
0.96
0.97
0.96
0.96
0.97
0.95
0.96
0.96
0.95
0.92
0.96
0.89
Mean F1
0.83
0.86
0.81
0.82
0.84
0.81
0.83
0.85
0.81
0.82
0.85
0.81
0.81
0.86
0.79
0.80
0.83
0.78
0.65
0.68
0.77
Table 4: Accuracy measures for management system classification
Figure 4: Crop type and management system predictions from model runs with different spatial input dimensions (30 x 30, 10 x 10, 2 x 2 pixel, as described in 2.3, IACS data as reference, Random Forest model predictions as described in 2.8)
Figure 5: Crop type and management form predictions from model runs with different spatial input dimensions (30 x 30, 10 x 10, 2 x 2 pixel, as described in 2.3, IACS data as reference, Random Forest model predictions as described in 2.8)
Training Data
Validation Data
Test Data
Crop type
Area[ha]
Share[%]
Org.[%]
Area[ha]
Share[%]
Org.[%]
Area[ha]
Share[%]
Org.[%]
Winter wheat
26274.5
13.42
8.32
2159.2
8.78
6.86
5902.5
8.2
6.86
Winter barley
12470.7
6.37
1.85
1476.8
6
1.01
3204.9
4.45
1.51
Winter rye
2198.8
1.12
21.27
466.7
1.9
10.59
776.7
1.08
15.32
Triticale
4168.2
2.13
10.82
859.7
3.49
5.84
1426
1.98
7.2
Spring barley
4851.5
2.48
11.85
494.6
2.01
10.23
764.9
1.06
11.73
Table Appendix A1: Data distribution of Train, Validation and Test data set, share relative to total area in respective data set
Crop Type
Samples [n]
Winter wheat
97533
Winter barley
53350
Winter rye
11223
Triticale
20706
Spring barley
24089
Spring oat
8325
Table Appendix A2: Sample Distribution used for Random Forest Crop Type Model
Managemnt Form
Samples [n]
Conventional
617042
Organic
66792
Background
19721
Table Appendix A3: Sample Distribution used for Random Forest Management Form Model
30 x 30
10 x 10
2 x 2
1 x 1
Multitask
Singletask
Multitask
Singletask
Multitask
Singletask
RF
Crop Types
F1
Re.
Pr .
F1
Re.
Pr.
F1
Re.
Pr.
F1
Re.
Pr.
F1
Re.
Pr.
F1
Re.
Pr.
F1
Re.
Pr.
Winter wheat
0.94
0.94
0.95
0.95
0.94
0.95
0.94
0.94
0.94
0.94
0.94
0.94
0.93
0.93
0.94
0.92
0.92
0.93
0.80
0.72
0.90
Winter barley
0.95
0.95
0.96
0.96
0.95
0.96
0.95
0.95
0.95
0.95
0.95
0.95
0.95
0.94
0.95
0.94
0.93
0.94
0.79
0.75
0.84
Winter rye
0.83
0.86
0.81
0.84
0.85
0.83
0.81
0.81
0.81
0.82
0.84
0.80
0.80
0.84
0.77
0.78
0.79
0.76
0.20
0.40
0.13
Triticale
0.83
0.82
0.83
0.84
0.83
0.86
0.81
0.82
0.81
0.81
0.83
0.79
0.79
0.80
0.79
0.77
0.79
0.75
0.40
0.58
0.31
Table Appendix A4: Classification results, F1-scores, Recall and Precision values for single crop types
Crop recommendation systems in precision agriculture have long suffered from a fundamental modality gap: visual soil characterization and chemical nutrient profiling are typically treated as independent inference problems, with fusion often reduced to late-stage feature concatenation. AgroSense2.0 addresses this limitation through three architectural advances. First, we introduce continental-scale geospatial integration via a seven-band soil raster (india\_soil\_7bands.tif) spanning India, encoding Nitrogen, pH, SOC, Clay, Sand, Silt, and Bulk Density as 32×32 spatial patches, a modality entirely absent from prior work. Second, we replace naive feature concatenation with a cross-modal Transformer fusion module, where tabular nutrient features attend over image representations via multi-head attention, enabling richer inter-modal dependency modeling than shallow fusion. Third, we adopt a multi-task objective jointly optimizing soil classification and crop recommendation through a shared backbone, improving generalization via complementary cross-task signal. To enhance interpretability, we apply TreeSHAP to the tabular branch, revealing crop-conditioned nutrient sensitivity: humidity and rainfall emerge as the most influential features globally, while crop-specific profiles diverge meaningfully rainfall dominates rice, nitrogen and potassium dominate maize, and humidity and nitrogen dominate coffee. These explanations provide transparency into model decisions and surface both agronomically consistent patterns and dataset-specific divergences worth further study. Together, these contributions establish AgroSense2.0 as a more principled, interpretable, and geospatially grounded framework for precision agriculture.
Vishal Pandey, Rishav Tewari, Ruzina Haque Laskar
Research Engineer London, UK · Independent Researcher Kolkata, IN · Scientist-B Centre for Development of Telematics Delhi, IN
This work presents a configurable pipeline for generating semantic-segmentation-ready agricultural datasets from Sentinel-2 imagery and EuroCrops parcel-level annotations. The workflow transforms heterogeneous vector crop annotations into aligned multispectral image--mask pairs through label harmonization, Sentinel-2 product selection, spatial alignment, rasterization, patch extraction, quality filtering, and class-aware sample selection. The generated dataset contains 67,337 patches from five European countries and uses a reduced taxonomy of ten crop classes plus background. A four-level U-Net with Group Normalization was trained using 10 Sentinel-2 spectral bands and a composite loss combining class-weighted cross-entropy and Dice loss. On the internal EuroCrops-based test split, the model achieved a mean Intersection over Union (mIoU) of 0.7665, a pixel accuracy of 0.8693, and a mean class accuracy of 0.9072. Compared with spectral and spatial-context Random Forest baselines, the U-Net showed the importance of learned multi-scale spatial representations for crop segmentation. External evaluation was performed on unseen Belgian EuroCrops subsets, DACIA5, and PASTIS. The results show a clear performance gap under external and cross-dataset evaluation, especially for benchmarks with different taxonomies, annotation protocols, spatial coverage, or temporal organization. The model transfers more reliably to dominant and taxonomically aligned classes such as maize and wheat, while performance remains limited for several minority classes and for the adapted single-date PASTIS setting. These findings highlight both the potential and the limitations of using EuroCrops-derived supervision for Sentinel-2 crop segmentation under realistic domain shifts.
Alexandra Nicoleta Scarlat, Ioana Cristina Plajer, Alexandra Baicoianu
Transilvania University of Braşov, Faculty of Mathematics and Computer Science, Braşov, Romania
The landscape of satellite imagery time series datasets and boundary-pushing architectures for cropland segmentation has never been richer. However, in this gold rush, important truths are being missed on both fronts, as a drive for the most novel concepts or the largest datasets pushes finer details to the side. In this paper, we present our hybrid transformer-convolutional model, Cropland Parallel Attention and Refinement Network for Segmentation (PAtteRNS), the first model to use self-attention mechanisms separately for each of the temporal, spectral, and spatial aspects of Sentinel-2 multispectral SITS data. To achieve fully-factorised attention in our proposed model, we introduce a novel parallel transformer architecture which significantly reduces the computational complexity of triple-factorised self-attention. We validate our architecture with an in-depth ablation study, and analyse the performance of our model against state-of-the-art crop segmentation models on multiple tile-size variants of the popular PASTIS and MTLCC datasets. Our findings show our model to outperform all others in the task of crop class segmentation, verified across multiple important segmentation metrics, with especially strong performance against compared models seen in the often under-reported parcel delineation quality, for which we use the Boundary IoU metric. We also find that flawed class groupings within datasets can have a significant negative impact on model performance, and report that alternate tile-size variants of crop segmentation datasets produce results incomparable to one-another, invalidating fair comparison between model performance when trained on different tile-sizes. Based on these findings, we suggest further work is required to standardise best practices when constructing SITS crop segmentation datasets, and to enable future dynamic-tile-sizing for ideal model performance.
Joseph Metcalfe, Sara Sharifzadeh, Fabio Caraffini
Department of Computer Science, Swansea University, Swansea, SA1 8EN, United Kingdom