Organizations: The National School of Artificial Intelligence (ENSIA), Algiers, 16046, Algeria · National Higher School of Agronomy (ENSA), Algiers, 16051, Algeria · National Higher School of Computer Science (ESI), Algiers, 16000, Algeria · National Higher School of Advanced Technologies (ENSTA), Algiers, 16000, Algeria
Machine learning models in agricultural vision often achieve high accuracy on curated datasets but fail to generalize under real field conditions due to distribution shifts between training and deployment environments. Moreover, most machine learning competitions focus primarily on model design while treating datasets as fixed resources, leaving the role of data collection practices in model generalization largely unexplored. We introduce the AgrI Challenge, a data-centric competition framework in which multiple teams independently collect field datasets, producing a heterogeneous multi-source benchmark that reflects realistic variability in acquisition conditions. To systematically evaluate cross-domain generalization across independently collected datasets, we propose Cross-Team Validation (CTV), an evaluation paradigm that treats each team's dataset as a distinct domain. CTV includes two complementary protocols: Train-on-One-Team-Only (TOTO), which measures single-source generalization, and Leave-One-Team-Out (LOTO), which evaluates collaborative multi-source training. Experiments reveal substantial generalization gaps under single-source training: models achieve near-perfect validation accuracy yet exhibit validation-test gaps of up to 16.20% (DenseNet121) and 11.37% (Swin Transformer) when evaluated on datasets collected by other teams. In contrast, collaborative multi-source training dramatically improves robustness, reducing the gap to 2.82% and 1.78%, respectively. The challenge also produced a publicly available dataset of 50,673 field images of six tree species collected by twelve independent teams, providing a diverse benchmark for studying domain shift and data-centric learning in agricultural vision.
Compound, pinnate; 7–11 leaflets; foliage turns red in autumn
Smooth grey-green trunk; dense, compact crown
Table 3: Visual and morphological properties of the six target species.
Figure 2: Samples from the AgrI Challenge dataset across the six target tree species.
Team
Carob tree
Oak
Peruvian pepper tree
Ash
Pistachio tree
Tipu tree
Total
AI-4o
960
1625
1221
934
1111
1493
7344
AiGro
686
604
595
572
541
635
3633
CACTUS
400
597
694
803
755
552
3801
CHAJARA
610
549
428
308
262
383
2540
GreenAI
675
705
609
559
722
515
3785
PLT
1089
1131
1000
1297
973
1066
6556
Table 4: Distribution of images across teams and tree species classes
Figure 3: Histogram showing the distribution of dataset images by capture device model. The dataset contains images from more than 40 device types, with the largest groups originating from iPhone 11 and Oppo Reno5, and a substantial portion labeled as Unknown due to missing metadata.
Model
Type
Parameters
Purpose
DenseNet121
CNN
8M
Efficient backbone suitable for resource-constrained scenarios and strong feature reuse.
Swin-Tiny
Transformer
28M
Lightweight transformer baseline capturing both local and global contextual information through window-based attention.
Table 5: Baseline model architectures used for comparison.
Parameter
Value
Hardware
NVIDIA H100 NVL (22 GB)
Pre-training
ImageNet-1K
Optimizer
AdamW
Weight decay
10−4
Initial learning rate
10−4
Learning rate schedule
Cosine annealing ( ηmin=10−6 )
Table 6: Training configuration used for all experiments.
Team
DenseNet121
Swin Transformer
Val Acc.
Test Acc.
VTG
Val Acc.
Test Acc.
VTG
AI-4o
96.60
86.94
9.65
98.09
91.08
7.02
AiGro
98.99
82.32
16.67
99.27
87.68
11.58
CACTUS
96.84
84.36
12.49
98.86
90.19
8.67
CHAJARA
97.51
81.56
15.94
98.82
88.03
10.79
GreenAI
96.92
85.47
11.45
98.42
90.86
7.55
Table 7: Controlled TOTO evaluation with standardized architectures
Figure 4: Cross-team test accuracy for DenseNet121 and Swin Transformer in TOTO protocol. Horizontal dashed lines indicate means: 81.19% (DenseNet) and 87.21% (Swin).
Figure 5: Global model performance under the TOTO protocol for DenseNet121 and Swin Transformer (mean ± std across 12 runs). The large validation–test gap highlights the cross-team generalization challenge under single-team training.
Figure 6: Aggregate learning curves for TOTO protocol ( n=12 runs, mean ± std). Left: Swin Transformer. Right: DenseNet121. The gap between validation and test accuracy persists across all epochs.
Figure 7: Distribution of validation–test gaps under the TOTO protocol for Swin Transformer and DenseNet121. The red dashed line marks zero gap. Larger gaps indicate stronger cross-team generalization loss.
Figure 8: DenseNet121 cross-team accuracy matrix. Rows: training teams; columns: test teams. Range: 48.2%–95.3%.
Table 8: LOTO evaluation using standardized DenseNet121 and Swin Transformer architectures.
Figure 10: Distribution of validation-test gaps under LOTO protocol for Swin Transformer and DenseNet121. The red dashed line marks zero gap. Two teams achieved negative gaps in both models, indicating the held-out data was easier to generalize to than the validation set. The single outlier (DenseNet, Condimenteum: +10.76%) reflects that team’s distinctive collection methodology.
Figure 11: Global model performance under LOTO protocol for DenseNet121 and Swin Transformer (mean ± std across 12 folds). The compressed train-validation-test spread contrasts sharply with the large gaps observed under TOTO.
Figure 12: Aggregate LOTO learning curves. Left: Swin Transformer (mean final test: 97.04%). Right: DenseNet121 (mean final test: 95.31%).
Figure 13: Per-held-out-team test accuracy under LOTO protocol for Swin Transformer (left) and DenseNet121 (right). Dashed blue lines indicate overall means (97.04% and 95.31%). The consistency of team rankings across both architectures (Spearman ρ=0.92 ) confirms that residual variation reflects dataset-level properties rather than architecture-specific effects.