cs.CVOct 5, 2026
SaveEnvironmental sensor readings in two crop disease image datasets identify the session in which each image was taken
Organizations: Department of Electrical and Computer Engineering, Korea University Seoul, Republic of Korea
Abstract
Integrating environmental sensor data with leaf imagery is widely reported to boost crop disease classification accuracy. In this work, we reveal that these reported gains are often artifacts of dataset construction: because a single sensor reading is shared across many images collected in a single session (one farm on one date), multimodal networks can predict disease simply by memorizing session identities. Analyzing two widely used Korean datasets, the Crop Disease Diagnosis (CDD) benchmark and an AI Hub pest/disease dataset, we demonstrate that nearly all images share sensor values, with 91.9% of CDD test images having exact sensor duplicates in the training set. Remarkably, an image-free classifier given only timestamps matches or exceeds sensor-driven predictions across all seven evaluated crops, and matches the published macro-F1 of a state-of-the-art CDD fusion model. These results indicate that performance gains on standard random splits cannot be disentangled from session leakage. We propose that multimodal crop studies must evaluate on session-held-out splits and report performance against sensor-free date-time baselines to ensure genuine generalization.
Figures & tables
Figure 1: How one sensor reading is attached to many images, and the two parts of the diagnostic. (A) A schematic of two sessions, each one farm on one day. Each sensor reading (diamond) is attached to every image (square) taken after it until the next reading, so the images of a session share far fewer readings than there are images. In the schematic, every image of a session carries that session’s disease class. (B) The two parts of the diagnostic. The granularity probe counts the images and the sessions that share each reading and measures the share of each variable’s variance that farm identity explains. The timestamp baseline compares a classifier given only the timestamp with one given the sensor readings. Neither part reads an image.
| Setting | Value |
|---|---|
| Timestamp baseline and CDD re-analysis | |
| Random forest | 60 trees |
| AI Hub rows drawn per crop and seed | 20,000 |
| AI Hub cross-validation | stratified, 3 folds |
| AI Hub seeds | 7 |
| CDD test partition (drawn splits) | one of 7 folds |
Table 1: Settings of the classifiers and of the fusion training. A random-forest setting that neither this table nor the text gives is the scikit-learn default, and a fusion-training setting that neither gives is the CDD study’s.
Figure 2: Farm identity explains more than half the variance of every panel variable of the AI Hub dataset except air humidity and electrical conductivity, and many CDD images share each environment series. (A) The share of each panel variable’s variance that farm identity explains on the AI Hub dataset. The dashed line marks one half. The five soil-chemistry variables that the panel classifier reads are shown in blue. (B) The CDD images grouped by how many images share their environment series. A series shared by one image is a per-image reading.
Figure 3: Timestamp classifier minus panel classifier on the AI Hub dataset, per crop, in accuracy and in balanced accuracy. Markers are means over 7 random seeds, and bars are 95% confidence intervals; a marker hides an interval narrower than itself. The shaded region is the comparable band, within which the timestamp classifier is comparable to the panel classifier. Values are positive where the timestamp classifier is the more accurate.
| Accuracy | Balanced accuracy | |
| Majority class, crop mean | ||
| Panel, crop mean | ||
| Timestamp, crop mean | ||
| Timestamp margin per crop | ||
| Fig | ||
| Apple mango | ||
Table 2: Timestamp classifier versus panel classifier at disease-class prediction on the AI Hub dataset. Values are means over 7 random seeds with 95% confidence intervals, and the first three rows are also means over crops. The per-crop rows give the timestamp classifier’s margin, its score minus the panel classifier’s in the column’s metric, for the two crops in which the timestamp classifier is superior and for the one crop with a negative margin. Figure 3 shows the margins of all 7 crops.
| Random split | Group-disjoint split | |
| Macro-F1 | ||
| Majority class | ||
| Environment | ||
| Timestamp | ||
| Timestamp and environment | ||
| Timestamp and environment minus timestamp | ||
Table 3: Macro-F1 and balanced accuracy of the CDD predictors on the two splits that we draw over the whole dataset, one random and one disjoint on the environment series, each over 7 random seeds with 95% confidence intervals. Each row is a random forest on the features that the row names, or the constant majority-class predictor. No row uses the images or the study’s split. Paired differences are taken within seed. The leak rate is the share of test images whose environment series recurs in training.
| The study’s split | The other set | |
| Macro-F1 | ||
| Majority class | ||
| Environment | ||
| Timestamp | ||
| Timestamp and environment | ||
| Timestamp minus timestamp and environment | ||
Table 4: The predictors of Table 3 scored on the two sets of identifiers in the CDD study’s repository that hold a test partition: the study’s split, from which its training script reads the training, validation, and test partitions, and the other set. Each split is fixed, so the 7 random seeds behind each interval vary the forest alone, just as the runs behind the intervals that the study reports vary only the random seed of its training. Values are given to four decimals, the precision of the study’s reported figures. The paired difference is taken within seed. No row uses the images.
Figure 4: Macro-F1 of three CDD predictors on the random and group-disjoint splits that we draw and on the study’s own split. Markers are means and bars are 95% confidence intervals. The vertical lines mark the macro-F1 that the study reports for its image model and for its fusion model on its own split. The majority-class predictor is omitted; Tables 3 and 4 give its scores.
| Split and budget | Image-only macro-F1 | 95% CI | |
|---|---|---|---|
| Macro-F1 | |||
| In distribution, reduced budget | |||
| Date-disjoint, reduced budget | |||
| Farm-disjoint, reduced budget | |||
| In distribution, matched budget | |||
| Date-disjoint, matched budget | |||
Table 5: Late-fusion gain on the AI Hub dataset in macro-F1 and in balanced accuracy, the fusion model’s score minus the image-only model’s, with the image-only model’s macro-F1 beside it. Values are means over 5 random seeds, with 95% confidence intervals for . The reduction is the in-distribution gain minus the date-disjoint gain, paired within seed. The farm-disjoint test fold falls below the coverage floor of Section 2.4 .