cs.CVOct 16, 2026

A100: A Hybrid Vision Foundation Model for Agricultural Plant Image Analysis

Authors: Mayur Pratim Das, Ajai Dubey, Aksshay Mathew P, Sana Afreen, Rishi Kumar Soni, Deepak Meena, Pratibha Kumari, Mukesh Saini

Organizations: Annam.AI, IIT Ropar Rupnagar, India · IIT Ropar Rupnagar, India · Dept. of DSE, IISER Bhopal Bhopal, India · Dept. of CSE, IIT Ropar Rupnagar, India

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

The use of accurate vision models for crop monitoring, disease diagnosis, and plant phenotyping in agriculture is hindered by the lack of expertly annotated data, especially in developing countries. We propose A100, an agricultural vision foundation model that is pre-trained using self-supervised learning on 3 million unlabeled images from fields covering more than 100 crop species. A100 uses a hybrid CNN-Transformer backbone that fuses MBConv stages for local feature extraction with Transformer stages for global contextual representation. A100 uses pre-training with the integration of DINO self-distillation, iBOT masked patch reconstruction, and KoLeo uniformity regularization, resulting in features that are better than ImageNet-12K supervised representations for agricultural applications without any fine-tuning. By utilizing a frozen backbone, the accuracy for crop classification of A100 is 95.37%, which is +2.56% better than ImageNet-12K, and reduces the leaf count error by 10.4% on the GroMo25 phenotyping benchmark, proving that the same backbone can be used for classification and regression tasks. After full fine-tuning, the performance of A100 is 99.90% on 32-class crop identification and 99.02% on 44-class disease detection. On the difficult PlantDoc dataset, A100 outperforms all baselines, such as CvT-13 (64.89% against 63.56%). Remarkably, A100 obtains 98.01% accuracy on plant classification using just 1% of the labelled data (about 284 images per category), showing great label efficiency for use cases with limited labeling. Under an identical protocol, A100 also outperforms the publicly released agricultural foundation models FoMo4Wheat and LeafVision by more than 14 points at 1% labels. We plan to publicly release our pretrained model to allow agricultural professionals to adapt A100 for various plant analysis applications using limited annotations.

Figures & tables

Figure 1. Deployment on downstream tasks for Crop Identification, Disease Detection, and Plant Phenotyping (Leaf Counting)
Figure 2. Representative samples from the datasets used for (a) self-supervised pretraining, (b) crop classification, (c) disease detection, and (d) plant phenotyping.
Pretraining CorpusCrop IdentificationDisease Detection
( ∼{\sim} 3,000,000 Images, 100 Crops)(1,142,326 Images, 32 Classes)(51,994 Images, 44 Classes)
SourceImagesCrop ClassImagesCrop–DiseaseImages
Proprietary On-Field Collection    
Field agents (100 crops, multi-state)1,367,927Wheat123,631Chilli–Leaf Curl2,608
Farm survey imagery (diverse fields)110,538Sugarcane84,127Cotton–Healthy4,869
  Gram77,779Rice–Brown Spot3,721
Table 1. Composition of the pretraining corpus and downstream fine-tuning datasets.
Figure 3. A100 architecture overview: (a) CNN-Transformer hybrid backbone with MBConv stages (S0–S1) and Transformer stages (S2–S3), (b) SADG pretraining framework with three self-supervised objectives. A block diagram showing the A100 architecture. The left panel shows the CNN-Transformer hybrid backbone processing an input image through stem, MBConv, and Transformer stages. The right panel shows the SADG self-supervised pretraining framework utilizing Student and Teacher networks with DINO, iBOT, and KoLeo objectives.
 DINO head hDh_{D}iBOT head hIh_{I}
InputGAP(S3), 1024-dS2, 512×14×14512{\times}14{\times}14
Appliedper imageper spatial position
MLP1024 →\to 2048 →\to 2048 →\to 256512 →\to 2048 →\to 2048 →\to 256
PrototypesK=65,536K=65{,}536K′=8,192K^{\prime}=8{,}192
Teacher norm.Sinkhorn-KnoppSinkhorn-Knopp
Parameters≈{\approx} 23.6M≈{\approx} 7.9M
Table 2. SADG Projection Heads
Evaluation ProtocolMethodCrop IdentificationDisease Detection
(data fraction) (32 cls)(44 cls)
Fine-tune (100%)A10099.9099.02
 ImageNet-12K99.9099.00
Fine-tune (10%)A10099.7295.94
 ImageNet-12K98.6994.82
Fine-tune (1%)A10098.0186.10
Table 3. A100 vs ImageNet-12K Baseline across evaluation protocols (Test Accuracy %)
Figure 4. A100 Performance Across Data Regimes. Label efficiency of A100 under limited supervision (1% and 10% labels) compared to the fully supervised (100%) and frozen-backbone (linear probe) setups on crop identification and disease detection.
ModelPlantVillagePlantDocPlantWild
 (38 cls)(28 cls)(89 cls)
A100 (Ours)99.9664.8976.96
Hybrid Baseline (IN-12K) ‡99.9660.5975.75
ResNet-101 (IN) †99.7157.6373.88
Swin-T (IN) †99.7359.7576.32
CvT-13 (IN) †99.7363.5677.31
Table 4. External Benchmark Performance (Test Accuracy %)
ModelPretraining dataParamsAcc. (%)Macro-F1 (%)
A100 (Ours)3M agri., 100+ crops73M98.0197.10 ±\pm 0.12 §
ImageNet-12K (same arch.)ImageNet-12K (sup.)73M95.20 ∗93.26 ±\pm 0.17
FoMo4Wheat ( Han et al., 2025 ) ViT-B/142.5M wheat87M83.86 ±\pm 0.2879.70 ±\pm 0.36
LeafVision ( Han et al., 2026 ) ResNet-50540K leaf24M79.25 ±\pm 0.7976.18 ±\pm 0.68
LeafVision ( Han et al., 2026 ) ViT-B/16540K leaf86M77.83 ±\pm 0.3874.77 ±\pm 0.35
∗ Re-run over three seeds with the same pipeline: 95.43 ±\pm 0.07.
Table 5. Comparison with publicly released agricultural foundation models on crop identification (32 classes, 114,260 test images) with 1% of the training labels (9,136 images).
Figure 5. Plant phenotyping (leaf counting): A100 vs ImageNet-12K. (Top) RMSE and (Bottom) MAE. A100 consistently reduces prediction error across all protocols, with the largest improvement at 1% labels (-16.4% RMSE, -14.0% MAE).
More in this topic

Agricultural Image Analysis

Agricultural image analysis uses computer vision and multimodal learning to monitor crops, livestock, produce quality, and field conditions.

See all 91 papers

Explore similar work

CardsList
Mar 29, 2026

SPROUT: A Scalable Diffusion Foundation Model for Multi-Crop Plant Phenotyping

Image-based plant phenotyping depends on dense structural understanding of crops, yet pixel-level annotation remains expensive across species, organs, growth stages, and field conditions. General-purpose vision foundation models offer a natural route to label efficiency, but their web-scale pretraining objectives transfer weakly to agricultural imagery, where semantics are often determined by fine organ geometry inside repetitive, texture-dominated scenes. We introduce SPROUT, a diffusion foundation model for multi-crop plant phenotyping. SPROUT learns from 2.6 million unlabeled open-field images (MCD-2.6M) using a pixel-space Diffusion Transformer, and selects transferable features with a label-free effective-rank criterion over denoising timesteps. This design shifts pretraining from crop-based invariance to structure-preserving denoising, making the representation better aligned with dense phenotyping tasks. We evaluate SPROUT across dense phenotyping tasks, including organ segmentation, crop-weed parsing, depth estimation, and counting. SPROUT consistently improves over strong web-pretrained baselines, with the largest gains on dense structural prediction, and shows favorable label and compute efficiency compared with general-purpose and crop-specific foundation models. The source code and MCD-2.6M dataset are publicly available.

Shuai Xiang, James Burridge, Shouyang Liu +4

Graduate School of Agricultural and Life Sciences, The University of Tokyo, 1-1-1 Midori-cho, Nishitokyo, Tokyo 188-0002, Japan · Engineering Research Center of Plant Phenotyping, Ministry of Education; Jiangsu Collaborative Innovation Center for Modern Crop Production; Academy for Advanced Interdisciplinary Studies, Nanjing Agricultural University, Nanjing 210095, China · National Key Laboratory of Multispectral Information Intelligent Processing Technology, School of Artificial Intelligence and Automation, Huazhong University of Science and Technology, Wuhan 430074, China +3

Agricultural Image AnalysisPlant PhenotypingRepresentation LearningVision Foundation ModelsSemantic Segmentation
AskPDF
Sep 8, 2026

Vision-language models see more about agriculture than they report, and test-time scaling remains bounded by parametric knowledge

Vision-language models (VLMs) show promise for agricultural diagnosis but fail unpredictably in zero-shot settings. We test whether this stems from poor visual features or an inability to verbalize what the model already perceives, using a benchmark spanning classification and detection drawn from 118 AgML classification and 54 detection datasets. Linear probing shows agricultural features are linearly separable in frozen VLM vision towers, with Gemma-4-31B's encoder reaching F1-macro of 0.60--0.78 across tasks and approaching the vision-only DINOv3 backbone, indicating the bottleneck lies downstream of perception. We then test whether test-time scaling, an iterative in-context skill-memory loop, can close this gap without updating weights. Gains are modest and inconsistent. Skill-memory improves classification accuracy by up to 3.1 points on three of four tasks while regressing on species, with gains concentrated on classes seen during optimization and unseen classes regressing. Detection F1 is effectively unchanged (0.412 to 0.421), though an oracle filtering condition reaches 0.432 F1, revealing real headroom unconstrained prompting fails to surface. These results indicate VLMs see more agricultural features than they report, but test-time scaling alone fails to surface that hidden parametric knowledge, motivating agentic approaches grounded in external, retrievable knowledge.

Earl Ranario, Jared Smith, Lars Lundqvist +3

University of California, Davis

Agricultural Image AnalysisPlant Disease ClassificationVLM ReasoningClassificationRubric-Based EvaluationVLM Calibration
AskPDF
May 10, 2026

SAGE: Scalable Agentic Grounded Evaluation for Crop Disease Diagnosis

Plant disease diagnosis is critical for food security, yet training disease-recognition models that generalize across crops, pathogens, and field conditions remains challenging because labeled disease images are far less abundant and standardized than data for other biotic stresses such as insects or weeds. Frontier vision-language models offer new opportunities through improved visual reasoning, but they still struggle with fine-grained disease identification due to the lack of structured, crop-specific symptom knowledge. To address this gap, we curate the largest plant disease image--symptom dataset to date, covering 335 crops, 1{,}251 disease classes, and approximately 839K images, designed to support training-free, agentic disease prediction. A scalable automated pipeline generates source-grounded symptom descriptions in which each claim is linked to a verbatim web quote; domain experts validate sampled crops and reconcile disease-name variants across sources. As a baseline, we introduce an autonomous visual reasoning agent that identifies anatomical context, narrows candidate diseases using symptom knowledge, sequentially compares reference images, and produces a fully explainable reasoning trace. Incorporating symptom knowledge improves accuracy by 16.2 percentage points on average at the full reference budget, with consistent gains across all four evaluation crops. Because the framework only requires crop-specific reference images and symptom knowledge, it can be extended to new crops without retraining, while the agentic baseline can directly benefit from future improvements in foundation model capabilities. Dataset and code are available at:https://sage-dataset.github.io/.

Muhammad Arbab Arshad, Tirtho Roy, Yanben Shen +7

Iowa State University, USA · New York University, USA

VLM EvaluationAgricultural Image AnalysisPlant Disease Classification
AskPDF