PhenoIntel: A Lifecycle-Aligned Multi-Agent Web Application for Verified, Accessible Plant Phenotype Analysis

Narendren S; Soumyashree Kar

cs.AIAug 28, 2026

PhenoIntel: A Lifecycle-Aligned Multi-Agent Web Application for Verified, Accessible Plant Phenotype Analysis

Authors: Narendren S, Soumyashree Kar

Organizations: VIT Bhopal University Bhopal, Madhya Pradesh, India · Centre of Studies in Resources Engineering Indian Institute of Technology Bombay Mumbai, Maharashtra, India

Abstract

Existing conversational plant-phenotyping platforms are difficult for plant scientists to use and lack the reliability scientific research demands: failed analyses are reported as valid measurements rather than flagged as missing, statistical tests run without checking assumptions, predictions carry no uncertainty estimate, and specialised hardware limits accessibility. We present PhenoIntel, a lifecycle-aligned multi-agent web platform that turns the full machine-learning workflow into a reliable, user-friendly phenotyping system. Nine specialised agents divide the analysis into stages, from image collection through model selection, inference, and reporting, rather than handing the whole task to one AI manager. Independent checks separate these stages, and every agent reads from and writes to one shared, fixed-structure record, so an inconsistent output from one stage is caught before it reaches the next. Uncertainty is matched to each model family, conformal prediction, detection-confidence spread, or Monte Carlo Dropout, rather than applied uniformly, and quality thresholds adapt to crop and task instead of one global cutoff. When no suitable model exists, PhenoIntel can propose, validate, and integrate a new one on its own. The model repository spans ten trained models across five crops and four imaging modalities. Classification models reach Macro F1 of 0.78-0.996; object-detection models reach 0.96 mAP@50 with a 54% reduction in counting error over an unoptimised baseline; and a temporal model reaches held-out Macro F1 of 0.7050. PhenoIntel runs in a browser on standard hardware, requiring no GPU, and a 1,200-test automated suite confirms complete pipeline execution. Every result carries calibrated uncertainty, validated statistics, and FAIR-compliant provenance, a combination existing conversational phenotyping tools do not offer.

Figures & tables

Principle	Failure Category Addressed	Section
1. Lifecycle-aligned agent decomposition	Monolithic orchestrator conflates all stages; non-deterministic routing; $\sim$ 30% tool-selection error rate	Section 4.2
2. Typed state contracts (shared session state)	Free-text inter-tool communication; fragile filename hand-off (different segmentation backends save under different naming conventions); silent breakage on message format changes	Section 4.5
3. Inter-stage verification with bounded recovery	Detection failure recorded as a plain zero; corrupted values propagate silently into statistics	Section 4.5
4. Per-model uncertainty quantification	Single-point phenotype estimates; no confidence intervals; MC Dropout misapplied to architectures that disable dropout at inference time	Section 4.4
5. Biological plausibility validation	Implausible values propagate unchecked; exclusive-or mask combination underestimates overlapping leaf area	Section 4.4
6. Sandboxed statistical execution	Model-generated code runs unsandboxed on local machine with container isolation disabled; no filesystem restriction, no timeout, no import allowlist	Section 4.6

Table 1: Seven architectural principles of PhenoIntel and the failure categories from prior monolithic phenotyping orchestrators each addresses.

Crop	Task / Modality	Size	Source
Rice	Nitrogen severity / RGB close-range	$\sim$ 5,790	Author-processed (Kaggle), kaggle.com/datasets/narendrensv/pheno-assistant
Wheat	Nutrient deficiency / UAV aerial	$\sim$ 3,000	( Yi et al.,, 2025 ) , DND-Diko-WWWR, github.com/jh-yi/DND-Diko-WWWR
Maize	Nutrient deficiency / RGB close-range	$\sim$ 17,000	Author-processed (Kaggle), kaggle.com/datasets/narendrensv/pheno-assistant
Banana	Micronutrient deficiency / RGB	$\sim$ 1,500	Public (Kaggle), kaggle.com/datasets/warcoder/nutrient-deficient-banana-plant-leaves
Coffee	Nutrient deficiency / RGB close-range	$\sim$ 800	Author-processed CoLeaf (Kaggle), kaggle.com/datasets/narendrensv/coleaf
Rice (panicle)	Detection/counting / UAV aerial	$\sim$ 1,800	Public (Kaggle), kaggle.com/datasets/rifat963/…bangladesh

Table 2: Datasets used across the PhenoIntel model zoo. Sizes and splits describe the complete dataset used for fine-tuning and held-out evaluation; the smaller, separately curated case-study input sets are reported in Table 7 .

Panel of Figure 2 — Figure 2: End-to-end methodology at a glance. Curation (Section 3 ) and background concepts (Section 4.1 ) are prerequisites shown outside this figure; every box here is described in full architectural detail from Section 4 onward.

Agent (Stage)	Owns Exclusively	Failure Mode Addressed
Planning (0)	Structured session plan from an LLM, offline keyword fallback	No formal planning stage; ad-hoc per-turn routing
Data Collection (1)	Image validation, capture metadata, pixel calibration	Local-file/URL only; no calibration validation
Preprocessing (2)	Normalisation, channel-adapter projection, NaN checks	Corrupted tensors passed silently to inference
Feature Engineering (3)	ViT features, spectral indices, temporal assembly	No explicit feature stage; ad hoc prompts
Dataset Partitioning (4, Build)	Stratified split, leakage/near-duplicate/mislabel detection	No partitioning validation; leakage undetected
Model Selection (5)	Registry look-up, four-tier fallback	Text-based selection, $\sim$ 30% error rate

Table 3: The nine lifecycle-aligned pipeline agents and the specific failure mode from existing monolithic orchestrators each one’s responsibility is designed to prevent.

Source	What it provides	Example models
Built-in zoo	Small, hand-vetted set already used by the fixed pipeline; no download or matching step needed.	Fixed detection and classification backbones
timm	Pretrained convolutional and vision-transformer checkpoints spanning hundreds of architectures, fetched by name.	ConvNeXt, EfficientNet, ViT variants
Hugging Face Hub	Any compatible transformers / timm -hosted checkpoint, identified by repository name.	Community and research vision checkpoints
torchvision	Standard reference implementations bundled with PyTorch’s vision library.	ResNet, MobileNetV3
Custom generation	A language model proposes a new architecture layer-by-layer when no external source fits; checked against the same structural rules and critique pass as any other candidate.	e.g. the CS5 single-stream ViT-S/16 five-head regression network

Table 4: Backbone sources available to the Pipeline Adaptation Agent. All five paths compile through the same schema validation and verifier before a model is registered.

Model Family	UQ Method	Interval Produced	Calibration Source
Classification	Conformal prediction (RAPS) ( Angelopoulos et al.,, 2021 )	Prediction set guaranteed to contain the true class at a chosen coverage	Held-out calibration slice reserved during curation (Section 3.2 )
Detection & zero-shot segmentation	Detection-confidence spread ( Celikkan et al.,, 2023 )	Count/area interval width scaled by per-image detection confidence	Detection-confidence distribution across the image (no separate held-out set)
Temporal modelling	Monte Carlo Dropout ( $N{=}30$ ) ( Gal & Ghahramani,, 2016 ) + quantile regression heads ( Romano et al.,, 2019 )	95% interval from dropout spread; P10–P90 from quantile heads directly	Dropout kept active at inference; quantile heads trained with a monotonicity penalty

Table 5: Uncertainty-quantification method dispatched per model family (Section 4.4 ), the interval it produces, and its calibration source.

Checkpoint	Stage Boundary	Key Checks	Failure Action	Mutates State
V0 Pre-flight	Before the data-collection stage	Valid (crop, task, modality) triple; $\geq$ 1 image path; basic pixel content	Route to error before compute time is consumed	No
V1 Mass Integrity	After the data-collection stage	$\geq$ 1 validated image path; all files exist on disk; calibration mode valid; minimum resolution per modality	Route to recovery	No
V2 Normalisation	After Preprocessing	Preprocessed tensor count = image count; no tensor NaN; pixel values within expected normalised range	Route to recovery	No
V3 Feature Validity	After Feature Engineering	NDVI $\in[-1,\,1]$ ; temporal array shape $=(40,10,32,32)$ for the temporal model; spectral band ranges valid	Route to recovery	No
V4 Model Compatibility	After the model-selection stage	A model card was actually selected; modality match; checkpoint file exists (skipped for zero-shot/generated cards)	Route to recovery	No
V5 Post-Inference	After Inference	Detection status explicitly set, never left as "not run"; "no detections" pre-fills as missing (not a literal zero); a "success" status with empty output is a hard error	Missing-value pre-fill; Route to recovery	Yes

Table 6: PhenoIntel verification checkpoints (V0–V7): stage, check summary, failure action, and whether the checkpoint can mutate state.

Case Study	Mechanism Exercised	Min. Dataset
CS1 Wheat head detection	Tier-0 exact-match PREDICT; establishes baseline completion/F1	10 GWHD images
CS2 Arabidopsis leaf count	Tier-2 zero-shot segmentation fallback; manual calibration applied	5 CVPPP2015 images
CS3 Tomato disease (BUILD)	Classification backbone, quality gate passed first attempt	375 images, 5 classes
CS4 Cotton boll detection (BUILD)	Detection backbone, quality gate passed first attempt	60 images
CS5 Lettuce multi-trait regression (BUILD)	ADAPT recovery: 3 inference failures $\to$ generate-critique-verify loop	112 RGB images
CS6 Satellite crop type (BUILD)	Degenerate $1{\times}1$ -pixel case of the general temporal input format	800 pixel series, 4 classes

Table 7: Six cross-domain case studies. Group 1 exercises the PREDICT path (exact match or zero-shot fallback); Group 2 exercises BUILD (training a new model from an uploaded dataset).

Figure 7: PREDICT-workflow image-requirement notes generated automatically at model-matching time, as rendered in the web UI. CS1 resolves to an exact registered checkpoint, so no fallback notice is shown; CS2 has no registered checkpoint for Arabidopsis leaf counting, so the system surfaces the reduced-confidence, uncalibrated nature of the zero-shot path before the session proceeds.

Figure 9: Model-zoo cards auto-generated and registered at the end of each BUILD case study (CS3–CS6). Each records the model identifier, crop/task tags, headline metric, the dispatched uncertainty-quantification method, and the phenotype outputs the model exposes downstream: a provenance record a reviewer can inspect without touching the underlying code. The CS5 card is labelled auto_0_57b4b4ec : an architecture generated live by the Pipeline Adaptation Agent rather than drawn from the fixed model zoo.

Model	Crop / Domain	Backbone	Origin	$N$ (eval)	Primary Metric	Secondary Metric / UQ	Reliability
Group 1, Close-Range Classification
1. Rice N-severity classifier	Rice	Classification model	Manual	5,790 (full-pool)	Macro F1 0.9721 a	ECE 0.0227	Medium
2. Wheat UAV classification model (TTA)	Wheat	Classification model	Manual	1,896 (full-pool)	Macro F1 0.9052 a	ECE 0.1001	Medium
3. Maize classification model	Maize	Classification model	Manual	17,627 (full-pool)	Macro F1 0.9920 a	ECE 0.1219	Medium
4. Banana classification model	Banana	Classification model	Manual	1,500 (80/20 strat., seed = 42)	Macro F1 0.8555	ECE 0.0560	High
5. Coffee classification model	Coffee	Classification model	Manual	90 (held-out, seed = 42 split)	Macro F1 0.7811	ECE 0.1418	High

Table 8: All 14 trained PhenoIntel checkpoints (10 model-zoo, manually trained offline; 4 case-study Build sessions run live through the web application), grouped by task family with a common column set.

N

is the evaluation-pool size actually used for the reported figure, not the full dataset size. Reliability tiers ( High / Medium ) are defined in Section 6.5 .

Metric	PhenoIntel
Reliability
Pipeline completion rate (all 16 sessions: 10 model-zoo checkpoints + 6 case studies, Section 6.2 )	100.0%
Zero-from-detection failure rate (enforced at checkpoint V5, Section 4.5 )	0% (by design)
Unhandled exception / crash rate (0 crashes across all 16 sessions, Section 6.2 )	0.0%
Internal test-suite pass rate (1,128 unit + 72 integration tests, Section 6.9 )	1200/1200 (100%)
Phenotype Accuracy, CVPPP2015 (Zero-Shot, CS2)

Table 9: PhenoIntel evaluation summary across all evaluation dimensions and metrics. Rows marked a–d report two or more figures under distinct evaluation protocols (see footnotes below); all other rows report a single, unambiguous protocol.

(a) Classification, Close-Range Deficiency / Disease Classifiers (Models 1–6)
1. Rice N-severity, Confusion Matrix	1. Rice N-severity, Reliability	2. Wheat UAV, Confusion Matrix
2. Wheat UAV, Reliability	3. Maize, Confusion Matrix	3. Maize, Reliability
4. Banana, Confusion Matrix	4. Banana, Reliability	5. Coffee, Confusion Matrix
5. Coffee, Reliability	6. Tomato (CS3), Confusion Matrix	6. Tomato (CS3), Reliability

Figure 10: (a) Classification , Close-Range Deficiency/Disease Classifiers (Models 1–6): confusion matrix and reliability diagram per classification model (this page); continued over the following pages, with the full caption given after the final panel (e).

(b) Detection (Models 7–9) and (c) Temporal (Model 10) Results
7. Rice Panicle, Detections	7. Rice Panicle, PR Curve	8. Wheat Spike (GWHD), Detections
8. Wheat Spike (GWHD), PR Curve	9. Cotton Boll (CS4), Confusion Matrix	9. Cotton Boll (CS4), PR Curve
10. Crop-Type Head, Ensemble Comparison	10. Crop-Type Head, Per-Fold F1	10. Satellite (CS6), Training Curve
10. Satellite (CS6), Self-Eval Accuracy/F1	10. Satellite (CS6), Self-Eval Confidence Dist.	CS2 Arabidopsis, Pred. vs. True Leaf Count

Figure 10 continued: (b) Detection models 7–9; (c) Temporal model 10.

(d) Segmentation (CS2) and Regression (Models 11–12) Results
CS2 Arabidopsis, Per-Image Residuals	11. Growth-Rate Head, Pred. vs. True	11. Growth-Rate Head, Residuals
12. Harvest-Timing Head, Pred. vs. True	12. Harvest-Timing Head, Residuals
(e) Multi-Trait Regression (Lettuce, CS5): Predicted vs. True
13. Lettuce, Diameter, Pred. vs. True	13. Lettuce, Dry Weight (Shoot), Pred. vs. True	13. Lettuce, Fresh Weight (Shoot), Pred. vs. True
13. Lettuce, Height, Pred. vs. True	13. Lettuce, Leaf Area, Pred. vs. True	13. Lettuce, Diameter, Residuals

Figure 10 continued: (d) Segmentation (CS2) and Regression models 11–12; (e) Lettuce (CS5) predicted vs. true, per trait.

Panel of Figure 11 — Figure 11: FAIR provenance package: 12 always-present files, checksum-indexed in the package manifest. Badges indicate which FAIR principle each file satisfies.

Plant Phenotyping

Plant phenotyping uses imaging, sensing, and computational methods to measure plant traits, growth, health, and responses to the environment.

See all 68 papers

Sep 28, 2026

Recent Advances in Agentic Agri-Robotic Phenotyping: A Perspective Review from Fragmented Multimodal Sensing to Unified PhenoAgent Intelligence

This review examines the evolution of plant phenotyping from conventional manual trait measurement to high-throughput, robotic, and artificial intelligence-driven crop monitoring. Despite significant advances in imaging, autonomous platforms, multimodal sensing, and deep learning, current phenotyping systems remain fragmented across sensing modalities, crop traits, growth stages, environments, and management objectives. We therefore frame phenotyping as an integrated seed-soil-plant-environment-management (SSPEM) intelligence problem, where crop performance reflects interactions among seed quality, root-zone conditions, plant development, environmental exposure, and management actions. The review synthesizes conventional, high-throughput, robotic, and AI-driven phenotyping approaches, highlighting their capabilities and persistent limitations in temporal integration, multimodal reasoning, biological interpretation, and actionable decision support. Building on this analysis, we introduce a conceptual PhenoAgent framework that extends phenotyping beyond the estimation of isolated traits to evidence-based crop-state interpretation, uncertainty-aware reasoning, and management-oriented support. The PhenoAgent concept primarily brings together scattered advances in phenotyping to deliver insights ranging from detailed to high-level, such as what is happening in the crop, why it might be occurring, what evidence is missing, and what actions or additional measurements should be considered. We also discuss challenges in dataset scarcity, annotation, benchmarking, model generalization, and explainability. By linking multimodal phenotyping with agentic AI and closed-loop decision support, this review outlines a path to interpretable, scalable, and deployment-oriented crop intelligence.

Muhammad Owais, Ehtesham Iqbal, Samee Ullah Khan +3

Khalifa University Center for Autonomous Robotic Systems (KU-CARS), Khalifa University of Science and Technology, Abu Dhabi, United Arab Emirates. · Advanced Research and Innovation Center (ARIC), Khalifa University of Science and Technology, Abu Dhabi, United Arab Emirates. · Department of Computer Science and Engineering, Chung-Ang University, Seoul, South Korea. +1

Plant Phenotyping Agricultural Robotics Multimodal Agents Precision Agriculture

Ask PDF

Jun 30, 2026

An Agentic AI Framework to Accelerate Scientific Discovery in Plant Phenotyping

High-throughput plant phenotyping now generates image derived datasets far faster than scientists can analyze them. At Oak Ridge National Laboratory's Advanced Plant Phenotyping Laboratory (APPL), automated stations image hundreds of plants daily across multiple remote sensing modalities; yet, trait extraction and interpretation remain manual, expert-bound, and strictly post-hoc, making analysis, not acquisition, the binding constraint on discovery. We present an end-to-end agentic AI framework that turns the facility from a data factory into an interactive autonomous, discovery platform, where scientists partner with AI agents to accelerate time to insight. A conversational Co-Scientist Agent translates a scientist's natural-language question into a structured analysis plan, and a headless Compute Agent dispatches Vision Transformer segmentation and trait extraction on the Frontier exascale supercomputer. The two agents run in separate security and resource domains and communicate over a secure, token-authenticated streaming channel, a design that accounts for the federation, data-movement, and provenance realities cloud-native agentic frameworks ignore, ensuring end-to-end provenance is captured for every interaction. The framework turns a days- to weeks-long analysis process into an interactive loop where agents reason over results, recommend next analyses, and respond to follow-up questions in seconds.

Renan Souza, Daniel Rosendo, Kelsey Carter +6

Oak Ridge National Laboratory, Oak Ridge, TN, USA

Plant Phenotyping LLM Agent Orchestration Scientific Workflow Automation Human-AI Collaboration

Ask PDF

Apr 10, 2026

From Scientific Crop Imagery to Agronomic Reasoning: A Multimodal LLM Benchmark for Plant Phenotyping

High-throughput, effective and comprehensive phenotyping is a critical prerequisite for crop improvement. While phenotypic analysis was traditionally performed manually, recent advances in multimodal foundation models, especially in vision-language models, have offered new opportunities for automated plant phenotyping. However, plant science remains particularly challenging for general-purpose VLMs because it requires domain-specific knowledge, fine-grained visual interpretation, and complex agronomic reasoning. We introduce PlantXpert, an evidence-grounded multimodal reasoning benchmark for plant phenotyping across four major crops: soybean, cotton, rice, and corn. Our benchmark provides a structured and reproducible framework for agronomic adaptation of VLMs, and enables controlled comparison between base models and their domain-adapted counterparts. We constructed a dataset comprising 545 digital images and 4,351 benchmark samples spanning key agronomic domains including disease, pest control, weed management and yield. The benchmark can assess diverse capabilities including visual expertise, quantitative reasoning, and multi-step agronomic reasoning. We trained and evaluated a total of 11 state-of-the-art VLMs. Fine-tuning on our dataset improves overall accuracy for all models, with many achieving around 70% accuracy, and improvements on 9 of 11 remaining statistically significant after Holm correction. We also identify diminishing returns from model scaling, uneven performance across crop subsets, and persistent weaknesses in quantitative reasoning and specialized image analysis. These limitations show that general multimodal competence alone is insufficient for reliable agronomic reasoning and demonstrate the value of domain-specific adaptation. We hope PlantXpert can serve as a foundation for advancing evidence-grounded multimodal reasoning in plant science.

Yu Wu, Guangzeng Han, Ibra Niang Niang +6

Computer Science, University of Memphis, Memphis, 38111, TN, United States · Fisher Delta Research Extension and Education Center, University of Missouri, Portageville, 63873, MO, United States · University of Arkansas Division of Agriculture, Batesville, 72501, AR, United States +1

Plant Phenotyping VLM Evaluation VLM Reasoning Multimodal Model Evaluation

Ask PDF

Abstract

Plant Phenotyping

Explore similar work

Recent Advances in Agentic Agri-Robotic Phenotyping: A Perspective Review from Fragmented Multimodal Sensing to Unified PhenoAgent Intelligence

An Agentic AI Framework to Accelerate Scientific Discovery in Plant Phenotyping

From Scientific Crop Imagery to Agronomic Reasoning: A Multimodal LLM Benchmark for Plant Phenotyping