CLIP-Guided Multi-Task Regression for Multi-View Plant Phenotyping

Simon Warmers; Muhammad Zawish; Fayaz Ali Dharejo; Steven Davy; Radu Timofte

cs.CVMar 4, 2026

CLIP-Guided Multi-Task Regression for Multi-View Plant Phenotyping

Authors: Simon Warmers, Muhammad Zawish, Fayaz Ali Dharejo, Steven Davy, Radu Timofte

Organizations: Computer Vision Lab, CAIDAS, IFI, University of Würzburg, Germany · Technological University Dublin, Ireland

Abstract

Modeling plant growth dynamics plays a central role in modern agricultural research. However, learning robust predictors from multi-view plant imagery remains challenging due to strong viewpoint redundancy and viewpoint-dependent appearance changes. We propose a level-aware vision language framework that jointly predicts plant age and leaf count using a single multi-task model built on CLIP embeddings. Our method aggregates rotational views into angle-invariant representations and conditions visual features on lightweight text priors encoding viewpoint level for stable prediction under incomplete or unordered inputs. On the GroMo25 benchmark, our approach reduces mean age MAE from 7.74 to 3.91 and mean leaf-count MAE from 5.52 to 3.08 compared to the GroMo baseline, corresponding to improvements of 49.5% and 44.2%, respectively. The unified formulation simplifies the pipeline by replacing the conventional dual-model setup while improving robustness to missing views. The models and code is available at: https://github.com/SimonWarmers/CLIP-MVP

Figures & tables

Figure 1: The proposed preprocessing pipeline based on Grounding DINO [ 17 ] and CLIP [ 8 ] that encodes images into rich visual embeddings for downstream tasks.

Figure 2: The proposed multi-task pipelines: (a) shows unimodal baseline based on CLIP vision embeddings, (b) shows conditioning CLIP on level priors for multimodal regression.

Approach	MAE (Age)				MAE (Leaf Count)
	Mustard	Radish	Wheat	Mean	Mustard	Radish	Wheat	Mean
GroMo [8]	10.62	5.71	8.80	7.74	4.99	4.04	10.80	5.52
CropIQ [ 3 ]	21.70	16.54	28.60	19.41	5.65	6.48	8.70	5.83
Agro_Geek [ 3 ]	11.30	18.85	28.45	18.01	7.38	3.98	30.41	10.90
Rishi [ 3 ]	12.64	10.08	16.79	12.98	4.15	6.89	5.18	8.68
SoumikDas [ 3 ]	10.18	2.68	10.60	8.65	3.33	1.33	7.20	3.73

Table 1: Comparison of different approaches with proposed methods based on Mean Absolute Error (MAE) for plant age and leaf count prediction. Lower values indicate better performance.

Approach	Multimodal	Multitask
GroMo [ 5 ]	$\times$	$\times$
CropIQ [ 3 ]	$\times$	$\times$
Agro Geek [ 3 ]	$\times$	$\times$
Rishi [ 3 ]	$\times$	$\times$
SoumikDas [ 3 ]	$\times$	$\times$
PlantPixels [ 3 ]	$\times$	$\times$

Table 2: Comparison of multimodality and multitask capability across different approaches.

Figure 3: MAE of both approaches as a function of the percentage of viewpoint images removed at runtime.

Figure 4: Overall degradation from 0% removal to only 1 image remaining.

Plant Phenotyping

Plant phenotyping uses imaging, sensing, and computational methods to measure plant traits, growth, health, and responses to the environment.

See all 68 papers

Meta-Learning for Data-Efficient Plant Growth Estimation via Vision Transformers and Fuzzy Clustering
Sep 9, 2026Sheikh Hasan Elahi, Rusith Chamara Hathurusinghe Dewage, Habib Ullah +3Plant Phenotyping Meta-Learning
Department of Data Science, Norwegian University of Life Sciences (NMBU), Norway · Photosynthetic AS, Hegdehaugsveien 24, 0352 Oslo, Norway · Department of Mechanical Engineering and Technology Management, Norwegian University of Life Sciences (NMBU), Norway
MCLC-NET: Multimodal Continual Learning for Leaf Counting
Sep 16, 2026Ruchi Bhatt, Pratibha Kumari, Shreya Bansal +3Plant Phenotyping Object Counting
Department of Computer Science & Engineering, Indian Institute of Technology Ropar, Punjab, India · Department of Informatics and Data Science, University of Regensburg, Regensburg, Germany · Department of Computer Science, Khalifa University, UAE
Using Vision Language Foundation Models to Generate Plant Simulation Configurations via In-Context Learning
Mar 9, 2026Heesup Yun, Isaac Kazuo Uyehara, Earl Ranario +4VLM Evaluation Agricultural Image Analysis
University of California, Davis

Sep 9, 2026J/K move · Enter open · S save

Meta-Learning for Data-Efficient Plant Growth Estimation via Vision Transformers and Fuzzy Clustering

Sheikh Hasan Elahi, Rusith Chamara Hathurusinghe Dewage, Habib Ullah, Muhammad Salman Siddiqui+2

Department of Data Science, Norwegian University of Life Sciences (NMBU), Norway · Photosynthetic AS, Hegdehaugsveien 24, 0352 Oslo, Norway · Department of Mechanical Engineering and Technology Management, Norwegian University of Life Sciences (NMBU), Norway

Plant Phenotyping Meta-Learning Few-Shot Learning

Open paper AskPDF arXiv

Accurate plant growth estimation is essential for greenhouse monitoring, yet obtaining labeled data remains costly and time-consuming. To address this, we propose a few-shot regression framework that combines Vision Transformer (ViT) feature embeddings, clustering-based task construction, and gradient-based meta-learning, and show that task construction in embedding space is a primary driver of performance. The approach leverages an unlabeled image pool to organize data into structured tasks using fuzzy c-means clustering, enabling efficient learning from a small number of labeled samples. We systematically evaluate meta-learning methods and show that second-order methods (e.g., Model-Agnostic Meta-Learning variants such as MAML++) outperform classical baselines in the few-shot regime. Furthermore, intra-cluster support selection has a limited and dataset-dependent impact. Experiments on two plant datasets show that structured task design combined with meta-learning enables reliable plant growth estimation under severe label scarcity.

Abstract

Plant Phenotyping

Explore similar work

Meta-Learning for Data-Efficient Plant Growth Estimation via Vision Transformers and Fuzzy Clustering

MCLC-NET: Multimodal Continual Learning for Leaf Counting

Using Vision Language Foundation Models to Generate Plant Simulation Configurations via In-Context Learning