Organizations: Department of Computer Science and Engineering, Kathmandu University, 45210, Nepal · Department of Agricultural and Biological Engineering, Mississippi State University, MS 39762, USA · Electrical and Computer Engineering, Michigan State University, MI 48824, USA · Electrical Engineering and Computer Science, University of Missouri, MO 65211, USA
Timely and accurate crop health diagnosis is essential for sustainable agricultural production, yet accurate diagnosis often requires expert knowledge that is not readily available. Recent vision-language models (VLMs) have demonstrated remarkable capabilities in visual understanding, reasoning, and natural-language interaction, creating new opportunities for intelligent crop health assessment. However, deploying VLMs on resource-constrained devices remains challenging due to their substantial computational and memory requirements. This paper presents PlantVLM-Edge, a quantized vision-language framework for conversational crop health diagnosis that can run entirely on resource-constrained edge devices. PlantVLM-Edge combines agricultural domain adaptation, post-training quantization, and on-device serving with an offline chat interface, allowing growers to ask open-ended questions and receive diagnoses, symptom explanations, and management guidance without cloud connectivity. The framework is backbone-agnostic: we validate it with two different VLM types with varying scale and architecture (Qwen3-VL-2B and LLaVA-1.5-7B), four quantization schemes, and two diagnostic domains (plant diseases and insect pests), and we deploy all variants on an NVIDIA Jetson AGX Orin. Eight-bit quantization affects answer quality minimally relative to the full-precision model, and 4-bit activation-aware weight quantization (AWQ) retains 98.6% of full-precision answer quality on both domains while reducing the weight memory of the 2B backbone by 49%. In its most compact form, PlantVLM-Edge requires 2.01 GiB of weight memory, returns its first token in about 0.12 s, and completes a 64-token diagnostic response in 0.78 s. These results demonstrate that timely and accurate crop health diagnosis is practical on edge devices, providing a foundation for next-generation crop monitoring and decision support systems.
Figures & tables
Figure 1: Representative image-question-answer pairs from the PlantExpertVQA and IP-VQA datasets.
Question Category
n
Existence & sanity check
72
Plant species identification
2,147
General health assessment
1,337
Visual attribute grounding
716
Detailed verification
1,689
Specific disease identification
3,972
Table 1: Question categories in the PlantExpertVQA evaluation set, where n is the number of question-answer pairs in each category.
Category
Question focus
Pest identification
Pest identity and type (4354)
Pest feeding and damage
Feeding, damage, and impact (4135)
Optimal timing
Timing and season of control (345)
Pest monitoring
Monitoring and detection (1121)
Pest control methods
Control, treatment, and prevention (3234)
Visual / characteristics
Appearance and visual characteristics (2199)
Table 2: Question categories in the IP-VQA evaluation set. Values in parentheses are the number of question-answer pairs in each category.
Figure 2: Overview of the PlantVLM-Edge framework. A pretrained vision-language model is adapted to plant-disease and insect-pest question answering with LoRA, with the vision encoder and projector kept frozen. The fine-tuned model is then quantized into four configurations and served with vLLM on an NVIDIA Jetson AGX Orin, where its weight footprint and latency are measured. Growers interact with the deployed model offline through a browser-based chat interface.
Qwen3-VL-2B
LLaVA-1.5-7B
LoRA
Rank r
64
64
α
128
128
Dropout
0.05
0.05
Trainable params
69.73M
159.91M
% of model
3.17%
2.21%
Table 3: Fine-tuning configuration. Optimization settings are shared across both models and all fine-tuning runs; model-specific settings are listed separately.
Size (GiB)
Config.
Weights
Act.
Calib.
Qwen
LLaVA
Fine-tuned
16
16
-
3.96
13.16
W8A16-RTN
8 (per-ch.)
16
None
2.65
7.13
W8A8-SQ
8 (per-ch.)
8
128 ex.
2.65
7.13
W4A16-RTN
4 (g128)
16
None
2.01
4.20
W4A16-AWQ
4 (g128)
16
128 ex.
2.01
4.20
Table 4: Quantization configurations and the resulting weight footprint (GiB; 1 GiB = 1.07 GB). All configurations quantize only the linear layers of the language backbone; the vision pathway remains at 16-bit precision.
Figure 3: Illustration of the time to first token (TTFT) and inter-token latency (ITL) metrics, following ( NVIDIA Corporation, 2024 ) .
PlantExpertVQA
IP-VQA
Configuration
R-L
F1
EM
BERT
BLEU
R-L
F1
EM
BERT
BLEU
Qwen3-VL-2B
Zero-shot
15.45
19.35
0.29
85.03
3.10
30.45
33.22
0.33
89.35
5.70
Fine-tuned (BF16)
78.18
79.52
33.64
96.56
65.87
54.67
57.13
4.91
94.06
30.03
W8A16-RTN
78.04
79.42
33.53
96.55
65.73
54.70
57.18
4.87
94.06
30.05
W8A8-SmoothQuant
78.11
79.46
33.60
96.55
65.90
54.40
56.84
4.57
94.02
29.58
Table 5: Answer quality of both backbones on the PlantExpertVQA and the IP-VQA test set. All metrics are reported on a 0-100 scale; higher is better. For each model, the best result among the fine-tuned configurations in each column is in bold.
Qwen3-VL-2B
LLaVA-1.5-7B
Category
n
16-bit
W8
SQ
W4
AWQ
16-bit
W8
SQ
W4
AWQ
Causal Reasoning
2,180
66.02
−0.26
−0.58
−1.41
−1.44
65.74
−0.03
−0.09
−0.52
−0.25
Comprehensive Descr.
1,767
64.74
−0.21
+0.41
−3.67
−0.93
63.75
+0.05
0.00
−0.19
−0.26
Counterfactual Reas.
1,202
74.00
−0.15
+0.26
−1.84
−0.41
74.58
−0.06
−0.11
+0.41
−0.73
Detailed Verification
1,689
79.79
−0.14
+0.08
−0.65
−0.79
80.38
0.00
−0.20
−0.24
−0.51
Existence & Sanity
72
100.00
0.00
0.00
−0.32
−1.57
100.00
0.00
0.00
0.00
0.00
Table 6: Category-level ROUGE-L on the PlantExpertVQA test set. For each backbone, the first column gives the 16-bit score and the remaining columns give the change under each quantized configuration.
Qwen3-VL-2B
LLaVA-1.5-7B
Category
n
16-bit
W8
SQ
W4
AWQ
16-bit
W8
SQ
W4
AWQ
Optimal Timing
345
53.40
+0.17
+0.29
−0.98
−1.87
52.64
−0.22
+0.79
−0.85
−0.50
Other
3,562
58.34
+0.11
−0.12
−1.97
−0.62
57.94
−0.03
+0.27
−0.17
−0.25
Pest Control Methods
3,234
47.40
+0.12
−0.15
−2.43
−0.43
47.27
+0.10
+0.07
−0.01
−0.18
Pest Feeding & Damage
4,135
48.55
−0.03
−0.38
−1.65
−0.97
48.57
−0.03
−0.01
−0.29
−0.43
Pest Identification
4,354
62.77
+0.07
−0.32
−1.34
−0.80
62.54
+0.08
−0.31
−0.31
−0.69
Table 7: Category-level ROUGE-L on the IP-VQA test set. For each backbone, the first column gives the 16-bit score and the remaining columns give the change under each quantized configuration.
Config.
Weights
TTFT
E2E
ITL
(GiB)
(ms)
(ms)
(ms/tok)
Qwen3-VL-2B
BF16
3.96
130.4 (30)
1535.6 (31)
22.3
W8A16-RTN
2.65
123.3 (31)
1042.2 (32)
14.6
W8A8-SQ
2.65
116.1 (29)
1117.5 (30)
15.9
W4A16-RTN
2.01
124.1 (32)
782.5 (33)
10.5
Table 8: On-device efficiency on the Jetson AGX Orin for the plant disease models. Latencies are means over 40 requests of 64 tokens each. Standard deviations of TTFT and E2E are in parentheses. BF16 and FP16 denote the fine-tuned 16-bit models; SQ: SmoothQuant. The lowest value for each model is in bold.
Figure 4: Accuracy-efficiency trade-off for the ten PlantExpertVQA configurations on the Jetson AGX Orin: ROUGE-L against (a) weight footprint and (b) inter-token latency. Lines connect the Pareto-optimal configurations of each model.
Figure 5: Screenshots of multi-turn conversations with the deployed PlantVLM-Edge system (Qwen3-VL-2B, W8A16-RTN) on the Jetson AGX Orin, accessed offline through a browser.