Lightweight Vision Transformer Compression for On-Device Plant Disease Detection in Resource-Constrained Agricultural Field Conditions

Mahadev Sunil Kumar; Bhavika Gondi; Desaisetty Venkata Satya Sai Swapnith; Gangireddy Rahul Jogi; Sudheesh Manalil; Arnab Raha; Amitava Mukherjee; Parthasarathy Seethapathy

cs.CVSep 4, 2026

Lightweight Vision Transformer Compression for On-Device Plant Disease Detection in Resource-Constrained Agricultural Field Conditions

Authors: Mahadev Sunil Kumar, Bhavika Gondi, Desaisetty Venkata Satya Sai Swapnith, Gangireddy Rahul Jogi, Sudheesh Manalil, Arnab Raha, Amitava Mukherjee, Parthasarathy Seethapathy,

Organizations: Department of Computer Science and Engineering, Amrita School of Computing, Amrita Vishwa Vidyapeetham, Amritapuri, Kerala, India · Amrita School of Agricultural Sciences, Amrita Vishwa Vidyapeetham, Coimbatore, 642109, Tamil Nadu, India · Intel Corporation, Santa Clara, California, USA · Department of Computer Science, Birla Institute of Technology and Science, Pilani Dubai Campus, Dubai, United Arab Emirates ·

Abstract

Chilli (Capsicum annuum) is one of India's most economically significant crops, yet its productivity is persistently threatened by diseases that are difficult to identify without expert intervention. While Vision Transformers (ViTs) have achieved high classification accuracy, their large computational footprint makes deployment on resource constrained devices challenging. Existing compression approaches typically address pruning, quantization, and knowledge distillation in isolation, leaving the potential benefits and interactions of their combined application insufficiently explored. We propose a unified Vision Transformer compression framework that combines Hessian-Balanced Adaptive Block Pruning (H-BAC), guided by second-order sensitivity estimation, with quantization and attention-based knowledge distillation. To systematically identify the most effective configuration within each compression family, each technique is first evaluated independently through controlled ablation studies, after which the best-performing components are integrated into a sequential deployment pipeline tailored to real-world agricultural constraints. On a chilli 3-class village-split dataset with a genuine cross-village, cross-device out-of-distribution test split, the resulting compressed models match or exceed the 95.13% FP32 baseline's accuracy, alongside 74-98% model size reduction, and the fully integrated compression pipeline achieves a 54.5x size reduction (327.42 MB to 6.01 MB) at 95.13 +/- 2.32% accuracy across four tested configurations. A direct comparison further reveals that, on this dataset, a directly-trained student of the same final size, without pruning or distillation, reaches comparable accuracy of 94.87%, at the same 6.01 MB INT8 size, indicating where H-BAC and knowledge distillation are, and are not yet shown to be, worth their computational cost.

Figures & tables

Figure 1: Indigenously collected chilli leaf images captured in field conditions.

Figure 2: Per-class image counts across the four village-partitioned splits (log scale).

Metric	Value
OOD Accuracy	95.13%
Precision (weighted)	95.55%
Recall (weighted)	95.13%
F1-Score (weighted)	95.10%
Model Size	327.42 MB
Parameters	85.80M

Table 1: Baseline ViT-B/16 Performance and Deployment Characteristics

Figure 3: Workflow of a size/accuracy-budget-driven compression workflow: compress, check against target constraints, escalate if not met. Section 4 details the concrete 4-stage cascade used by the constrained search mode and its results.

Model	Accuracy (%)	FLOPs
ViT-B/16 Baseline	95.13	35.13 GFLOPs
ViT-B/16 + H-BAC (50%)	95.53	17.86 GFLOPs

Table 2: H-BAC 50% Pruning – Classification Performance (post-finetune)

Model	Active Params	Size (MB)	CPU (ms)	GPU (ms)
ViT-B/16 Baseline	85.80M	327.42	7.18	5.560
ViT-B/16 + H-BAC (50%)	43.70M	327.42	7.18	5.560

Table 3: H-BAC 50% Pruning – Deployment Characteristics, uncompacted mask-based model (CPU: Apple M4 Pro via Core ML; GPU: NVIDIA RTX 4060 via PyTorch)

Stage	Accuracy (%)	Size (MB)	Active Params (M)	CPU (ms)	GPU (ms)
FP32 Baseline †	95.13	327.42	85.80	7.18	5.560
H-BAC Pruned (pre-finetune)	64.87	327.42	43.70	7.18	5.560
H-BAC Pruned (finetuned)	95.53	327.42	43.70	7.18	5.560
+ Attention-Based KD	93.68	21.15	5.52	1.02	0.660
+ PTQ-Dynamic (deployed)	91.97	6.01	5.52	1.02	–
Direct-training alternative ∗	94.87	6.01	5.52	1.02	—

Table 4: Integrated Compression Pipeline: stage-by-stage results (H-BAC 50%, OOD split; CPU: Apple M4 Pro via Core ML; GPU: NVIDIA RTX 4060 via PyTorch)

Figure 4: OOD accuracy (%) of H-BAC pruned ViT-B/16 at each global pruning ratio, before and after one epoch of recovery fine-tuning. Both curves are measured.

Pruning ratio (%)	Active params (M)	FLOPs (G)	Acc. pre-FT (%)	Acc. post-FT (%)	CPU (ms)	GPU (ms)
0 (baseline)	85.80	35.13	95.13	93.95	7.13	5.560
10	77.14	31.57	94.87	96.45	6.76	5.034
20	68.56	28.04	89.34	96.32	6.13	4.554
30	60.34	24.68	73.03	94.61	5.50	4.027
40	51.72	21.13	68.68	97.37	4.88	3.494
50	43.70	17.86	64.87	95.53	4.27	3.003

Table 5: Ablation of H-BAC pruning ratio. FLOPs and active parameters (mask-based model) decrease with pruning ratio; CPU/GPU latency measured on a structurally compacted copy (CPU: Apple M4 Pro via Core ML, 3 trials; GPU: NVIDIA RTX 4060 via PyTorch, 10 trials).

Method	Size (MB)	Acc. (%)	$\Delta$ Acc. (%)	Latency (ms)	Speedup
FP32 Baseline	327.42	95.13	—	7.18	1.00 $\times$
PTQ-Dynamic	84.42	94.34	$\mathbf{-}$ 0.79	7.21 a	1.00 $\times$
PTQ-Static	85.78	94.61	$-$ 0.52	7.21 a	1.00 $\times$

Table 6: Ablation of quantization methods (CPU: Apple M4 Pro via Core ML, 3 trials). Both methods achieve a

\sim

74% model size reduction; the difference is in accuracy retention.

Method	Size (MB)	Compression Ratio	Acc. (%)	CPU Speedup b
Teacher (ViT-B/16)	327.42	$1.00\times$	$95.13$	$1.00\times$
TinyViT (no KD) a	21.15	$15.48\times$	$96.58\pm 0.26$	7.07 $\times$
Response KD	21.15	$15.48\times$	$96.27\pm 0.65$	7.07 $\times$
Feature KD	21.15	$15.48\times$	$96.18\pm 0.60$	$7.07\times$
Attention KD	21.15	$15.48\times$	$96.71\pm 1.03$	$7.07\times$

Table 7: KD Ablation Study: Model Size, Performance, and Speedup Comparison (CPU: Apple M4 Pro via Core ML, 3 trials), mean

\pm

std over 3 runs

Params	No-KD (%)	Attn-KD (%)	Advantage (pp)
1.31 M	$77.24\pm 13.75$	$70.26\pm 19.15$	$-6.97\pm 32.59$
1.89 M	$70.57\pm 19.35$	$76.80\pm 15.86$	$+6.23\pm 34.25$
2.63 M	$77.15\pm 6.01$	$81.32\pm 8.92$	$+4.17\pm 5.41$
3.16 M	$90.35\pm 0.88$	$82.11\pm 8.07$	$-8.25\pm 8.06$

Table 8: Capacity sweep: Attention-Based KD vs. no-KD direct training (both taken through PTQ-Dynamic quantization, applied directly with no extra fine-tuning), four from-scratch (non-pretrained) student sizes below TinyViT’s 5.52 M parameters, mean

\pm

std over 3 runs

Size budget	Stage reached	Family	Acc. (%)	Size (MB)
10–20 MB	9	1.89M-param student (FP32, no-KD)	94.08	7.27
25–30 MB	3	KD student (FP32)	93.68	21.15

Table 9: Constrained search: full 10-target grid (5 size budgets

\times

2 accuracy-drop budgets). All 10 combinations were satisfied.

Stage	Family	Acc. (%)	Size (MB)	Result
3	KD student (FP32)	93.68	21.15	size fails
4	KD student + INT8	91.97	6.01	drop fails
5	3.16M student (FP32)	86.32	12.11	drop fails
6	3.16M student + INT8	86.45	3.64	drop fails
7	2.63M student (FP32)	83.42	10.08	drop fails
8	2.63M student + INT8	83.29	3.08	drop fails

Table 10: Escalation trace for the tightest target in Table 9 (10 MB, 2% accuracy-drop budget), stages 3 through 9. Stages 1–2 (H-BAC pruning, with or without quantization) already fail every budget in this grid on size alone and are omitted; stages 5–8 fit the size budget but fail the accuracy budget.

Model	Mean Latency (ms)		Std. Dev. (ms)
	CPU	GPU	CPU	GPU
ResNet50	2.320	1.712	0.010	0.0001
EfficientFormer	1.386	1.031	0.010	0.0002
Our Model	1.015	0.660	0.002	0.0003

Table 11: Comparison of our optimized model to existing models (CPU: Apple M4 Pro via Core ML, 3 trials; GPU: NVIDIA RTX 4060 via PyTorch, 10 trials)

Research Work	Dataset	Acc. (%)	Architecture
Current Study (Ours)	Chilli 3-Class, Village-Split Field Images (OOD split)	95.13	ViT-B/16
Deep Learning-Based Disease Detection Model in Plants	PlantVillage	99.69	GoogLeNet
Multiclass Plant Disease Detection via Dense CNNs	PlantVillage	99.25	DenseNet201

Table 12: Comparison of Existing Research on Plant Disease Detection with the Current Study

Configuration	Size (MB)	Reduction (%)	Accuracy (%)	CPU Latency (ms)	GPU Latency (ms)
ViT-B/16 (FP32)	327.42	—	95.13	7.18	5.560
H-BAC (50% pruning)	327.42	49.1 ∗	95.53	4.27 †	3.003 †
PTQ-Dynamic	84.42	74.2	94.34	7.21	—
Attention KD	21.15	93.5	$96.71\pm 1.03$	1.02	0.660
Integrated Pipeline	6.01	98.2	$95.13\pm 2.32$ ‡	1.02 §	—

Table 13: Summary of the best configuration from each compression family, plus the fully integrated pipeline (last row). H-BAC reduces effective computation (FLOPs) without reducing stored model size, while quantization and KD reduce both model size and inference latency. CPU: Apple M4 Pro via Core ML; GPU: NVIDIA RTX 4060 via PyTorch.

Quantization

See all 49 papers

AgriKD: Cross-Architecture Knowledge Distillation for Efficient Leaf Disease Classification
May 2, 2026Minh-Dung Le, Minh-Duc Hoang, Hoang-Vu Truong +1Efficient Neural Network Inference Plant Disease Classification
AIT laboratory, Faculty of Artificial Intelligence, FPT University, Da Nang, 550000, Viet Nam
FTerViT: Fully Ternary Vision Transformer
May 20, 2026Szymon Ruciński, Pietro Bonazzi, Engin Türetken +3Ternary Quantization Vision Transformer
CSEM, Neuchâtel, Switzerland · ETH Zürich, Zurich, Switzerland
Rethinking Depth Pruning for Vision Transformers: A Heterogeneity-Aware Perspective
Jul 4, 2026Zhenfeng Su, Kang Zhao, Han Bao +4Vision Transformer Efficient ViTs
School of Data Science, The Chinese University of Hong Kong, Shenzhen, China · Huawei Technologies Co., Ltd., Shenzhen, China · Tsinghua University, Beijing, China +1

May 2, 2026J/K move · Enter open · S save

AgriKD: Cross-Architecture Knowledge Distillation for Efficient Leaf Disease Classification

Minh-Dung Le, Minh-Duc Hoang, Hoang-Vu Truong, Thi-Thu-Hong Phan

AIT laboratory, Faculty of Artificial Intelligence, FPT University, Da Nang, 550000, Viet Nam

Efficient Neural Network Inference Plant Disease Classification Cross-Architecture Knowledge Distillation Knowledge Distillation On-Device Inference

Open paper AskPDF arXiv

Automated leaf disease classification is critical for early disease detection in resource-constrained field environments. Vision Transformers (ViTs) provide strong representation capability by modeling long-range dependencies and inter-class relationships; however, their high computational cost makes them impractical for deployment on edge devices. As a result, existing approaches struggle to effectively transfer these rich representations to lightweight models. This paper introduces AgriKD, a cross-architecture knowledge distillation framework for efficient edge deployment, which transfers knowledge from a Vision Transformer (ViT) teacher to a compact convolutional student model. To bridge the representational gap between Transformer and CNN architectures, the proposed approach integrates multiple distillation objectives at the output, feature, and relational levels, where each objective captures a different aspect of the teacher knowledge. This enables the student model to better preserve and utilize transformer-derived global representations. Experiments on multiple leaf disease datasets show that the distilled student achieves performance comparable to the teacher while significantly improving efficiency, reducing model parameters by approximately 172 times, computational cost by 47.57 times, and inference latency by 18-22 times. Furthermore, the optimized model is deployed across multiple runtime formats, including ONNX, TFLite Float16, and TensorRT FP16, achieving consistent predictive performance with negligible accuracy degradation. Real-world deployment on NVIDIA Jetson edge devices and a mobile application demonstrates reliable real-time inference, highlighting the practicality of AgriKD for AI-powered agricultural applications in resource-constrained environments.

Abstract

Quantization

Explore similar work

AgriKD: Cross-Architecture Knowledge Distillation for Efficient Leaf Disease Classification

FTerViT: Fully Ternary Vision Transformer

Rethinking Depth Pruning for Vision Transformers: A Heterogeneity-Aware Perspective