CoAtNet-DeepMoE: A Convolution-Attention Hybrid with DeepSeek Mixture-of-Experts for Parameter-Efficient Tomato Disease Classification
Organizations: Department of Computer Science and Engineering Begum Rokeya University Rangpur, Bangladesh · Department of Computer Science Texas State University San Marcos, Texas, USA · Department of Information and Communication Technologies Asian Institute of Technology Pathum Thani, Thailand
Abstract
The world population is growing rapidly, and technology is improving in parallel. Meeting the huge demand for food for these 7 billion people not only depends on increasing food production but also on reducing food loss. Crop losses due to disease affect both the food supply and the financial and economic stability of a country. Tomatoes are among the top food-producing crops globally, and a significant portion of this production is lost due to disease. People have used Machine Learning techniques for feature extraction and early diagnosis of tomato diseases, and nowadays, Deep Learning-based models are widely used for disease recognition. However, most existing models are highly parameter-intensive, which increases the time required for training and inference. As a result, while lightweight models are more suitable for user-friendly applications, they often show a reduction in performance. To balance performance and model size, we propose CoAtNet-DeepMoE, a Convolution-Attention hybrid architecture for rich feature extraction, further enhanced with a DeepSeek Mixture of Experts to substantially reduce the number of parameters without sacrificing accuracy. We evaluate our model on both balanced and imbalanced datasets from Kaggle and PlantVillage, demonstrating robustness and achieving 99.80% accuracy, 99.80% precision, 99.80% recall, and 99.80% F1-score on Kaggle, and 99.83% accuracy, 99.85% precision, 99.76% recall, and 99.80% F1-score on PlantVillage, representing state-of-the-art performance with only 2.47M parameters. The source code will be available at https://github.com/nadimbrur/CoAt-MoE.
Figures & tables
| Module | CoAt-Base | CoAt-DeepMoE |
| Stem | 19,360 | 19,360 |
| MbConv-0 | 236,768 | 83,648 |
| MbConv-1 | 1,410,720 | 208,416 |
| Transformer2D-2 | 12,145,374 | 437,174 |
| Transformer2D-3 | 12,852,804 | 1,715,492 |
| Classifier Head | 9,226 | 9,226 |
| Dataset | BS | EB | LB | LM | SLS | SE | TS | YLCV | MV | Healthy | Total |
| Kaggle | 1100 (0.10) | 1100 (0.10) | 1100 (0.10) | 1100 (0.10) | 1100 (0.10) | 1100 (0.10) | 1100 (0.10) | 1100 (0.10) | 1100 (0.10) | 1100 (0.10) | 11000 (1.00) |
| PlantVillage | 2127 (0.12) | 1000 (0.06) | 1909 (0.11) | 952 (0.05) | 1771 (0.10) | 1676 (0.09) | 1404 (0.08) | 5357 (0.30) | 373 (0.02) | 1591 (0.09) | 18160 (1.00) |
| Total | 3227 (0.11) | 2100 (0.07) | 3009 (0.10) | 2052 (0.07) | 2871 (0.09) | 2776 (0.10) | 2504 (0.09) | 6457 (0.22) | 1473 (0.05) | 2691 (0.09) | 29160 (1.00) |
| Bacterial spot: BS; Leaf Mold: LM; Early blight: EB; Late blight: LB; Septoria leaf spot: SLS; Spider mites: SE; Target Spot: TS; Yellow Leaf Curl Virus: YLCV; Mosaic virus: MV | |||||||||||
| Parameter | Value |
| Learning Rate (LR) | |
| Loss Function | CE (label_smoothing = 0.1) |
| Optimizer | AdamW (weight_decay = ) |
| LR Scheduler | CosineAnnealingLR ( ) |
| Stopping Patience | 50 epochs |
| Batch Size | 32 |
| Metric | Expression |
| Accuracy | |
| Recall | |
| Precision | |
| F1-score |
| SL | Reference | Parameters | Model Name | Accuracy | Precision | Recall | F1-score |
| 1 | [ 37 ] | 11.00 M | YOLOv8s | 93.2 | 90.3 | – | – |
| 2 | [ 9 ] | 32.29 M | DenseNet | 95.4 | 95.7 | 95.4 | 95.4 |
| 3 | [ 23 ] | 23.50 M | ResNet50 | 92.4 | – | – | – |
| 4 | [ 23 ] | 23.56 M | SDE-ResNet50 | 98.0 | 98.0 | – | 98.0 |
| 5 | [ 23 ] | 54.32 M | Inception-ResNet | 98.2 | 98.23 | – | 98.2 |
| 6 | [ 49 ] | 200.00 M | AlexNet + ResNet50 + VGG16 + QSSVM | 98.3 | – | – | – |
| SL | Reference | Parameters | Model Name | Accuracy | Precision | Recall | F1-score |
| 1 | [ 52 ] | 138.36 M | VGG16 | 96.46 | – | – | – |
| 2 | [ 52 ] | 3.50 M | MobileNetV2 | 95.76 | – | – | – |
| 3 | [ 52 ] | 27.53 M | Swin Transformer | 99.10 | – | – | – |
| 4 | [ 52 ] | 8.06 M | Vision Transformer | 99.45 | – | – | – |
| 5 | [ 41 ] | 25.60 M | ResNet50 | 99.80 | – | – | – |
| 6 | [ 51 ] | 5.70 M | ViT | 96.50 | 93.90 | 96.70 | 94.20 |
| PlantVillage Kaggle | Kaggle PlantVillage | |||||||
| Class | Accuracy | Precision | Recall | F1-score | Accuracy | Precision | Recall | F1-score |
| BS | 1.0000 | 1.0000 | 1.0000 | 1.0000 | 0.9994 | 1.0000 | 0.9951 | 0.9975 |
| EB | 1.0000 | 1.0000 | 1.0000 | 1.0000 | 0.9997 | 1.0000 | 0.9953 | 0.9977 |
| LB | 1.0000 | 1.0000 | 1.0000 | 1.0000 | 0.9994 | 0.9949 | 1.0000 | 0.9974 |
| LM | 1.0000 | 1.0000 | 1.0000 | 1.0000 | 1.0000 | 1.0000 | 1.0000 | 1.0000 |
| SLS | 0.9990 | 0.9901 | 1.0000 | 0.9950 | 0.9997 | 1.0000 | 0.9971 | 0.9986 |
| Model | Params (M) | GFLOPs | Time/img (ms) | p95 (ms) | p99 (ms) | FPS |
| CoAtNet-Base | 26.67 | 8.42 | 6.79 | 6.90 | 6.98 | 147.3 |
| ResNet50 | 23.53 | 8.26 | 4.47 | 4.65 | 4.70 | 223.6 |
| ViT-Tiny | 5.53 | 2.15 | 3.78 | 4.93 | 5.02 | 264.8 |
| CoAtNet-DeepMoE | 2.47 | 1.34 | 4.04 | 4.19 | 4.24 | 247.6 |
Plant Disease Classification
Plant disease classification uses image and sensor data to identify crop diseases and support timely, targeted agricultural intervention.
See all 60 papers