Explainable Multi-Loss Distillation Framework for Efficient and Interpretable Shrimp Disease Text Classification
Abstract
Shrimp disease classification has become an urgent issue due to its significant impact on the import-export output of producing countries, particularly Vietnam. Most existing studies focus on image-based classification, which typically operates at the late stage of disease manifestation. Therefore, text-based classification has the potential to enable early and timely disease detection. To address this limitation, we introduce the SALT (Shrimp disease text Analysis with multi-Loss disTillation) framework, which incorporates explainability analysis using Local Interpretable Model-agnostic Explanations (LIME) and SHapley Additive exPlanations (SHAP) to evaluate model predictions and interpret the learned linguistic features. Experimental results demonstrate that SALT achieves competitive performance across multiple distillation objectives, outperforming supervised baselines while providing a favorable trade-off between predictive performance and computational efficiency. Moreover, it exhibits strong explainability, accurately identifying key linguistic features and semantic patterns relevant to disease descriptions. These findings highlight the potential of knowledge distillation-based text classification for future applications in early shrimp disease diagnosis and related research directions.
Figures & tables
| # | Vietnamese | English |
|---|---|---|
| 1 | Khi bệnh tiến triển, gan tụy từ màu vàng nhạt sang trắng đục, ban đầu sưng nhưng teo dần theo thời gian. Trên bề mặt có nhiều đốm đen, ruột tôm trống không, cơ thể suy yếu và chết hàng loạt. | As the disease progresses, the hepatopancreas changes from pale yellow to opaque white, initially swollen but gradually shrinking over time. Numerous black spots appear on the surface, the intestine becomes empty, and the shrimp’s body weakens, leading to mass mortality. |
| 2 | Tôm bị bệnh có ruột yếu, thức ăn không hấp thụ hết, phân chứa nhiều chất nhầy, nổi trên bề mặt ao. | Diseased shrimp have fragile intestines, incomplete feed absorption, and mucus-laden feces that float on the pond surface. |
| 3 | Tôm bơi yếu, tập trung gần mép bờ và có dấu hiệu cơ thịt hơi đục khi quan sát. | Shrimp exhibit weak swimming behavior, gather near the pond edges, and display slightly opaque musculature upon observation. |
| 4 | Quan sát thấy tôm có rong bám ở mang, vỏ nhớt, tôm chết rải rác dưới đáy ao, số lượng tăng dần theo mức độ bệnh. | Shrimp are observed with algal attachments on the gills, a mucous carapace, and scattered mortalities on the pond bottom, which increase in frequency as the disease progresses. |
| 5 | Tôm bơi không linh hoạt, thường tụ lại ở các góc ao hoặc vùng nước cạn, mang có dấu hiệu phì đại, kèm theo màu nâu hoặc đen dần rõ rệt. | Shrimp swim sluggishly, often cluster in pond corners or shallow areas. The gills appear hypertrophied, with a gradual darkening to brown or black coloration. |
| 6 | Quan sát thấy tôm khó lột xác, chân bơi bị bao phủ bởi lông tơ, mang có màu nâu hoặc đen. | Molting becomes difficult; pleopods are covered with fine filaments, and the gills turn brown or black. |
| Class | Train | Valid | Test | Total |
| Acute Hepatopancreatic Necrosis Disease | 56 | 19 | 19 | 94 |
| White Feces Disease | 60 | 20 | 20 | 100 |
| White Spot Disease | 60 | 20 | 20 | 100 |
| Luminous Bacteria Disease | 58 | 19 | 19 | 96 |
| Loose Shell Syndrome | 59 | 19 | 20 | 98 |
| Black Gill Disease | 60 | 20 | 20 | 100 |
| Class | Train | Valid | Test | Total |
| Acute Hepatopancreatic Necrosis Disease | 332 | 19 | 19 | 370 |
| White Feces Disease | 360 | 20 | 20 | 400 |
| White Spot Disease | 350 | 20 | 20 | 390 |
| Luminous Bacteria Disease | 344 | 19 | 19 | 382 |
| Loose Shell Syndrome | 354 | 19 | 20 | 393 |
| Black Gill Disease | 359 | 20 | 20 | 399 |
| Name | Value |
| Batch size | 16 |
| Epoch | 15 |
| Learning rate | 3e-5 |
| Optimizer | AdamW |
| Loss Function | Focal Loss |
| Component | Parameter | Value |
| Distillation | 0.8 | |
| Distillation | 0.2 | |
| Distillation | Temperature | 3.0 |
| Teacher (Focal) | 2.5 | |
| Teacher (Focal) | Class weight | Class-balanced |
| Center Loss | 0.003 |
| Metrics | PhoBERT-base | mBART |
| Acc (%) | 89.08 1.71 | 86.43 2.03 |
| F1 | 0.8899 0.0182 | 0.8647 0.0200 |
| Precision | 0.8948 0.0163 | 0.8691 0.0193 |
| Recall | 0.8903 0.0175 | 0.8637 0.0203 |
| Inference Time (ms) | 13.34 0.64 | 342.16 61.21 |
| Params (M) | 135.01 | 611.91 |
| Metric | Acc (%) | F1 | Precision | Recall | Time (ms) |
| KL | 88.67( 2.17) | 0.8880 ( 0.0213) | 0.8997 ( 0.0172) | 0.8872( 0.0219) | 3.78( 0.11) |
| MSE | 88.16 ( 1.22) | 0.8822 ( 0.0134) | 0.8940 ( 0.0144) | 0.8818 ( 0.0130) | 3.77 ( 0.04) |
| JSD | 87.65 ( 2.21) | 0.8773 ( 0.021) | 0.8882 ( 0.0191) | 0.8768 ( 0.0219) | 3.78 ( 0.09) |
| SupCon | 86.02 ( 1.23) | 0.8601 ( 0.0138) | 0.8758 ( 0.0103) | 0.8601 ( 0.0135) | 3.77 ( 0.06) |
| Center | 87.45 ( 2.31) | 0.8743 ( 0.0231) | 0.8810 ( 0.0247) | 0.8745 ( 0.023) | 3.80 ( 0.09) |
| Triplet | 87.35 ( 1.22) | 0.8737 ( 0.0138) | 0.8830 ( 0.0141) | 0.8738 ( 0.0125) | 3.75 ( 0.02) |
| Metric | SFT | TF-IDF + DNN | F2LLM-v2-80M |
| Acc (%) | 86.53 ( 2.44) | 88.98 ( 0.72) | 82.86 ( 2.03) |
| F1 | 0.8666 ( 0.0250) | 0.8896 ( 0.0071) | 0.8287 ( 0.0201) |
| Precision | 0.8803 ( 0.0280) | 0.9021 ( 0.0062) | 0.8367 ( 0.0193) |
| Recall | 0.8659 ( 0.0246) | 0.8896 ( 0.0071) | 0.8297 ( 0.0200) |
| Inference Time (ms) | 3.91 ( 0.04) | 76.46 ( 0.44) | 5.53 ( 0.13) |
| Metric | Acc (%) | F1 | Precision | Recall |
| KL | 88.06 ( 0.96) | 0.8819 ( 0.0099) | 0.8941 ( 0.0076) | 0.8815 ( 0.0096) |
| MSE | 87.04 ( 1.46) | 0.8708 ( 0.0143) | 0.8797 ( 0.0152) | 0.8706 ( 0.0139) |
| Cosine | 87.76 ( 1.27) | 0.8784 ( 0.0115) | 0.8911 ( 0.0056) | 0.8779 ( 0.0127) |
| JSD | 86.33 ( 1.81) | 0.8646 ( 0.0171) | 0.8740 ( 0.014) | 0.8636 ( 0.0178) |
| SupCon | 85.92 ( 2.85) | 0.8597 ( 0.0282) | 0.8731 ( 0.0263) | 0.8589 ( 0.0285) |
| Triplet | 86.53 ( 1.31) | 0.8647 ( 0.0138) | 0.8765 ( 0.01) | 0.8649 ( 0.0138) |
| Metric | Acc (%) | F1 | Precision | Recall |
| KL | 88.27 ( 0.78) | 0.8833 ( 0.0076) | 0.8957 ( 0.0070) | 0.8828 ( 0.0081) |
| MSE | 87.96 ( 0.85) | 0.8800 ( 0.0085) | 0.8920 ( 0.0090) | 0.8794 ( 0.0089) |
| Cosine | 88.47 ( 1.31) | 0.8855 ( 0.0130) | 0.8967 ( 0.0099) | 0.8848 ( 0.0133) |
| SupCon | 85.10 ( 1.64) | 0.8505 ( 0.0162) | 0.8691 ( 0.0135) | 0.8511 ( 0.0171) |
| Center | 88.88 ( 1.64) | 0.8891 ( 0.0160) | 0.9008 ( 0.0178) | 0.8891 ( 0.0169) |
| Triplet | 88.67 ( 1.64) | 0.8879 ( 0.0163) | 0.8999 ( 0.0157) | 0.8872 ( 0.0165) |
| Metric | Acc (%) | F1 | Precision | Recall |
| KL | 82.55 (±2.07) | 0.8248 (±0.0211) | 0.8312 (±0.0207) | 0.8263 (±0.0207) |
| MSE | 83.88 (±1.88) | 0.8389 (±0.0187) | 0.8429 (±0.0173) | 0.8394 (±0.0188) |
| JSD | 84.39 (±1.71) | 0.8443 (±0.0161) | 0.8510 (±0.0140) | 0.8449 (±0.0167) |
| SupCon | 82.96 (±2.31) | 0.8299 (±0.0243) | 0.8385 (±0.0275) | 0.8306 (±0.0237) |
| Center | 81.63 (±3.58) | 0.8156 (±0.0363) | 0.8222 (±0.0375) | 0.8169 (±0.0359) |
| Triplet | 81.84 (±2.48) | 0.8189 (±0.0238) | 0.8320 (±0.0174) | 0.8194 (±0.0254) |
| Metric | KL | MSE | Cosine | JSD | SupCon | Center | Triplet |
| Avg SHAP | 0.0179 | 0.0186 | 0.0097 | 0.0212 | 0.0131 | 0.0168 | 0.0093 |
| SHAP Std. | 0.0396 | 0.0399 | 0.0204 | 0.0457 | 0.0283 | 0.0388 | 0.0174 |
| F1-score | 0.8880 | 0.8822 | 0.8935 | 0.8773 | 0.8601 | 0.8743 | 0.8737 |
| Model | Params (M) | Size (MB) | GFLOPs | GPU Latency (ms) | CPU Latency (ms) | Throughput (samples/s) |
| PhoBERT-base (Teacher) | 135.01 | 515.01 | 43.49 | 1.93 | ||
| Student (Cosine KD) | 79.88 | 304.73 | 14.50 | 5.69 | ||
| SFT | 79.88 | 304.73 | 14.50 | 5.69 | ||
| TF-IDF + DNN | 11.45 | 131.05 | - † | 996.21 | ||
| F2LLM-v2-80M | 80.38 | 306.64 | 32.21 | 14.83 |