cs.CVAug 29, 2026
SaveExplainable Multi-Loss Distillation Framework for Efficient and Interpretable Shrimp Disease Text Classification
Abstract
Shrimp disease classification has become an urgent issue due to its significant impact on the import-export output of producing countries, particularly Vietnam. Most existing studies focus on image-based classification, which typically operates at the late stage of disease manifestation. Therefore, text-based classification has the potential to enable early and timely disease detection. To address this limitation, we introduce the SALT (Shrimp disease text Analysis with multi-Loss disTillation) framework, which incorporates explainability analysis using Local Interpretable Model-agnostic Explanations (LIME) and SHapley Additive exPlanations (SHAP) to evaluate model predictions and interpret the learned linguistic features. Experimental results demonstrate that SALT achieves competitive performance across multiple distillation objectives, outperforming supervised baselines while providing a favorable trade-off between predictive performance and computational efficiency. Moreover, it exhibits strong explainability, accurately identifying key linguistic features and semantic patterns relevant to disease descriptions. These findings highlight the potential of knowledge distillation-based text classification for future applications in early shrimp disease diagnosis and related research directions.
Figures & tables
| # | Vietnamese | English |
|---|---|---|
| 1 | Khi bệnh tiến triển, gan tụy từ màu vàng nhạt sang trắng đục, ban đầu sưng nhưng teo dần theo thời gian. Trên bề mặt có nhiều đốm đen, ruột tôm trống không, cơ thể suy yếu và chết hàng loạt. | As the disease progresses, the hepatopancreas changes from pale yellow to opaque white, initially swollen but gradually shrinking over time. Numerous black spots appear on the surface, the intestine becomes empty, and the shrimp’s body weakens, leading to mass mortality. |
| 2 | Tôm bị bệnh có ruột yếu, thức ăn không hấp thụ hết, phân chứa nhiều chất nhầy, nổi trên bề mặt ao. | Diseased shrimp have fragile intestines, incomplete feed absorption, and mucus-laden feces that float on the pond surface. |
| 3 | Tôm bơi yếu, tập trung gần mép bờ và có dấu hiệu cơ thịt hơi đục khi quan sát. | Shrimp exhibit weak swimming behavior, gather near the pond edges, and display slightly opaque musculature upon observation. |
| 4 | Quan sát thấy tôm có rong bám ở mang, vỏ nhớt, tôm chết rải rác dưới đáy ao, số lượng tăng dần theo mức độ bệnh. | Shrimp are observed with algal attachments on the gills, a mucous carapace, and scattered mortalities on the pond bottom, which increase in frequency as the disease progresses. |
| 5 | Tôm bơi không linh hoạt, thường tụ lại ở các góc ao hoặc vùng nước cạn, mang có dấu hiệu phì đại, kèm theo màu nâu hoặc đen dần rõ rệt. | Shrimp swim sluggishly, often cluster in pond corners or shallow areas. The gills appear hypertrophied, with a gradual darkening to brown or black coloration. |
| 6 | Quan sát thấy tôm khó lột xác, chân bơi bị bao phủ bởi lông tơ, mang có màu nâu hoặc đen. | Molting becomes difficult; pleopods are covered with fine filaments, and the gills turn brown or black. |
Table 1: Vietnamese symptom descriptions and their English translations.
Figure 1: Data preprocessing pipeline for disease descriptions.
Figure 2: Text transformation of a specific description under our pipeline.
Figure 3: Inter-class semantic similarity matrix based on the average pairwise cosine similarity of sentence embeddings.
| Class | Train | Valid | Test | Total |
| Acute Hepatopancreatic Necrosis Disease | 56 | 19 | 19 | 94 |
| White Feces Disease | 60 | 20 | 20 | 100 |
| White Spot Disease | 60 | 20 | 20 | 100 |
| Luminous Bacteria Disease | 58 | 19 | 19 | 96 |
| Loose Shell Syndrome | 59 | 19 | 20 | 98 |
| Black Gill Disease | 60 | 20 | 20 | 100 |
Table 2: Data statistics of the dataset after preprosessing.
Figure 4: Workflow for constructing the augmented training set used in the linguistic augmentation baseline.
Figure 5: Prompt-guided linguistic augmentation pipeline for generating additional shrimp disease descriptions.
| Class | Train | Valid | Test | Total |
| Acute Hepatopancreatic Necrosis Disease | 332 | 19 | 19 | 370 |
| White Feces Disease | 360 | 20 | 20 | 400 |
| White Spot Disease | 350 | 20 | 20 | 390 |
| Luminous Bacteria Disease | 344 | 19 | 19 | 382 |
| Loose Shell Syndrome | 354 | 19 | 20 | 393 |
| Black Gill Disease | 359 | 20 | 20 | 399 |
Table 3: Data statistics of the ShrimpCap dataset with an augmented training set.
Figure 6: Overall framework of model distillation, comparison, and explainability.
Figure 7: Architecture of the student model used in the SALT framework with a truncated PhoBERT encoder and gated fusion of contextual representations.
| Name | Value |
| Batch size | 16 |
| Epoch | 15 |
| Learning rate | 3e-5 |
| Optimizer | AdamW |
| Loss Function | Focal Loss |
Table 4: Training hyperparameters
| Component | Parameter | Value |
| Distillation | 0.8 | |
| Distillation | 0.2 | |
| Distillation | Temperature | 3.0 |
| Teacher (Focal) | 2.5 | |
| Teacher (Focal) | Class weight | Class-balanced |
| Center Loss | 0.003 |
Table 5: Method-specific hyperparameters used in SALT
| Metrics | PhoBERT-base | mBART |
| Acc (%) | 89.08 1.71 | 86.43 2.03 |
| F1 | 0.8899 0.0182 | 0.8647 0.0200 |
| Precision | 0.8948 0.0163 | 0.8691 0.0193 |
| Recall | 0.8903 0.0175 | 0.8637 0.0203 |
| Inference Time (ms) | 13.34 0.64 | 342.16 61.21 |
| Params (M) | 135.01 | 611.91 |
Table 6: Evaluation results of fine-tuned teacher models on the shrimp disease classification validation set.
Figure 8: Accuracy and loss curves on the validation set for the two teacher models: PhoBERT-base and mBART.
Figure 9: Normalized confusion matrices after fine-tuning on the shrimp disease classification dataset for two models: PhoBERT and mBART. Class indices: 0 = Acute Hepatopancreatic Necrosis Disease; 1 = White Feces Disease; 2 = White Spot Disease; 3 = Luminous Bacteria Disease; 4 = Loose Shell Syndrome; 5 = Black Gill Disease; 6 = Filamentous Bacterial Disease; 7 = Vitamin C Deficiency Disease; 8 = Yellow Head Disease; 9 = Taura Syndrome.
| Metric | Acc (%) | F1 | Precision | Recall | Time (ms) |
| KL | 88.67( 2.17) | 0.8880 ( 0.0213) | 0.8997 ( 0.0172) | 0.8872( 0.0219) | 3.78( 0.11) |
| MSE | 88.16 ( 1.22) | 0.8822 ( 0.0134) | 0.8940 ( 0.0144) | 0.8818 ( 0.0130) | 3.77 ( 0.04) |
| JSD | 87.65 ( 2.21) | 0.8773 ( 0.021) | 0.8882 ( 0.0191) | 0.8768 ( 0.0219) | 3.78 ( 0.09) |
| SupCon | 86.02 ( 1.23) | 0.8601 ( 0.0138) | 0.8758 ( 0.0103) | 0.8601 ( 0.0135) | 3.77 ( 0.06) |
| Center | 87.45 ( 2.31) | 0.8743 ( 0.0231) | 0.8810 ( 0.0247) | 0.8745 ( 0.023) | 3.80 ( 0.09) |
| Triplet | 87.35 ( 1.22) | 0.8737 ( 0.0138) | 0.8830 ( 0.0141) | 0.8738 ( 0.0125) | 3.75 ( 0.02) |
Table 7: Performance of student models distilled from PhoBERT-base using different distillation objectives on the test set (The best indicators are in bold).
Figure 10: Normalized confusion matrices for student models distilled from PhoBERT-base using different loss functions: KL, MSE, Cosine, JSD, SupCon, Center, and Triplet. Class indices: 0 = Acute Hepatopancreatic Necrosis Disease; 1 = White Feces Disease; 2 = White Spot Disease; 3 = Luminous Bacteria Disease; 4 = Loose Shell Syndrome; 5 = Black Gill Disease; 6 = Filamentous Bacterial Disease; 7 = Vitamin C Deficiency Disease; 8 = Yellow Head Disease; 9 = Taura Syndrome.
| Metric | SFT | TF-IDF + DNN | F2LLM-v2-80M |
| Acc (%) | 86.53 ( 2.44) | 88.98 ( 0.72) | 82.86 ( 2.03) |
| F1 | 0.8666 ( 0.0250) | 0.8896 ( 0.0071) | 0.8287 ( 0.0201) |
| Precision | 0.8803 ( 0.0280) | 0.9021 ( 0.0062) | 0.8367 ( 0.0193) |
| Recall | 0.8659 ( 0.0246) | 0.8896 ( 0.0071) | 0.8297 ( 0.0200) |
| Inference Time (ms) | 3.91 ( 0.04) | 76.46 ( 0.44) | 5.53 ( 0.13) |
Table 8: Performance comparison of baseline models on the test set (The best indicators are in bold).
Figure 11: Normalized confusion matrices of the student model trained with supervised fine-tuning. Class indices: 0 = Acute Hepatopancreatic Necrosis Disease; 1 = White Feces Disease; 2 = White Spot Disease; 3 = Luminous Bacteria Disease; 4 = Loose Shell Syndrome; 5 = Black Gill Disease; 6 = Filamentous Bacterial Disease; 7 = Vitamin C Deficiency Disease; 8 = Yellow Head Disease; 9 = Taura Syndrome.
Figure 12: Normalized confusion matrices of the TF-IDF + DNN reference model. Class Indices: 0 = Acute Hepatopancreatic Necrosis Disease; 1 = White Feces Disease; 2 = White Spot Disease; 3 = Luminous Bacteria Disease; 4 = Loose Shell Syndrome; 5 = Black Gill Disease; 6 = Filamentous Bacterial Disease; 7 = Vitamin C Deficiency Disease; 8 = Yellow Head Disease; 9 = Taura Syndrome.
Figure 13: Normalized confusion matrix of the F2LLM-v2-80M baseline. Class indices: 0 = Acute Hepatopancreatic Necrosis Disease; 1 = White Feces Disease; 2 = White Spot Disease; 3 = Luminous Bacteria Disease; 4 = Loose Shell Syndrome; 5 = Black Gill Disease; 6 = Filamentous Bacterial Disease; 7 = Vitamin C Deficiency Disease; 8 = Yellow Head Disease; 9 = Taura Syndrome.
Figure 14: Accuracy and Loss Curves on the Validation Set for PhoBERT-large.
Figure 15: Normalized confusion matrices after fine-tuning on the shrimp disease classification dataset for PhoBERT-large. Class indices: 0 = Acute Hepatopancreatic Necrosis Disease; 1 = White Feces Disease; 2 = White Spot Disease; 3 = Luminous Bacteria Disease; 4 = Loose Shell Syndrome; 5 = Black Gill Disease; 6 = Filamentous Bacterial Disease; 7 = Vitamin C Deficiency Disease; 8 = Yellow Head Disease; 9 = Taura Syndrome.
| Metric | Acc (%) | F1 | Precision | Recall |
| KL | 88.06 ( 0.96) | 0.8819 ( 0.0099) | 0.8941 ( 0.0076) | 0.8815 ( 0.0096) |
| MSE | 87.04 ( 1.46) | 0.8708 ( 0.0143) | 0.8797 ( 0.0152) | 0.8706 ( 0.0139) |
| Cosine | 87.76 ( 1.27) | 0.8784 ( 0.0115) | 0.8911 ( 0.0056) | 0.8779 ( 0.0127) |
| JSD | 86.33 ( 1.81) | 0.8646 ( 0.0171) | 0.8740 ( 0.014) | 0.8636 ( 0.0178) |
| SupCon | 85.92 ( 2.85) | 0.8597 ( 0.0282) | 0.8731 ( 0.0263) | 0.8589 ( 0.0285) |
| Triplet | 86.53 ( 1.31) | 0.8647 ( 0.0138) | 0.8765 ( 0.01) | 0.8649 ( 0.0138) |
Table 9: Performance of student models distilled from PhoBERT-large using different distillation objectives on the test set (the best indicators are in bold).
Figure 16: Performance of student models distilled from PhoBERT-large using different distillation objectives on the test set.
Figure 17: Normalized confusion matrices for student models distilled from PhoBERT-large using different distillation objectives: KL, MSE, Cosine, JSD, SupCon, Center, and Triplet. Class Indices: 0 = Acute Hepatopancreatic Necrosis Disease; 1 = White Feces Disease; 2 = White Spot Disease; 3 = Luminous Bacteria Disease; 4 = Loose Shell Syndrome; 5 = Black Gill Disease; 6 = Filamentous Bacterial Disease; 7 = Vitamin C Deficiency Disease; 8 = Yellow Head Disease; 9 = Taura Syndrome.
| Metric | Acc (%) | F1 | Precision | Recall |
| KL | 88.27 ( 0.78) | 0.8833 ( 0.0076) | 0.8957 ( 0.0070) | 0.8828 ( 0.0081) |
| MSE | 87.96 ( 0.85) | 0.8800 ( 0.0085) | 0.8920 ( 0.0090) | 0.8794 ( 0.0089) |
| Cosine | 88.47 ( 1.31) | 0.8855 ( 0.0130) | 0.8967 ( 0.0099) | 0.8848 ( 0.0133) |
| SupCon | 85.10 ( 1.64) | 0.8505 ( 0.0162) | 0.8691 ( 0.0135) | 0.8511 ( 0.0171) |
| Center | 88.88 ( 1.64) | 0.8891 ( 0.0160) | 0.9008 ( 0.0178) | 0.8891 ( 0.0169) |
| Triplet | 88.67 ( 1.64) | 0.8879 ( 0.0163) | 0.8999 ( 0.0157) | 0.8872 ( 0.0165) |
Table 10: Performance of student models without gated fusion distilled from PhoBERT-base on the test set (the best indicators are in bold).
Figure 18: Performance of student models without gated fusion distilled from PhoBERT-base on the test set.
Figure 19: Normalized confusion matrices for student models distilled from PhoBERT-base without the gated fusion module using different distillation objectives. Class indices: 0 = Acute Hepatopancreatic Necrosis Disease; 1 = White Feces Disease; 2 = White Spot Disease; 3 = Luminous Bacteria Disease; 4 = Loose Shell Syndrome; 5 = Black Gill Disease; 6 = Filamentous Bacterial Disease; 7 = Vitamin C Deficiency Disease; 8 = Yellow Head Disease; 9 = Taura Syndrome
Figure 20: Validation accuracy and loss curves on the validation set for the PhoBERT-base teacher trained with linguistic augmentation.
| Metric | Acc (%) | F1 | Precision | Recall |
| KL | 82.55 (±2.07) | 0.8248 (±0.0211) | 0.8312 (±0.0207) | 0.8263 (±0.0207) |
| MSE | 83.88 (±1.88) | 0.8389 (±0.0187) | 0.8429 (±0.0173) | 0.8394 (±0.0188) |
| JSD | 84.39 (±1.71) | 0.8443 (±0.0161) | 0.8510 (±0.0140) | 0.8449 (±0.0167) |
| SupCon | 82.96 (±2.31) | 0.8299 (±0.0243) | 0.8385 (±0.0275) | 0.8306 (±0.0237) |
| Center | 81.63 (±3.58) | 0.8156 (±0.0363) | 0.8222 (±0.0375) | 0.8169 (±0.0359) |
| Triplet | 81.84 (±2.48) | 0.8189 (±0.0238) | 0.8320 (±0.0174) | 0.8194 (±0.0254) |
Table 11: Performance of student models distilled from the linguistically augmented PhoBERT-base teacher on the test set (the best indicators are in bold).
Figure 21: Performance of student models distilled from the linguistically augmented PhoBERT-base teacher on the test set.
Figure 22: Normalized confusion matrices for student models distilled from the linguistically augmented PhoBERT-base teacher using different distillation objectives. Class indices: 0 = Acute Hepatopancreatic Necrosis Disease; 1 = White Feces Disease; 2 = White Spot Disease; 3 = Luminous Bacteria Disease; 4 = Loose Shell Syndrome; 5 = Black Gill Disease; 6 = Filamentous Bacterial Disease; 7 = Vitamin C Deficiency Disease; 8 = Yellow Head Disease; 9 = Taura Syndrome
Figure 23: Number of LIME-Unmatched samples for each distillation objective
Figure 24: Heatmap of LIME-Unmatched samples by class and distillation objective. Class indices: 0 = Acute Hepatopancreatic Necrosis Disease; 1 = White Feces Disease; 2 = White Spot Disease; 3 = Luminous Bacteria Disease; 4 = Loose Shell Syndrome; 5 = Black Gill Disease; 6 = Filamentous Bacterial Disease; 7 = Vitamin C Deficiency Disease; 8 = Yellow Head Disease; 9 = Taura Syndrome
Figure 25: Representative LIME visualizations for the best-performing student model (Cosine KD). One correctly classified example is shown for each disease class: (a) Acute Hepatopancreatic Necrosis Disease; (b) White Feces Disease; (c) White Spot Disease; (d) Luminous Bacteria Disease; (e) Loose Shell Syndrome; (f) Black Gill Disease; (g) Filamentous Bacterial Disease; (h) Vitamin C Deficiency Disease; (i) Yellow Head Disease; and (j) Taura Syndrome
| Metric | KL | MSE | Cosine | JSD | SupCon | Center | Triplet |
| Avg SHAP | 0.0179 | 0.0186 | 0.0097 | 0.0212 | 0.0131 | 0.0168 | 0.0093 |
| SHAP Std. | 0.0396 | 0.0399 | 0.0204 | 0.0457 | 0.0283 | 0.0388 | 0.0174 |
| F1-score | 0.8880 | 0.8822 | 0.8935 | 0.8773 | 0.8601 | 0.8743 | 0.8737 |
Table 12: Comparison of average SHAP score (Avg SHAP), SHAP standard deviation (SHAP Std.), and F1-Score across different distillation objectives
Figure 26: Distribution of SHAP scores per loss function.
Figure 27: Explainability vs. Performance
Figure 28: Keyword deletion faithfulness analysis: (a) Classification accuracy after removing the Top- keywords identified by LIME (Solid Lines) and randomly selected keywords (Dashed Lines), (b) Comprehensiveness is measured as the average reduction in the predicted-class probability after keyword removal.
| Model | Params (M) | Size (MB) | GFLOPs | GPU Latency (ms) | CPU Latency (ms) | Throughput (samples/s) |
| PhoBERT-base (Teacher) | 135.01 | 515.01 | 43.49 | 1.93 | ||
| Student (Cosine KD) | 79.88 | 304.73 | 14.50 | 5.69 | ||
| SFT | 79.88 | 304.73 | 14.50 | 5.69 | ||
| TF-IDF + DNN | 11.45 | 131.05 | - † | 996.21 | ||
| F2LLM-v2-80M | 80.38 | 306.64 | 32.21 | 14.83 |
Table 13: Computational efficiency comparison of the Teacher, Student, and baseline models. † GFLOPs correspond only to the DNN classifier; sparse TF-IDF vectorization is excluded.