Compression Effects on Low-Resource Multilingual Translation
Organizations: Independent Researcher / Arewa Data Science / Aboki AI, Kano, Nigeria
Abstract
We present a pilot study characterising the effect of post-training quantisation on multilingual machine translation quality and inference cost for three African languages - Hausa, Swahili, and Yoruba - across three model families: a small general-purpose LLM (Qwen2.5-1.5B), a larger multilingual instruction-tuned LLM (Aya-23-8B), and a dedicated multilingual MT model (NLLB-200-distilled-1.3B). Evaluating on the FLORES-200 devtest set (1,012 sentences per direction) under FP16, bitsandbytes INT8, and NF4 INT4 precisions on consumer-grade NVIDIA T4 GPUs, we find: (i) the specialised MT model outperforms both LLMs by 32.7-43.3 chrF on the Hausa and Swahili directions, and by 12.9-24.7 chrF on Yoruba, despite being roughly a sixth the size of the 8B instruction-tuned LLM; (ii) post-training quantisation preserves aggregate translation quality within approximately 1 chrF across all model families and language directions, with a single worst case of 2.19 chrF; (iii) on consumer GPUs, INT8 and INT4 quantisation provide substantial memory savings (49-61% at INT4) but do not reduce inference latency relative to FP16 - in fact, latency increases by 1.3x to 5.3x; (iv) a clear performance gradient tracking training-resource availability is visible across languages (Swahili > Hausa > Yoruba). These findings motivate a research agenda focused on hardware-aware compression and error-type analysis beyond aggregate metrics for low-resource deployment.
Figures & tables
| Model | Size | Architecture | Coverage of our languages |
|---|---|---|---|
| Qwen/Qwen2.5-1.5B- Instruct | 1.5B | Causal LM | None of the three languages is a stated training focus. |
| CohereForAI/aya-23-8B | 8B | Causal LM, multilingual instruction-tuned | Aya-23 covers 23 languages; Hausa, Swahili, and Yoruba are not among them. |
| facebook/nllb-200- distilled-1.3B | 1.3B | Seq2seq, dedicated MT | All three languages ( hau_Latn , swh_Latn , yor_Latn ) are supported with real training data. |
| Model | Precision | hau eng | eng hau | swh eng | eng swh | yor eng | eng yor |
|---|---|---|---|---|---|---|---|
| Qwen2.5-1.5B | FP16 | 23.28 | 16.85 | 24.19 | 24.39 | 22.90 | 12.74 |
| Qwen2.5-1.5B | INT8 | 23.33 | 18.91 | 24.16 | 24.09 | — | — |
| Qwen2.5-1.5B | INT4 | 22.49 | 17.33 | 23.46 | 23.34 | — | — |
| Aya-23-8B | FP16 | 23.35 | 15.48 | 29.73 | 16.73 | 19.88 | 5.27 |
| Aya-23-8B | INT8 | 23.12 | 15.15 | 29.45 | 16.53 | — | — |
| Aya-23-8B | INT4 | 22.23 | 15.61 | 27.54 | 16.15 | — | — |
| INT8 FP16 | INT4 FP16 | |||||
|---|---|---|---|---|---|---|
| Direction | Qwen-1.5B | Aya-23-8B | NLLB-1.3B | Qwen-1.5B | Aya-23-8B | NLLB-1.3B |
| hau eng | ||||||
| eng hau | ||||||
| swh eng | ||||||
| eng swh | ||||||
| Model | Precision | Peak GPU memory | Saving vs FP16 | Latency (s/sent) | vs FP16 |
|---|---|---|---|---|---|
| Qwen-1.5B | FP16 | 3.64 GB | — | 0.124 | — |
| Qwen-1.5B | INT8 | 2.41 GB | 34% | 0.511 | 4.13 slower |
| Qwen-1.5B | INT4 | 1.70 GB | 53% | 0.161 | 1.30 slower |
| Aya-23-8B | FP16 | 17.14 GB | — | 0.659 | — |
| Aya-23-8B | INT8 | 10.01 GB | 42% | 1.271 | 1.93 slower |
| Aya-23-8B | INT4 | 6.64 GB | 61% | 0.850 | 1.29 slower |
Appendix figures & tables1 asset
Supplementary material from the paper’s appendix.
Appendix
| Model | Prec. | Direction | BLEU | spBLEU | chrF | chrF++ | Len. ratio | s/sent |
|---|---|---|---|---|---|---|---|---|
| Qwen2.5-1.5B | FP16 | hau eng | 3.30 | 4.96 | 23.28 | 20.98 | 1.038 | 0.085 |
| FP16 | eng hau | 1.09 | 1.56 | 16.85 | 14.59 | 0.846 | 0.163 | |
| FP16 | swh eng | 4.02 | 5.23 | 24.19 | 22.00 | 0.983 | 0.080 | |
| FP16 | eng swh | 1.97 | 3.05 | 24.39 | 21.55 | 1.132 | 0.167 | |
| FP16 | yor eng | 2.82 | – | 22.90 | – | – | 0.402 | |
| FP16 | eng yor | 0.67 | – | 12.74 | – | – | 1.320 |
Low-Resource MT
Low-resource machine translation develops translation systems for languages with scarce parallel text, tools, or standardized evaluation.
See all 97 papers