Choosing the Right Language Mode at Inference Time for Multilingual Reliability
Organizations: PortNLP, Portland State University, USA
Abstract
Multilingual large language models often struggle to reason in low- to mid-resource languages. Prior work has shown that translation can improve multilingual reasoning by helping models access stronger English-centric representations. This raises a central question: How much translation is needed for multilingual large language models to reason reliably, and when does more translation instead trigger interference and overconfidence? Using LLaMA and Qwen models, we run extensive experiments varying text scope and language mode (target-only, English-only, bilingual) to evaluate both accuracy and reliability. Our results reveal a clear trade-off: English context often improve understanding and recover errors caused by non-English comprehension, yet adding redundant bilingual context intensifies interference. We address this trade-off with Reliability-Aware Adaptive Inference (RAAI), a training-free test-time framework that (i) performs Expected Calibration Error (ECE)-aware routing and prompt fusion, and (ii) uses a mid-layer Risk Index (RI) to gate sequential reasoning, allocating compute only when it is likely to help and suppressing harmful bilingual redundancy. Across two model families, RAAI enhances accuracy by 25-37.7% on low-resource languages and lowers calibration error by approximately 3-6%, with the most pronounced benefits in the lowest-resource language tiers.
Figures & tables
| Prior work | Language Mode | Mapped Condition |
| Target-language multilingual QA ( Bandarkar et al., 2025 ; Yang et al., 2023 ) | T | Question, passage, and choices remain in the target dialect throughout, with no English support at inference (our T-mode baseline) |
| English-pivot / translate-test ( Shi et al., 2023 ; Qin et al., 2023 ; Zhu et al., 2024 ) | EN | All input fields are translated into English and scored with the same decoding scheme as T, isolating the effect of full English pivoting (our EN-mode) |
| Bilingual prompting ( Shi et al., 2023 ; Zhu et al., 2024 ; Huang et al., 2023 ) | TEN | Target and English fields are provided jointly, with translation scope varied independently across Q/P/QP/C/All (our TEN-mode) |
| LLaMA 3.1-8B Instruct | Qwen 3-4B | |||||||||||
| Resource | T | EN | TEN | T | EN | TEN | ||||||
| Acc | ECE | Acc | ECE | Acc | ECE | Acc | ECE | Acc | ECE | Acc | ECE | |
| Belebele | ||||||||||||
| High | 81.26 | 9.15 | 81.79 | 10.05 | 81.78 | 8.75 | 80.28 | 9.14 | 87.73 | 7.43 | 88.69 | 8.25 |
| Mid | 68.88 | 14.05 | 65.15 | 12.42 | 61.79 | 9.81 | 66.28 | 10.67 | 86.55 | 8.64 | 79.86 | 10.92 |
| Low | 44.59 | 12.36 | 44.30 | 12.81 | 44.30 | 14.05 | 45.95 | 9.42 | 85.44 | 6.48 | 81.65 | 9.31 |
| LLaMA 3.1-8B-Instruct | Qwen3-4B | |||||||||||||||
| Tier | T Static | Best Static | RouteGate | SeqGate | T Static | Best Static | RouteGate | SeqGate | ||||||||
| Acc | ECE | Acc | ECE | Acc | ECE | Acc | ECE | Acc | ECE | Acc | ECE | Acc | ECE | Acc | ECE | |
| Belebele | ||||||||||||||||
| High | 81.26 | 9.15 | 81.79 | 10.05 | 83.58 | 5.44 | 82.58 | 6.52 | 80.28 | 9.14 | 88.69 | 8.25 | 83.77 | 5.19 | 88.54 | 7.69 |
| Mid | 68.88 | 14.05 | 68.88 | 14.05 | 73.19 | 6.56 | 67.68 | 10.62 | 66.28 | 10.67 | 86.55 | 8.64 | 69.33 | 10.76 | 62.96 | 11.68 |
| Low | 44.59 | 12.36 | 44.59 | 12.36 | 55.19 | 7.71 | 69.70 | 11.34 | 45.95 | 9.42 | 85.44 | 6.48 | 71.14 | 13.19 | 83.64 | 8.35 |
| LLaMA 3.1-8B-Instruct | Qwen3-4B | |||||||||
| Tier | N | MedErr | AUROC | N | MedErr | AUROC | ||||
| Belebele | ||||||||||
| High | -0.54 (0.013) | +0.55 (0.012) | 20 | 17.88 | 0.73 | -0.29 (0.207) | -0.13 (0.583) | 20 | 18.64 | 0.58 |
| Mid | -0.21 (0.457) | -0.21 (0.457) | 15 | 31.25 | 0.50 | -0.55 (0.033) | +0.75 (0.001) | 15 | 30.00 | 0.60 |
| Low | -0.58 (0.275) | +0.68 (0.275) | 10 | 55.41 | 0.72 | -0.87 (0.001) | +0.74 (0.014) | 10 | 54.05 | 1.00 |
| MMLU-ProX-Lite | ||||||||||
| Example 1 (Bengali): T works, EN fails | |
| (P) | Vision, or the ability to see depends on visual system sensory organs or eyes … |
| (Q) | What is the ability to see dependent upon? |
| (C) | A) Visual system sensory organs B) The requirements of an organism C) A sensitivity to certain wave-lengths D) Varying degrees of acuity |
| Ans. | A |
| Pred. | T = A (correct); EN = blank/invalid; TEN = A (correct). |
| Example 2 (Yoruba): T fails, EN fails, TEN works | |
Appendix figures & tables30 assets
Supplementary material from the paper’s appendix.
Appendix
| Resource | T | EN | TEN | |||||||||
| Acc | Conf | ECE | Ent | Acc | Conf | ECE | Ent | Acc | Conf | ECE | Ent | |
| High | 81.26 | 76.58 | 9.15 | 0.64 | 81.79 | 74.75 | 10.05 | 0.70 | 81.78 | 76.34 | 8.75 | 0.66 |
| Mid | 68.88 | 60.01 | 14.05 | 1.00 | 65.15 | 63.11 | 12.42 | 0.93 | 61.79 | 64.67 | 9.81 | 0.90 |
| Low | 44.59 | 39.38 | 12.36 | 1.29 | 44.30 | 47.36 | 12.81 | 1.20 | 44.30 | 45.67 | 14.05 | 1.22 |
| Resource | T | EN | TEN | |||||||||
| Acc | Conf | ECE | Ent | Acc | Conf | ECE | Ent | Acc | Conf | ECE | Ent | |
| High | 34.65 | 38.78 | 11.02 | 1.77 | 31.02 | 40.50 | 13.19 | 1.74 | 38.84 | 33.43 | 12.16 | 1.92 |
| Mid | 25.64 | 33.70 | 13.13 | 1.91 | 35.19 | 34.13 | 10.44 | 1.91 | 31.57 | 27.27 | 11.67 | 2.05 |
| Low | 23.47 | 29.72 | 10.02 | 1.99 | 29.35 | 32.83 | 13.10 | 1.95 | 33.70 | 28.44 | 13.46 | 2.05 |
| Resource | T | EN | TEN | |||||||||
| Acc | Conf | ECE | Ent | Acc | Conf | ECE | Ent | Acc | Conf | ECE | Ent | |
| High | 80.28 | 82.77 | 9.14 | 0.55 | 87.73 | 85.78 | 7.43 | 0.50 | 88.69 | 85.20 | 8.25 | 0.48 |
| Mid | 66.28 | 63.51 | 10.67 | 0.95 | 86.55 | 87.87 | 8.64 | 0.43 | 79.86 | 86.81 | 10.92 | 0.40 |
| Low | 45.95 | 45.67 | 9.42 | 1.22 | 85.44 | 86.72 | 6.48 | 0.46 | 81.65 | 78.81 | 9.31 | 0.61 |
| Resource | T | EN | TEN | |||||||||
| Acc | Conf | ECE | Ent | Acc | Conf | ECE | Ent | Acc | Conf | ECE | Ent | |
| High | 37.78 | 45.37 | 13.83 | 1.71 | 36.27 | 44.47 | 13.02 | 1.71 | 45.12 | 49.33 | 15.06 | 1.58 |
| Mid | 32.67 | 40.35 | 13.82 | 1.79 | 34.52 | 43.22 | 16.69 | 1.73 | 46.38 | 55.70 | 21.51 | 1.41 |
| Low | 28.57 | 38.15 | 12.81 | 1.83 | 35.87 | 44.28 | 18.24 | 1.72 | 44.57 | 55.30 | 17.63 | 1.43 |
| Resource | T | TEN | RouteGate | |||||||||
| Acc | Conf | ECE | Ent | Acc | Conf | ECE | Ent | Acc | Conf | ECE | Ent | |
| High | 81.26 | 76.58 | 9.15 | 0.64 | 83.23 | 78.20 | 8.55 | 0.61 | 83.58 | 79.92 | 5.44 | 0.59 |
| Mid | 68.88 | 60.01 | 14.05 | 1.00 | 71.52 | 68.11 | 11.10 | 0.82 | 73.19 | 74.00 | 6.56 | 0.74 |
| Low | 44.59 | 39.38 | 12.36 | 1.29 | 55.22 | 52.58 | 10.83 | 1.10 | 55.19 | 58.90 | 7.71 | 1.01 |
| Resource | T | TEN | RouteGate | |||||||||
| Acc | Conf | ECE | Ent | Acc | Conf | ECE | Ent | Acc | Conf | ECE | Ent | |
| High | 34.65 | 38.78 | 11.02 | 1.77 | 31.29 | 39.07 | 13.92 | 1.78 | 35.92 | 34.28 | 5.42 | 1.91 |
| Mid | 25.64 | 33.70 | 13.13 | 1.91 | 30.75 | 31.89 | 10.09 | 1.96 | 27.95 | 36.62 | 3.38 | 2.07 |
| Low | 23.47 | 29.72 | 10.02 | 1.99 | 31.10 | 30.39 | 13.04 | 2.00 | 28.62 | 37.57 | 4.00 | 2.06 |
| Resource | T | TEN | RouteGate | |||||||||
| Acc | Conf | ECE | Ent | Acc | Conf | ECE | Ent | Acc | Conf | ECE | Ent | |
| High | 80.28 | 82.77 | 9.14 | 0.55 | 84.21 | 83.88 | 8.87 | 0.52 | 83.77 | 84.70 | 5.19 | 0.52 |
| Mid | 66.28 | 63.51 | 10.67 | 0.95 | 72.84 | 73.38 | 9.98 | 0.73 | 69.33 | 79.72 | 10.76 | 0.61 |
| Low | 45.95 | 45.67 | 9.42 | 1.22 | 56.44 | 59.08 | 10.92 | 0.99 | 71.14 | 62.78 | 13.19 | 0.96 |
| Resource | T | TEN | RouteGate | |||||||||
| Acc | Conf | ECE | Ent | Acc | Conf | ECE | Ent | Acc | Conf | ECE | Ent | |
| High | 37.78 | 45.37 | 13.83 | 1.71 | 36.83 | 42.84 | 13.28 | 1.75 | 42.04 | 46.45 | 8.44 | 1.65 |
| Mid | 32.67 | 40.35 | 13.82 | 1.79 | 34.66 | 41.77 | 14.37 | 1.77 | 41.78 | 47.28 | 11.09 | 1.61 |
| Low | 28.57 | 38.15 | 12.81 | 1.83 | 34.86 | 41.48 | 16.39 | 1.78 | 38.04 | 47.87 | 10.65 | 1.61 |
| Resource | T | TEN | SeqGate | |||||||||
| Acc | Conf | ECE | Ent | Acc | Conf | ECE | Ent | Acc | Conf | ECE | Ent | |
| High | 81.26 | 76.58 | 9.15 | 0.64 | 84.79 | 78.05 | 8.43 | 0.62 | 82.58 | 78.52 | 6.52 | 0.61 |
| Mid | 68.88 | 60.01 | 14.05 | 1.00 | 72.87 | 64.70 | 11.99 | 0.91 | 67.68 | 66.93 | 10.62 | 0.86 |
| Low | 44.59 | 39.38 | 12.36 | 1.29 | 50.68 | 47.61 | 6.47 | 1.18 | 69.70 | 66.13 | 11.34 | 0.88 |
| Resource | T | TEN | SeqGate | |||||||||
| Acc | Conf | ECE | Ent | Acc | Conf | ECE | Ent | Acc | Conf | ECE | Ent | |
| High | 34.65 | 38.78 | 11.02 | 1.77 | 32.16 | 35.29 | 12.51 | 1.88 | 33.76 | 34.83 | 10.59 | 1.88 |
| Mid | 25.64 | 33.70 | 13.13 | 1.91 | 24.67 | 27.17 | 13.17 | 2.06 | 33.73 | 24.61 | 8.76 | 2.12 |
| Low | 23.47 | 29.72 | 10.02 | 1.99 | 34.12 | 28.46 | 8.83 | 2.04 | 34.12 | 25.32 | 9.16 | 2.10 |
| Resource | T | TEN | SeqGate | |||||||||
| Acc | Conf | ECE | Ent | Acc | Conf | ECE | Ent | Acc | Conf | ECE | Ent | |
| High | 80.28 | 82.77 | 9.14 | 0.55 | 81.47 | 83.41 | 8.17 | 0.52 | 88.54 | 83.36 | 7.69 | 0.54 |
| Mid | 66.28 | 63.51 | 10.67 | 0.95 | 69.14 | 68.48 | 8.74 | 0.84 | 62.96 | 71.76 | 11.68 | 0.77 |
| Low | 45.95 | 45.67 | 9.42 | 1.22 | 49.32 | 51.00 | 10.70 | 1.13 | 83.64 | 71.71 | 8.35 | 0.78 |
| Resource | T | TEN | SeqGate | |||||||||
| Acc | Conf | ECE | Ent | Acc | Conf | ECE | Ent | Acc | Conf | ECE | Ent | |
| High | 37.78 | 45.37 | 13.83 | 1.71 | 41.00 | 44.76 | 17.77 | 1.71 | 47.13 | 42.47 | 10.41 | 1.77 |
| Mid | 32.67 | 40.35 | 13.82 | 1.79 | 38.70 | 44.01 | 13.52 | 1.71 | 51.64 | 37.57 | 14.04 | 1.87 |
| Low | 28.57 | 38.15 | 12.81 | 1.83 | 36.96 | 45.04 | 17.72 | 1.69 | 51.36 | 37.89 | 13.49 | 1.86 |
| Model | Dataset | Tier | #Langs | RouteGate Acc vs. T [95% CI] | SeqGate Acc vs. T [95% CI] | |
| LLaMA | Belebele | High | 4 | 317 | +2.32 [-3.60, +8.24] | +1.32 [-4.67, +7.31] |
| LLaMA | Belebele | Mid | 3 | 239 | +4.31 [-3.81, +12.43] | -1.20 [-9.54, +7.14] |
| LLaMA | Belebele | Low | 2 | 158 | +10.60 [-0.36, +21.56] | +25.11 [+14.55, +35.67] |
| LLaMA | MMLU-ProX-Lite | High | 4 | 189 | +1.27 [-8.36, +10.90] | -0.89 [-10.45, +8.67] |
| LLaMA | MMLU-ProX-Lite | Mid | 3 | 146 | +2.31 [-7.85, +12.47] | +8.09 [-2.35, +18.53] |
| LLaMA | MMLU-ProX-Lite | Low | 2 | 92 | +5.15 [-7.51, +17.81] | +10.65 [-2.34, +23.64] |
| Variant | High-Res. err/AUROC | Mid-Res. err/AUROC | Low-Res. err/AUROC | err average | AUROC average |
| RF only | 0.43 / 0.94 | 0.57 / 0.86 | / 0.32 | 0.28 | 0.71 |
| ECE only | 0.48 / 0.91 | 0.27 / 0.68 | 0.80 / 1.00 | 0.51 | 0.86 |
| Entropy only | 0.62 / 0.91 | 0.45 / 0.79 | 0.58 / 0.76 | 0.55 | 0.82 |
| RF ECE | 0.49 / 0.96 | 0.40 / 0.79 | 0.42 / 0.68 | 0.44 | 0.81 |
| RF Entropy | 0.62 / 1.00 | 0.64 / 0.89 | / 0.36 | 0.40 | 0.75 |
| ECE Entropy | 0.56 / 0.91 | 0.31 / 0.71 | 0.56 / 0.84 | 0.48 | 0.82 |
| Model / Dataset | Top 25% | Top 50% | Top 75% | Top 100% |
| LLaMA / Belebele | +3.27 / | +6.54 / | +10.73 / | +10.93 / |
| LLaMA / MMLU-ProX-Lite | +1.27 / +0.63 | +1.45 / +0.11 | +4.45 / +0.54 | +4.41 / +0.27 |
| Qwen / Belebele | +2.12 / +1.22 | +3.62 / +0.86 | +5.44 / +0.52 | +5.43 / +0.87 |
| Qwen / MMLU-ProX-Lite | +0.80 / +1.99 | +0.92 / +2.27 | +1.43 / +1.94 | +10.54 / +1.49 |
| Model | Dataset | Tier | RI threshold | RouteGate routed | SeqGate routed | RouteGate cost ( T) | SeqGate cost ( T) |
| LLaMA | Belebele | High | 0.500 | 19.8% | 35.1% | 1.35 | 1.75 |
| LLaMA | Belebele | Mid | 0.315 | 37.8% | 55.0% | 1.62 | 2.00 |
| LLaMA | Belebele | Low | 0.115 | 83.7% | 90.3% | 2.43 | 2.00 |
| LLaMA | MMLU-ProX-Lite | High | 0.500 | 3.3% | 40.0% | 1.03 | 2.00 |
| LLaMA | MMLU-ProX-Lite | Mid | 0.315 | 15.5% | 60.0% | 1.04 | 2.00 |
| LLaMA | MMLU-ProX-Lite | Low | 0.115 | 35.8% | 89.0% | 1.06 | 2.00 |
Cross-Lingual Reasoning in Language Models
Cross-lingual reasoning in language models studies how models solve problems and transfer knowledge across languages, scripts, and cultures.
See all 69 papers