We investigate how large language models perform on low-resource languages by benchmarking eight LLMs across five experimental conditions in English, Kazakh, and Mongolian. Using 50 hand-crafted questions spanning factual, reasoning, technical, and culturally grounded categories, we evaluate 2,000 responses on accuracy, fluency, and completeness. We find a consistent performance gap of 13.8-16.7 percentage points between English and low-resource language conditions, with models maintaining surface-level fluency while producing significantly less accurate content. Cross-lingual transfer-prompting models to reason in English before translating back-yields selective gains for bilingual architectures (+2.2pp to +4.3pp) but provides no benefit to English-dominant models. Our results demonstrate that current LLMs systematically underserve low-resource language communities, and that effective mitigation strategies are architecture-dependent rather than universal.
Figures & tables
Condition
Total
Acc.
Flu.
Comp.
Time
Len.
C1: English
90.7
78.5
100.0
93.6
7.3
237
C2: Kazakh Dir.
76.9
65.5
89.0
76.1
14.6
128
C3: Kazakh CLT
77.2
67.1
88.6
76.0
35.4
79
C4: Mongol. Dir.
74.0
62.7
87.5
71.8
25.0
143
C5: Mongol. CLT
74.6
64.8
86.6
72.5
18.5
86
Table 1: Aggregate performance across all eight models per condition. All scores are percentages of maximum possible. Time in seconds, length in words.
Cat.
Cond.
Score
Δ CLT
Time
Len.
Eng.-First
C1
92.2
—
7.2
231
C2
88.2
− 1.1
11.8
130
C3
87.1
11.0
80
C4
87.3
− 0.7
25.4
137
C5
86.7
14.7
89
Bilingual
C1
91.2
—
6.4
251
Table 2: Performance by model category. Eng.-First ( n =250): Claude Opus 4.5, GPT-5.2, Gemini 3 Pro, Grok 4.1, Llama 4 Maverick. Bilingual ( n =100): Qwen3, DeepSeek V3.2. Multiling. ( n =50): Aya Expanse.
Figure 1: Mean score vs. mean response time by model category and condition. The Multilingual C3 point (191.9s, 20.0%) is off-scale and annotated.