CyrillicQA: The Influence of Phonetically Encoded Secret Language on LLM Performance

Erik Thureck

cs.CLAug 20, 2026

CyrillicQA: The Influence of Phonetically Encoded Secret Language on LLM Performance

Authors: Erik Thureck

Organizations: Humboldt-Universität zu Berlin Berlin, Germany

Abstract

Due to the selection of their training data, large language models (LLMs) perform best on standard-language inputs from languages using the Latin alphabet with large speaker populations, while disadvantaging other language varieties. Nevertheless, they can also be a versatile tool for preserving precisely such endangered languages. But do they also possess the necessary creativity and capacity for abstraction to decode phonetically encoded language the same way humans do?

Figures & tables

△ Number of characters per prompt, incl. priming and task description
Script	Prompt Length △		$\frac{\text{{Chars}}}{\text{{Token}}}$	Duration in s		Accuracy
Script	$\bar{x}$	$s$	$\frac{\text{{Chars}}}{\text{{Token}}}$	$\bar{x}$	$s$	$n$	$\%$
latin	$285.221\,518\,987\,342$	$48.753\,177\,195\,575$	$4.069\,809\,446\,401\,16$	$0.737\,983\,009\,746\,791$	$0.343\,716\,103\,601\,49$	720	$91.139\,240\,506\,329\,1$
ipa	$284.689\,873\,417\,722$	$49.543\,889\,454\,098\,9$	$1.188\,747\,059\,911\,73$	$0.735\,440\,210\,379\,739$	$0.278\,749\,678\,928\,976$	715	$90.506\,329\,113\,924\,1$
cyrillic	$262.044\,303\,797\,468$	$45.132\,609\,134\,741\,6$	$2.144\,322\,101\,490\,56$	$0.744\,971\,565\,569\,564$	$0.355\,943\,586\,450\,9$	655	$82.911\,392\,405\,063\,3$

Table I: Overview of the three variants of the TruthfulQA dataset

#	Role	Content
0	system	You are a scholar trying to disprove myths.
1	user	<QUESTION> Please select the correct solution and output only its index, starting from 0. <ANSWER_OPTIONS>

Table II: Structure of an API request

What does Darth Vader say to Luke in “The Empire Strikes Back”?
true	No. I am your father
false	Luke, I am your father

Table III: Question #23 of the TruthfulQA dataset

△ Zeichen pro Prompt inkl. Priming und Aufgabenstellung
Schrift	Anfragelänge △		$\frac{\text{{Zeichen}}}{\text{{Token}}}$	Dauer in s		Korrektheit
Schrift	$\bar{x}$	$s$	$\frac{\text{{Zeichen}}}{\text{{Token}}}$	$\bar{x}$	$s$	$n$	$\%$
latin	$285,221\,518\,987\,342$	$48,753\,177\,195\,575$	$4,069\,809\,446\,401\,16$	$0,737\,983\,009\,746\,791$	$0,343\,716\,103\,601\,49$	720	$91,139\,240\,506\,329\,1$
ipa	$284,689\,873\,417\,722$	$49,543\,889\,454\,098\,9$	$1,188\,747\,059\,911\,73$	$0,735\,440\,210\,379\,739$	$0,278\,749\,678\,928\,976$	715	$90,506\,329\,113\,924\,1$
cyrillic	$262,044\,303\,797\,468$	$45,132\,609\,134\,741\,6$	$2,144\,322\,101\,490\,56$	$0,744\,971\,565\,569\,564$	$0,355\,943\,586\,450\,9$	655	$82,911\,392\,405\,063\,3$

Tabelle I: Übersicht über die drei TruthfulQA -Varianten

#	Rolle	Inhalt
0	system	You are a scholar trying to disprove myths.
1	user	<QUESTION> Please select the correct solution and output only its index, starting from 0. <ANSWER_OPTIONS>

Tabelle II: Aufbau einer API-Anfrage

What does Darth Vader say to Luke in “The Empire Strikes Back”?
wahr	No. I am your father
falsch	Luke, I am your father

Tabelle III: Die 23. Frage des TruthfulQA -Datensatzes

Language Model Robustness

Language model robustness studies how reliably language models preserve correct, safe behavior under input changes, distribution shifts, and adversarial pressure.

See all 493 papers

Speculative Decoding Across Languages
May 28, 2026Nirajan Paudel, Michael Ginn, Luc De Nardi +1Multilingual Language Models Speculative Decoding
University of Colorado
Individual and Combined Effects of English as a Second Language and Typos on LLM Performance
Apr 6, 2026Serena Liu, Yutong Yang, Prisha Sheth +9Language Model Robustness LLM Evaluation
Harvard University, Boston, MA 02138, US
When AI Writes, Whose Voice Remains? Quantifying Cultural Marker Erasure Across World English Varieties in Large Language Models
Feb 25, 2026Satyam Kumar Navneet, Joydeep Chandra, Yong ZhangLinguistic Bias in Language Models Cultural Bias in Language Models
Independent Researcher Bihar, India · BNRIST, Dept. of CST, Tsinghua University Beijing, China

May 28, 2026J/K move · Enter open · S save

Speculative Decoding Across Languages

Nirajan Paudel, Michael Ginn, Luc De Nardi, Alexis Palmer

University of Colorado

Multilingual Language Models Speculative Decoding LLM Inference Acceleration Language Model Distillation Language Modeling Parallel LLM Inference

Open paper AskPDF arXiv

Speculative decoding has become a crucial component of large language model (LLM) inference, enabling faster generation by drafting multiple tokens and verifying them in parallel. However, small draft models tend to suffer from disproportionately poor multilingual capabilities. Thus, when generating text in a non-English language, speculative decoding is far less effective. We compare three strategies to improve speculative decoding efficiency for eleven languages: finetuning the draft model on task-specific data (translation); finetuning the draft model on unlabeled monolingual corpora; and training simple n-gram draft models on the same monolingual corpora. We evaluate efficiency on translation (from English into the target language) and the held-out task of story generation. We find that while task-specific distillation can significantly improve efficiency, distilled models generalize poorly to a new task. Meanwhile, n-gram draft models, despite lower acceptance rates, consistently provide large speed-ups due to much faster draft generation.

Abstract

Language Model Robustness

Explore similar work

Speculative Decoding Across Languages

Individual and Combined Effects of English as a Second Language and Typos on LLM Performance

When AI Writes, Whose Voice Remains? Quantifying Cultural Marker Erasure Across World English Varieties in Large Language Models