Language-Conditioned Token and Reasoning Efficiency in Large Language Models: A Paired Cross-Lingual Study Protocol
Authors: Genliang Zhu, Chu Wang
Organizations: Accentrust, Vancouver, Canada · Georgia Institute of Technology, Atlanta, Georgia, USA · University of Illinois Urbana-Champaign, Urbana, Illinois, USA
Large language models incur language-dependent representation and inference costs, but existing comparisons often conflate input language, assigned observable-trace language, and answer realization. We specify a prospective paired study that separates these interfaces while holding the semantic item, checkpoint, and answer oracle fixed. The initial design instantiates 240 exactly scored items rendered from templates in English and seven non-English languages, three distinct-lineage open-weight checkpoints, three trace-token budgets, 22 input- and trace-language conditions, a fixed answer reserve, and a separately counted delimiter: 47,520 initial core runs before prospective sample-size selection. RQ1-RQ3 estimate input and trace effects by intention-to-treat with failure-inclusive terminal accounting and test answer realization by cloning a sealed prefix and runtime-native KV state into eight crossed branches. Pre-freeze independent language review, fixed-form ASCII selectors, and code/surface/solver agreement constrain the realization test. H1-H5 share one Holm family and a global simultaneous component band. A secondary randomized experiment compares one-long-attempt and complete K-short-attempt policies at equal trace allowance under frozen seeds and oracle-blind aggregation; it is a full-policy contrast because answer capacity differs. Outcomes include exact token spans, correctness, latency, runtime-exposed memory, and qualified same-host operating-system-reported energy over prespecified hardware rails. The protocol separates tokenizer expansion, observable-trace cost, and answer-realization cost without treating visible traces as internal cognition or operating-system estimates as physical cross-device energy. No confirmatory model outcome is reported; result fields remain disabled until the frozen evidence ledger passes independent verification.
Figures & tables
Figure 1. The planned design separates task presentation, assigned observable trace language, and answer realization. Input- and trace-language conditions form a complete paired schedule at all three assigned trace budgets per item–checkpoint block; execution order is randomized. Stage B fully crosses all eight answer languages after an exact parent-prefix/KV clone and randomizes only branch order. A left-to-right pipeline begins with a language-independent semantic item, followed by paired input-language and assigned-trace-language conditions under each of three assigned observable trace-token budgets. The RQ1–RQ2 answer is canonical; primary Stage B clones one exact parent prefix and KV state before applying eight fixed-form branch-local language selectors in randomized execution order. Below the stages are token, timing, OS-reported energy, compliance, and correctness measures.
Cell
Scope
Lin
assigned Ltrace
Lout
C1
per ℓ
ℓ
ℓ
canonical
C2
per ℓ
ℓ
English
canonical
C3
per ℓ
English
ℓ
canonical
C4
shared
English
English
canonical
Table 1. Primary schedule. C1–C3 are run for each non-English language ℓ ; C4 is one shared baseline.
Figure 2. RQ3 state, intervention, and denominator boundaries. The shared codebook is sealed before branching; the eight clones execute in randomized order, and only the fixed selector and answer are newly processed per branch. Every denominator is reported, and ineligible sampled parents are retained in attrition rather than replaced. The top row shows scientific admission, trace generation, delimiter and prefix/KV sealing with a shared codebook, eight copy-on-write clones, a fixed selector and answer, and terminal accounting. The bottom row shows target-frame parents, sampled parents, scientific-entry parents, exact-state-eligible parents, assigned branches, and terminal branches with a separately identified energy-observed subset.
Large language models (LLMs) are increasingly deployed in multilingual settings, yet the energy costs of serving these models across different languages remain poorly understood. We present a systematic study of inference energy consumption across languages with ML.Energy framework (Chung et al., 2026). We find striking disparities: energy consumption per output token varies by up to 8.3 times across languages, while total energy for a fixed set of requests varies by up to 179 times between the cheapest (English, 17.6 kJ) and the most expensive (Pashto, 3,147 kJ) languages. Our analysis shows that this disparity is driven by two compounding factors: (1) higher per-token energy costs for languages using complex or rare scripts, and (2) more tokens generated for low-resource languages. Moreover, we find a double cost + performance penalty: languages with the highest energy footprints also tend to achieve the lowest task accuracy. We reveal that the energy divide persists across models, hardware, and tasks, suggesting a systemic energy inequity in multilingual LLM deployment. Finally, we recommend that the community treat energy as a first-class evaluation axis, extend reporting checklists and model cards to include it, and adopt deployment-side mitigations for better energy efficiency.
Naihao Deng, Alissa Shen, Yiming Feng +5
University of Michigan · University of Cambridge · University of Aberdeen
Large Language Models (LLMs) solve many reasoning tasks via chain-of-thought (CoT) prompting, but smaller models (about 7 to 8B parameters) still struggle with multi-step reasoning under tight compute and token budgets. Existing test time reasoning methods such as self consistency (sampling multiple rationales and voting), Tree-of-Thoughts (search over intermediate thoughts), and critique revise loops improve performance, but often at high token cost and without fine-grained step-level control. This project1 aims to address that gap: can Small Language Models (SLMs) reason reliably using the same or fewer tokens? This question is both scientific and practical. Scientifically, it probes whether process supervision and simple test-time controls (such as token budgets and rejection of redundant steps) can substitute for model scale or large sampling counts. Practically, many deployments (on-device, low-latency, or cost-constrained settings) cannot afford huge models or dozens of sampled rationales per query. A method that improves SLM reasoning at fixed cost would therefore be directly useful.
Chain-of-Thought (CoT) faithfulness matters for monitoring, explaining and safely deploying large language models, yet it has been studied mostly in English. Languages differ in level, the amount of text available to learn from, so models know less in lower-resource languages, which may change how much they rely on their reasoning. To our knowledge, we are the first to relate CoT faithfulness to both resource level and task type (knowledge- or reasoning-intensive). On parallel multiple-choice questions from Multilingual Massive Multitask Language Understanding (MMMLU) in eight languages, faithfulness rises as resource level falls on knowledge-intensive tasks but not on reasoning-intensive ones, a gap that points the same way across four models and several faithfulness measures. We propose, and find support for, the Reliance Hypothesis: models rely more on their reasoning when they cannot recall the answer, as in lower-resource languages on knowledge-intensive tasks. Model confidence partly explains the effect of resource level, but not the knowledge-reasoning gap. Languages in which models are less confident rely more on reasoning, yet less confident answers within a language do not, a Simpson's paradox. In lower-resource languages, models rely on their reasoning for more answers, not more strongly for each. CoT monitoring may therefore be most informative in lower-resource languages, where it is least often validated.
Lucas Resck, Jingyi Sun, Qianli Wang +2
Language Technology Lab, University of Cambridge · University of Copenhagen · Technische Universität Berlin +1