cs.CLApr 6, 2026

Chinese Language Is Not More Efficient Than English in Vibe Coding: A Preliminary Study on Token Cost and Problem-Solving Rate

Authors: Simiao Ren, Xingyu Shen, Yuchen Zhou, Dennis, Ng, Ankit Raj

Organizations: Tsang · ∗,†Scam.ai

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

A claim has been circulating on social media and practitioner forums that Chinese prompts are more token-efficient than English for LLM coding tasks, potentially reducing costs by up to 40%. This claim has influenced developers to consider switching to Chinese for ``vibe coding'' to save on API costs. In this paper, we conduct a rigorous empirical study using SWE-bench Lite, a benchmark of software engineering tasks, to evaluate whether this claim of Chinese token efficiency holds up to scrutiny. Our results reveal three key findings: First, the efficiency advantage of Chinese is not observed. Second, token cost varies by model architecture in ways that defy simple assumptions: while MiniMax-2.7 shows 1.28x higher token costs for Chinese, GLM-5 actually consumes fewer tokens with Chinese prompts. Third, and most importantly, we found that the success rate when prompting in Chinese is generally lower than in English across all models we tested. We also measure cost efficiency as expected cost per successful task -- jointly accounting for token consumption and task resolution rate. These findings should be interpreted as preliminary evidence rather than a definitive conclusion, given the limited number of models evaluated and the narrow set of benchmarks tested due to resource constraints; they indicate that language effects on token cost are model-dependent, and that practitioners should not expect cost savings or performance gains just by switching their prompt language to Chinese.

Figures & tables

Figure 1: Graphical abstract showing the research workflow and key findings.
Figure 2: The popular social media claim that Chinese prompts are 40% more token-efficient, motivating this empirical study. This YouTube short has garnered over 2.6 million views and 63k likes, with top comments endorsing the claim. Notably, the original post does not cite any empirical research to support this assertion.
Figure 3: Comprehensive results visualization showing (A) resolution rates by language, and (B) normalized token cost ratio (Chinese/English) where English serves as the 1.0 baseline per model. Values above 1.0 indicate Chinese costs more tokens; values below 1.0 indicate Chinese costs fewer tokens.
Figure 4: ZH/EN token ratio by model and token type (English = 1.0 baseline). Values above 1.0 indicate Chinese requires more tokens; values below 1.0 indicate Chinese requires fewer tokens. GPT-5.4-mini has no reasoning mode.
Figure 5: Token count ratio (Chinese/English) across different tokenizer families. Values above 1.0 indicate Chinese requires more tokens; values below 1.0 indicate Chinese requires fewer tokens.
Figure 6: Character efficiency (chars/token) for English and Chinese across different tokenizer families. Higher values indicate more characters encoded per token, reflecting greater compression efficiency.
Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

English PromptChinese Prompt (original characters)
TSQL - L031 incorrectly triggers “Avoid using aliases in join condition” when no join present Expected Behaviour Both queries should pass; only difference is the addition of a table alias ‘a’: SELECT [hello] FROM mytable and with alias: SELECT a.[hello] FROM mytable AS a Observed Behaviour 1/ passes 2/ fails with: L031: Avoid using aliases in join condition. No join condition present :-) Steps to Reproduce Lint queries above Dialect: TSQL Version: sqlfluff 0.6.9, Python 3.6.9Chinese Prompt Translation
Appendix
Table 1: Side-by-Side Comparison of English and Chinese Prompts for the Same SWE-bench Instance (sqlfluff__sqlfluff-1625)
ModelLangInstResRatePrompt/InstOutput/InstReason/Inst
MiniMax-2.7EN503366.0%298,72061,76222,368
MiniMax-2.7ZH392461.5%382,97479,18228,677
GPT-5.4-miniEN501836.0%84,3472,6950 a
GPT-5.4-miniZH461226.1%91,6812,9290 a
GLM-5EN483164.6%1,417,012112,80387,474
GLM-5ZH492755.1%1,388,094110,50185,688
Appendix
Table 2: Resolution Rates and Token Metrics by Model and Language
TokenizerZH/EN RatioEN chars/tokZH chars/tok
GLM (chatglm3-6b)0.9232.692.16
Qwen (Qwen2-7B)1.0253.572.72
Mistral (Mistral-7B)1.0642.781.96
Llama (cl100k_base)1.1503.722.46
GPT (cl100k_base)1.1503.722.46
Appendix
Table 3: ZH/EN token ratios and character efficiency across five tokenizers on 23 SWE-bench Lite instances.
More in this topic

LLM Inference Efficiency

Large language model inference efficiency reduces the time, memory, energy, and cost of generating useful outputs.

See all 350 papers

Explore similar work

CardsList
Jun 2, 2026

Cross-Lingual Token Arbitrage: Optimizing Code Agent Context Windows via Local LLM Preprocessing

AI-assisted coding agents are bottlenecked by input-token cost. Two pathologies of raw human input drive much of this overhead: tokenization inefficiency for non-English text and structural entropy in conversational prompts. Existing approaches act reactively by compressing already-bloated contexts or intervening after failures occur. We introduce a pre-flight, edge-side prompt-rewriting middleware that operates between the developer and the cloud agent. A local Llama 3.2 (3B) model performs cross-lingual translation into English, structural rewriting into a compact task-oriented format, and regex-validated rewrite-with-fallback safeguards to ensure the optimized prompt is never larger than the original. We evaluate on OMH-Polyglot, a multilingual coding benchmark spanning Turkish, Arabic, Chinese, and code-switched specifications. Across three commercial LLM backends, the middleware reduces prompt tokens by 34-47 percent and total tokens by up to 18.8 percent while preserving or improving task accuracy. Ablation studies show that gains arise primarily from the rewriting stage rather than simple function-name extraction. Compared with LLMLingua-2 at matched compression rates, our method consistently achieves superior OckScore performance across all evaluated backends. These results demonstrate that proactive prompt optimization can substantially reduce inference costs without sacrificing coding quality.

Mehmet Utku Colak

Department of Computer Engineering, Istanbul Technical University · Istanbul, Turkey

AI Coding AgentsLLM PromptingLLM Inference AccelerationPrompt Compression
AskPDF
Aug 26, 2026

Can your AI agent be cheaper? Investigating the effects of task specifications on token spend in agentic coding tasks

Agentic coding workflows are now widely deployed in real-world systems. With long-horizon reasoning and tool use, token usage has become an important consideration for both cost and efficiency. Two engineers using AI will solve the same problem differently. How the specification of a task shapes an agent's token spend, and whether that spend can be predicted in advance, are open questions. Here, we study the effects of different task specifications on agentic token spend with the Kimi K3 model at three thinking efforts. Across 2,7002,700 runs, we show that reducing a full task specification to a bare user story raises token spend by 29.7%29.7\%, while run-to-run variance remains unaffected by any prompt changes. We show that prompt-sensitivity is task-dependent, running from 13%13\% to 115%115\%. We fit a simple predictor that can price a full distribution of task specifications and thinking effort configurations from a single cheap probe on an unseen task within 36%36\%, improving over prior work in predicting token spend. Our work provides initial results quantifying the effects of task specification on agentic token spend and introduces a method that can be used to systematically evaluate the cost of AI coding workflows.

Jakub Smékal

Stanford University

Coding AgentsAgentic Code GenerationCost-Aware InferencePrompt Sensitivity
AskPDF
Oct 14, 2026

Does the Linguistic Register of Thinking Matter? Wenyan as a Compact Reasoning Register for Chinese LLMs

Visible reasoning traces of Chinese reasoning-oriented LLMs are typically written in modern vernacular Chinese, which may be verbose under limited decoding budgets. We ask whether the linguistic register of reasoning can be used to improve efficiency. We propose Wenyan Thinking, which emits intermediate reasoning in Classical Chinese (wenyanwen) while retaining a short direct-answer output. Starting from modern-Chinese reasoning traces, we rewrite only the reasoning field into wenyan and fine-tune Qwen3-4B-Thinking-2507 on the rewritten supervision. To distinguish register shift from generic compression, we construct a compact modern-Chinese control using the same source data, teacher family, base model, and SFT recipe. Across budget-sensitive evaluations on MMLU-ProX-Lite-ZH, CEVAL, and AGIEval-CN, Wenyan Thinking substantially reduces reasoning length and missing-answer rates under constrained decoding. Compact rewriting explains much of the efficiency gain, while wenyan consistently outperforms the compact modern-Chinese control on the primary benchmark and remains competitive across the additional benchmarks. These results identify linguistic register as a practical design axis for efficient visible reasoning in Chinese LLMs.

Minghan Li, Guodong Zhou

Soochow University, Suzhou, China

Efficient Language Model ReasoningLanguage ModelingCoT Compression
AskPDF