Is continuous CoT better suited for multi-lingual reasoning?

Ali Hamza Bashir; Behzad Shomali; Markus Frey; Mehdi Ali; Rafet Sifa; David Berghaus

cs.CLMar 9, 2026

Is continuous CoT better suited for multi-lingual reasoning?

Authors: Ali Hamza Bashir, Behzad Shomali, Markus Frey, Mehdi Ali, Rafet Sifa, David Berghaus

Organizations: Lamarr Institute · Fraunhofer IAIS · University of Bonn

Abstract

We investigate whether performing reasoning in a continuous latent space leads to more robust multilingual capabilities. We compare Continuous Chain-of-Thought (using the CODI framework) against standard supervised fine-tuning across five typologically diverse languages: English, Chinese, German, French, and Urdu. Our experiments on GSM8k and CommonsenseQA demonstrate that continuous reasoning significantly outperforms explicit reasoning on low-resource languages, particularly in zero-shot settings where the target language was not seen during training. Additionally, this approach achieves extreme efficiency, compressing reasoning traces by approximately $29\times$ to $50\times$ . These findings indicate that continuous latent representations naturally exhibit greater language invariance, offering a scalable solution for cross-lingual reasoning.

Figures & tables

Test Language	GSM8K-AUG-NL		CommonsenseQA
Test Language	CODI	CoT-SFT	CODI	CoT-SFT
German	$31.79$	$36.09$	$58.48$	$38.41$
French	$32.35$	$38.21$	$57.66$	$46.03$
Chinese	$30.46$	$27.37$	$57.58$	$48.40$
English	$46.08$	$55.04$	$71.42$	$63.23$
Urdu	$9.58$	$5.38$	$32.57$	$21.87$

Table 1: Accuracy (%) on GSM8K-AUG-NL and CommonsenseQA (test splits) comparing CODI vs. CoT-SFT when trained on English.

Figure 1: Performance of LLaMA3.2-1B-Instruct trained on multi-lingual GSM8k-Aug-NL data. Both models perform similarly, with CoT-SFT performing better for high-resource languages and CODI performing better for low-resource languages. Moreover, CODI performs significantly better than CoT-SFT on Urdu when it is Out-of-distribution (OOD) (i.e., when Urdu was not part of the fine-tuning data).

Dataset	CoT-SFT (Avg Tokens)	CODI (Latent Tokens)	Compression Ratio
GSM8K-Aug-NL	$\sim$ 176	6	$\sim$ 29 $\times$
CommonsenseQA	$\sim$ 299	6	$\sim$ 50 $\times$

Table 2: Efficiency comparison between explicit and continuous reasoning. Average CoT lengths are calculated across all 5 test languages using the Multi-Lingual-with-Urdu training setup (see Tables 8 & 9 in the appendix).

Figure 3: Performance on a low-resource language (Urdu). The latent reasoning approach works significantly better than CoT-SFT in all cases.

Appendix figures & tables9 assets

Supplementary material from the paper’s appendix.

Appendix

Training Dataset	Train Setup	Test Language	Accuracy (%)
Training Dataset	Train Setup	Test Language	CODI	COT-SFT
GSM8K-Aug-NL	Multi-Lingual	German	$44.76$	$43.20$
		French	$45.19$	$45.07$
		Chinese	$43.75$	$35.60$
		English	$46.47$	$51.20$
		Urdu	$12.28$	$6.84$

Appendix

Table 3: Accuracy (%) on GSM8K-Aug-NL (test split): CODI vs. COT-SFT across training setups.

Training Dataset	Train Setup	Test Language	Accuracy (%)
Training Dataset	Train Setup	Test Language	CODI	COT-SFT
CommonSenseQA	Multi-Lingual	German	$58.72$	$49.71$
		French	$57.65$	$49.39$
		Chinese	$57.66$	$51.52$
		English	$71.42$	$62.49$
		Urdu	$35.95$	$23.01$

Appendix

Table 4: Accuracy (%) on CommonSenseQA (test split) across test languages for different training setups (CODI vs. COT-SFT).

Test Language	Base Accuracy (%)
German	$26.84$
French	$25.32$
Chinese	$23.50$
English	$36.01$
Urdu	$7.66$

Appendix

Table 5: Base model accuracy (%) on GSM8K-Aug-NL by test language (independent of fine-tuning setup).

Test Language	Base Accuracy (%)
German	$33.09$
French	$39.89$
Chinese	$41.28$
English	$44.49$
Urdu	$26.54$

Appendix

Table 6: Base model accuracy (%) on CommonSenseQA (test split) by test language (independent of fine-tuning setup).

Shared Parameters (CODI & CoT-SFT)
Hyperparameter	Value
Learning Rate	$8\times 10^{-4}$
LR Scheduler	Cosine
Warmup Ratio	0.03
Weight Decay	0.1
Max Gradient Norm	2.0

Appendix

Table 7: Training hyperparameters for CODI and CoT-SFT. All parameters are kept constant across both GSM8k-Aug-NL and CommonsenseQA datasets to ensure fair comparison.

Training Dataset	Train Setup	Test Language	Avg CoT Length (tokens)
GSM8K_AUG_NL	English	German	172.2
		French	161.6
		Chinese	171.4
		English	155.4
		Urdu	512.0
	Multi-Lingual	German	239.6

Appendix

Table 8: Average Chain-of-Thought (CoT) length (tokens) on GSM8K_AUG_NL across training setups and test languages (CoT-SFT).

Training Dataset	Train Setup	Test Language	Avg CoT Length (tokens)
CommonsenseQA	English	German	455.1
		French	329.1
		Chinese	203.8
		English	205.5
		Urdu	512.0
	Multi-Lingual	German	296.8

Appendix

Table 9: Average Chain-of-Thought (CoT) length (tokens) on CommonsenseQA across training setups and test languages (CoT-SFT).

Language	Samples	% of Total
English	95,131	24.5%
German	96,139	24.8%
French	95,868	24.7%
Chinese	96,178	24.8%
Urdu	4,619	1.2%
Total	387,935	100%

Appendix

Table 10: GSM8k-Aug-NL multilingual dataset statistics.

Language	Samples	% of Total
English	1,940	20.0%
German	1,940	20.0%
French	1,940	20.0%
Chinese	1,940	20.0%
Urdu	1,940	20.0%
Total	9,700	100%

Appendix

Table 11: CommonsenseQA multilingual dataset statistics.

Explore similar work

Less Languages, Less Tokens: An Efficient Unified Logic Cross-lingual Chain-of-Thought Reasoning Framework
Apr 22, 2026Chenyuan Zhang, Qiguang Chen, Xie Chen +6Cross-Lingual Reasoning in Language Models CoT Reasoning
Harbin Institute of Technology, Shenzhen · Shanghai Innovation Institute · Central South University +3
Cross-lingual Self-Consistency for Multilingual Reasoning with Language Models
May 31, 2026Ahmed Elhady, Eneko Agirre, Mikel ArtetxeMultilingual Language Models Cross-Lingual Reasoning in Language Models
HiTZ Center, University of the Basque Country (UPV/EHU) · Reka AI
Language as a Latent Variable for Reasoning Optimization
Apr 23, 2026Linjuan Wu, Haoran Wei, Jialong Tang +4Multilingual Language Models Cross-Lingual Reasoning in Language Models
Zhejiang University · Tongyi Lab, Alibaba Grou.

Apr 22, 2026J/K move · Enter open · S save

Less Languages, Less Tokens: An Efficient Unified Logic Cross-lingual Chain-of-Thought Reasoning Framework

Chenyuan Zhang, Qiguang Chen, Xie Chen, Zhuotao Tian+5

Harbin Institute of Technology, Shenzhen · Shanghai Innovation Institute · Central South University+3

Cross-Lingual Reasoning in Language Models CoT Reasoning Efficient Language Model Reasoning Efficient Language Model Inference Cross-Lingual Transfer

Open paper AskPDF arXiv

Cross-lingual chain-of-thought (XCoT) with self-consistency markedly enhances multilingual reasoning, yet existing methods remain costly due to extensive sampling of full trajectories across languages. Moreover, multilingual LLM representations vary strongly by language, hindering direct feature comparisons and effective pruning. Motivated by this, we introduce UL-XCoT, the first efficient unified logic cross-lingual reasoning framework that minimizes redundancy in token usage and latency, yielding the greatest efficiency under limited sampling budgets during inference. Specifically, UL-XCoT (1) achieves less languages by selecting, per query, a small candidate language set in a language-invariant unified logic space, (2) enables less tokens by monitoring logic-space trajectory dynamics during decoding to prune low-quality reasoning paths, and (3) aggregates the remaining high-quality trajectories via voting. Experiments on PolyMath across 18 languages and MMLU-ProX-Lite across 29 languages with DeepSeek-R1-DistillQwen-7B demonstrate that UL-XCoT achieves competitive accuracy while sharply cutting over 50% decoding token cost versus prior sampling baselines. UL-XCoT also delivers more stable gains on low-resource languages, underscoring consistently superior robustness where standard XCoT self-consistency method fails.

Abstract

Explore similar work

Less Languages, Less Tokens: An Efficient Unified Logic Cross-lingual Chain-of-Thought Reasoning Framework

Cross-lingual Self-Consistency for Multilingual Reasoning with Language Models

Language as a Latent Variable for Reasoning Optimization