cs.CLMar 4, 2026

Post-Training Language Models for Crosslingual Consistency

Authors: Tianyu Liu, Jirui Qi, Mrinmaya Sachan, Ryan Cotterell, Raquel Fernández, Arianna Bisazza

Organizations: ETH Zürich · CLCG, University of Groningen · University of Amsterdam

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Language models often respond inconsistently to translation-equivalent prompts across languages, undermining the reliability of multilingual systems. To quantify this, we give an information-theoretic definition of crosslingual consistency as a divergence bound between a model's response distribution and its round-trip pushforward across languages. We then introduce penalized consistency optimization (PCO), a post-training procedure that couples this divergence with a Kullback-Leibler penalty to a fixed reference language model. Because direct optimization of PCO requires expensive on-policy roll-outs, we propose a tractable surrogate, direct consistency optimization (DCO), which can be optimized off-policy. Across diverse language models and 26 languages, DCO significantly improves crosslingual consistency, outperforms existing methods, and enables targeted alignment of low-resource languages.

Figures & tables

Figure 1: Illustration of DCO. DCO promotes crosslingual consistency by aligning the response likelihoods assigned to parallel prompts. Before alignment, these likelihoods induce inconsistent preference rankings across languages. After alignment, the ranking of candidate responses agrees in both languages.
Figure 2: Let {\color[rgb]{0.7773,0.1563,0.1563}\pi}_{1} be a language model over {\color[rgb]{0.7773,0.1563,0.1563}L}_{1} , and {\color[rgb]{0.1172,0.3594,0.707}\pi}_{2} be a language model over {\color[rgb]{0.1172,0.3594,0.707}L}_{2} and {\color[rgb]{0.1172,0.3594,0.707}\tau}_{1}^{2}\colon{\color[rgb]{0.7773,0.1563,0.1563}L}_{1}\rightsquigarrow{\color[rgb]{0.1172,0.3594,0.707}L}_{2} and {\color[rgb]{0.7773,0.1563,0.1563}\tau}_{2}^{1}\colon{\color[rgb]{0.1172,0.3594,0.707}L}_{2}\rightsquigarrow{\color[rgb]{0.7773,0.1563,0.1563}L}_{1} be a pair of translators. The above commutativity diagram shows the case when {\color[rgb]{0.1172,0.3594,0.707}\tau}_{1}^{2}\mathbin{\sharp}{\color[rgb]{0.7773,0.1563,0.1563}\pi}_{1}\mathbin{\sharp}{\color[rgb]{0.7773,0.1563,0.1563}\tau}_{2}^{1}={\color[rgb]{0.1172,0.3594,0.707}\pi}_{2} and {\color[rgb]{0.7773,0.1563,0.1563}\tau}_{2}^{1}\mathbin{\sharp}{\color[rgb]{0.1172,0.3594,0.707}\pi}_{2}\mathbin{\sharp}{\color[rgb]{0.1172,0.3594,0.707}\tau}_{1}^{2}={\color[rgb]{0.7773,0.1563,0.1563}\pi}_{1} , i.e., when we have (f,0)(f,0) -consistency for all prompts {\color[rgb]{0.7773,0.1563,0.1563}\bm{x}}_{1}\in{\color[rgb]{0.7773,0.1563,0.1563}L}_{1} and {\color[rgb]{0.1172,0.3594,0.707}\bm{x}}_{2}\in{\color[rgb]{0.1172,0.3594,0.707}L}_{2} . For ε>0\varepsilon>0 , Def. 1 is a relaxation of commutativity.
MethodQwen2.5-14BGemma3-12B-ptQwen3-14BAya-Expanse-8BLlama3.1-8B
clc alla ena ¬\neg enclc alla ena ¬\neg enclc alla ena ¬\neg enclc alla ena ¬\neg enclc alla ena ¬\neg en
Base68.668.672.572.558.158.173.673.670.170.162.362.376.176.176.676.667.367.372.272.259.859.852.952.960.960.957.357.345.845.8
+ SFT ∗+0.6+0.6+1.5+1.5+6.7+6.7+0.6+0.6+0.7+0.7+1.6+1.6−0.2-0.2+0.1+0.1+0.5+0.5+3.5+3.5+0.7+0.7+0.5+0.5+4.3+4.3+6.7+6.7+5.9+5.9
+ DPO ∗+12.3+12.3+7.8+7.8+13.9+13.9+6.5+6.5+1.8+1.8+3.4+3.4+3.0+3.0+2.7+2.7+4.2+4.2+1.3+1.3+2.5+2.5+2.5+2.5+10.1+10.1+8.0+8.0+8.8+8.8
+ DCO ∗+13.1+13.1+7.6+7.6+13.5+13.5+10.2+10.2+1.2+1.2+2.9+2.9+4.4+4.4+2.8+2.8+4.3+4.3+3.1+3.1+2.7+2.7+2.6+2.6+13.8+13.8+7.3+7.3+8.9+8.9
+ CALM+4.2+4.2+0.0+0.0+4.1+4.1+3.0+3.0−0.4-0.4−0.0-0.0+0.3+0.3−2.1-2.1−1.1-1.1+1.4+1.4−2.2-2.2−2.1-2.1+3.0+3.0−2.2-2.2−5.0-5.0
Table 1: Comparison with prior methods on MMMLU under the joint-training setup. Methods marked with ∗ are trained on human-annotated prompt–response pairs. See Table 8 for the full results.
ModelMMMLUXCSQABMLAMA
clc alla ena ¬\neg enclc alla ena ¬\neg enclc alla ena ¬\neg en
Qwen2.5-14B68.668.672.572.558.158.164.664.687.087.056.956.941.941.962.762.738.638.6
+ DCO+12.6+12.6+1.6+1.6+8.5+8.5+6.8+6.8−2.5-2.5+4.7+4.7+15.4+15.4+6.3+6.3+14.2+14.2
Gemma3-12B-pt73.673.670.170.162.362.358.358.366.066.047.247.242.242.268.368.338.338.3
+ DCO+7.2+7.2−0.9-0.9+1.4+1.4+4.6+4.6+0.1+0.1+3.6+3.6+16.7+16.7+1.6+1.6+17.0+17.0
Qwen3-14B76.176.176.676.669.069.061.961.977.677.654.054.038.938.958.458.436.436.4
Table 2: Crosslingual consistency with English, together with average accuracy on English and non-English instances, on MMMLU, XCSQA, and BMLAMA. See Appendix K for the full results across nine LMs.
MethodAnatomyMedical GeneticsHigh School MathsCollege Maths
clc alla ena ¬\neg enclc alla ena ¬\neg enclc alla ena ¬\neg enclc alla ena ¬\neg en
Base59.4968.8946.3070.3882.0063.7967.8353.7043.2364.2557.0037.50
+ DCO+10.94+1.80+3.76+10.33+5.43+7.00+11.27+2.38+6.80+11.36-0.36+12.36
Table 3: Cross-domain performance on Qwen2.5-14B . The model is trained with DCO on high school microeconomics (390 prompts) and tested on distinct domains on MMMLU. Detailed results by language, and for more test domains, are shown in Appendix K .
Figure 3: Left: Response accuracy after performing DCO on English–Swahili. Right: Proportion of prompts for which the LM’s response changes after DCO, with CLC values marked in green.
ModelAcc enAcc zhConsistency
Qwen2.5-7B-Instruct89.283.684.7
+ on-policy RL90.086.886.9
gemma-3-12b-it90.387.187.7
+ on-policy RL92.388.189.2
Table 4: On-policy results on GSM8K.
ModelAcc enAcc zhConsistency
Qwen2.5-7B-Instruct66.258.868.9
+ on-policy RL70.159.772.3
gemma-3-12b-it71.964.870.9
+ on-policy RL72.466.771.8
Table 5: On-policy results on MMMLU.
Appendix figures & tables18 assets

Supplementary material from the paper’s appendix.

Appendix

DatasetKnowledge Type#LangsParallel#Train#TestResponse Format
PromptCandidate
MMMLUGeneral Knowledge12(+2)✓✓50009042A/B/C/D
XCSQACommonsense Reasoning16✓✓800200A/B/C/D/E
BMLAMAFactual Association17✓✓50001792Words
Appendix
Table 6: Statistics of MMMLU, XCSQA, and BMLAMA datasets.
DatasetPromptCandidatesReference Completion
MMMLUWhich cells in the blood do not have a nucleus? A. Lymphocyte B. Monocyte C. Erythrocyte D. Basophil[A, B, C, D]C
XCSQAWhat might lock after someone drives in? A. gate B. doorknob C. mouths D. entrance E. front door[A, B, C, D, E]A
BMLAMABerlin is the capital of[Greenland, Piedmont, Oman, Venezuela, Fiji, Latvia, Taiwan, Norway, Romania, Germany]Germany
Appendix
Table 7: Examples of instances in MMMLU, XCSQA and BMLAMA.
Figure 4: Left: Response accuracy after performing DCO on English–Yoruba. Right: Proportion of prompts for which the LM’s response changes after DCO, with CLC values marked in green.
MethodQwen2.5-14BGemma3-12B-ptQwen3-14BAya-Expanse-8BLlama3.1-8B
clc alla ena ¬\neg enclc alla ena ¬\neg enclc alla ena ¬\neg enclc alla ena ¬\neg enclc alla ena ¬\neg en
Base68.6±0.0268.6\pm 0.0272.5±0.472.5\pm 0.458.1±0.258.1\pm 0.273.6±0.0273.6\pm 0.0270.1±0.470.1\pm 0.462.3±0.262.3\pm 0.276.1±0.0276.1\pm 0.0276.6±0.476.6\pm 0.467.3±0.167.3\pm 0.172.2±0.0272.2\pm 0.0259.8±0.559.8\pm 0.552.9±0.252.9\pm 0.260.9±0.0260.9\pm 0.0257.3±0.557.3\pm 0.545.8±0.245.8\pm 0.2
+ SFT ∗+0.6±0.02+0.6\pm 0.02+1.5±0.4+1.5\pm 0.4+6.7±0.2+6.7\pm 0.2+0.6±0.02+0.6\pm 0.02+0.7±0.4+0.7\pm 0.4+1.6±0.2+1.6\pm 0.2−0.2±0.02-0.2\pm 0.02+0.1±0.5+0.1\pm 0.5+0.5±0.1+0.5\pm 0.1+3.5±0.02+3.5\pm 0.02+0.7±0.5+0.7\pm 0.5+0.5±0.2+0.5\pm 0.2+4.3±0.02+4.3\pm 0.02+6.7±0.5+6.7\pm 0.5+5.9±0.2+5.9\pm 0.2
+ DPO ∗+12.3±0.02+12.3\pm 0.02+7.8±0.4+7.8\pm 0.4+13.9±0.1+13.9\pm 0.1+6.5±0.02+6.5\pm 0.02+1.8±0.4+1.8\pm 0.4+3.4±0.1+3.4\pm 0.1+3.0±0.02+3.0\pm 0.02+2.7±0.5+2.7\pm 0.5+4.2±0.1+4.2\pm 0.1+1.3±0.02+1.3\pm 0.02+2.5±0.5+2.5\pm 0.5+2.5±0.2+2.5\pm 0.2+10.1±0.02+10.1\pm 0.02+8.0±0.4+8.0\pm 0.4+8.8±0.2+8.8\pm 0.2
+ DCO ∗+13.1±0.02+13.1\pm 0.02+7.6±0.4+7.6\pm 0.4+13.5±0.1+13.5\pm 0.1+10.2±0.02+10.2\pm 0.02+1.2±0.4+1.2\pm 0.4+2.9±0.1+2.9\pm 0.1+4.4±0.02+4.4\pm 0.02+2.8±0.5+2.8\pm 0.5+4.3±0.1+4.3\pm 0.1+3.1±0.02+3.1\pm 0.02+2.7±0.5+2.7\pm 0.5+2.6±0.2+2.6\pm 0.2+13.8±0.02+13.8\pm 0.02+7.3±0.4+7.3\pm 0.4+8.9±0.2+8.9\pm 0.2
+ CALM+4.2±0.02+4.2\pm 0.02+0.0±0.4+0.0\pm 0.4+4.1±0.2+4.1\pm 0.2+3.0±0.02+3.0\pm 0.02−0.4±0.4-0.4\pm 0.4−0.0±0.2-0.0\pm 0.2+0.3±0.02+0.3\pm 0.02−2.1±0.5-2.1\pm 0.5−1.1±0.1-1.1\pm 0.1+1.4±0.02+1.4\pm 0.02−2.2±0.5-2.2\pm 0.5−2.1±0.2-2.1\pm 0.2+3.0±0.02+3.0\pm 0.02−2.2±0.5-2.2\pm 0.5−5.0±0.2-5.0\pm 0.2
Appendix
Table 8: Comparison with previous methods in the joint training setup, on the MMMLU dataset. clc all : average crosslingual consistency (measured by RankC) between all language pairs; a en / a ¬\neg en : average accuracy on English/non-English instances. We report mean scores with their standard errors. Methods with ∗ are trained on reference completions.
Figure 5: The changes in CLC of Qwen2.5-14B after DCO. Left: CLC between all language pairs on the original model. Right: The improvements in CLC of the post-DCO model.
Figure 6: The changes in CLC of Gemma3-12B after DCO. Left: CLC between all language pairs on the original model. Right: The Improvements in CLC of the post-DCO model.
Figure 7: The changes in CLC of Qwen3-14B after DCO. Left: CLC between all language pairs on the original model. Right: The Improvements in CLC of the post-DCO model.
Figure 8: The changes in CLC of Aya-Expanse-8B after DCO. Left: CLC between all language pairs on the original model. Right: The Improvements in CLC of the post-DCO model.
Figure 9: The changes in CLC of Llama3.1-8B after DCO. Left: CLC between all language pairs on the original model. Right: The Improvements in CLC of the post-DCO model.
Modelen - esar - zhko - fr
Acc enAcc esCLCAcc arAcc zhCLCAcc koAcc frCLC
Qwen2.5-14B62.6744.8147.4542.3537.0536.3440.5736.7137.29
+DCO68.9757.3761.8050.3348.4946.7452.4653.6852.24
Appendix
Table 9: Non-English alignment results on BMLAMA.
 ardeesfrhiiditjakoptzh
Consistency
Qwen2.5-7B64.1870.6874.6073.3350.9569.2073.4668.0565.6372.4072.38
+ DCO+9.17+9.20+6.68+8.88+14.66+7.42+7.50+8.16+8.01+9.99+7.66
Qwen2.5-14B66.1970.9369.4570.1959.1570.4369.6870.1667.0268.5472.38
+ DCO+12.09+12.82+14.68+14.11+12.98+11.04+15.66+9.15+10.80+15.76+9.54
Gemma3-4B-pt63.2168.2471.7871.3256.8464.4870.5358.2860.7369.0060.75
Appendix
Table 10: Consistency improvements with English for each language across all models on MMMLU.
 enardeesfrhiiditjakoptzh
Accuracy
Qwen2.5-7B69.5754.5159.9863.4362.3044.3358.7162.6659.0757.1361.9363.98
+ DCO-0.01+1.24+2.02+0.72+1.46+4.32+1.80+1.76+1.74+1.44+1.97+1.50
Qwen2.5-14B72.4654.8859.5256.8358.3145.6661.0656.2163.2456.9656.3569.87
+ DCO+1.64+7.63+8.51+12.94+10.67+8.41+6.31+12.57+4.16+8.00+13.38+0.83
Gemma3-4B-pt56.0943.1449.0252.5150.5139.6448.4749.6442.7845.3950.2546.56
Appendix
Table 11: Accuracy of each model on MMMLU across all languages after DCO.
 zhdeesfritjanlplptruarvihiswur
Consistency
Qwen2.5-7B64.4172.2677.8271.5771.9657.7865.4965.4175.2766.6765.4968.8049.7635.5848.02
+ DCO+10.81+4.33+4.50+6.55+6.04+8.78+9.73+6.33+5.97+6.34+8.33+6.32+8.25-0.28+7.83
Qwen2.5-14B66.9470.9073.8271.6073.5864.5568.8265.0374.1364.3164.8664.4153.6839.5052.57
+ DCO+8.03+5.80+8.21+5.93+3.81+5.92+8.52+8.34+6.52+8.73+11.00+10.68+5.33+0.40+4.90
Gemma3-4B-pt54.2360.9157.8556.8360.2850.2156.9353.2957.6857.1853.5052.7851.5637.3647.26
Appendix
Table 12: Consistency improvements with English for each language across all models on XCSQA.
 enzhdeesfritjanlplptruarvihiswur
Accuracy
Qwen2.5-7B84.0055.5058.0066.0060.5062.0052.5054.5057.5063.0055.5053.5061.5039.5025.0041.00
+ DCO-1.93+7.50+5.00+1.50+2.50+5.00+4.50+5.00+6.50+4.50+3.00+5.00+1.50+7.000.00+4.00
Qwen2.5-14B87.0059.5062.0065.0062.5065.0057.5061.0060.0067.5056.0054.5056.5046.5032.5047.00
+ DCO-2.53+7.00+5.00+5.00+6.00+2.00+5.00+5.00+3.00+4.00+5.00+8.00+10.00+1.500.00+3.50
Gemma3-4B-pt56.5035.5041.0045.0045.0042.5032.0040.0039.0034.5039.0036.5043.0030.5027.0031.00
Appendix
Table 13: Accuracy of each model on XCSQA across all languages after DCO.
 frnlesrujazhkovielhuhetrcaarukfa
Consistency
Qwen2.5-7B44.1248.4145.5041.7839.0540.1836.5044.5430.8830.6630.8834.2939.1640.8940.5335.85
+ DCO+17.90+13.98+16.21+15.08+16.89+11.53+18.40+19.05+17.79+15.54+17.27+17.10+14.64+13.24+15.02+14.15
Qwen2.5-14B45.0150.8747.4542.8341.9442.6141.1946.9135.7032.0438.8437.9941.7141.7842.4840.58
+ DCO+17.33+14.48+14.35+16.05+16.66+9.77+16.23+16.32+18.86+16.48+14.32+15.93+13.36+15.40+16.67+14.38
Gemma3-4B-pt38.2648.7845.6340.5736.9931.7835.7245.2338.1634.8535.9139.7736.8835.5143.7535.92
Appendix
Table 14: Consistency improvements with English for each language across all models on BMLAMA.
 enfrnlesrujazhkovielhuhetrcaarukfa
Accuracy
Qwen2.5-7B61.8336.6143.0240.7939.0636.6134.8233.3738.8427.6828.0126.4031.8634.1539.7938.5632.42
+ DCO+5.61+19.31+14.57+15.68+12.61+16.57+14.23+17.75+15.35+12.61+12.56+13.89+12.73+13.23+9.32+13.34+10.88
Qwen2.5-14B62.6736.7146.4844.8142.0240.0737.0540.5741.4632.5328.1235.6034.9937.0542.3541.4636.66
+ DCO+6.33+21.10+12.90+12.56+13.67+15.18+11.89+14.28+15.85+13.12+15.85+13.34+12.89+14.62+11.17+15.12+13.51
Gemma3-4B-pt66.0732.4243.8639.6835.2130.3026.3432.4836.8331.8130.6929.9134.6032.4231.5838.9030.08
Appendix
Table 15: Accuracy of each model on BMLAMA across all languages after DCO.
Domainardeesfrhiiditjakoptswyozhbn
Anatomy
Base54.0865.2057.8667.7950.7665.1768.5064.7756.5663.3149.4546.7870.9251.68
+ DCO+13.10+7.02+17.56+12.04+9.30+13.69+11.42+11.12+10.49+19.53+4.81+3.20+12.00+7.91
Medical genetics
Base66.9178.7675.4271.4656.9274.5274.5477.2566.7677.8862.2559.9579.9762.66
+ DCO+9.21+11.71+16.51+18.54+12.85+8.24+13.17+3.18+10.13+14.60+4.06+4.11+8.99+9.38
Appendix
Table 16: Consistency with English under cross-domain settings using Qwen2.5-14B . The model is post-trained with data of ‘high school microeconomics’.
Domainenardeesfrhiiditjakoptswyozhbn
Anatomy
Base68.8943.7046.6745.9353.3334.0750.3751.8557.0444.4443.7032.5931.8572.5940.00
+ DCO+1.800.00+5.92+5.18+6.67+2.97+8.89+7.410.00+5.93+18.52-2.96-4.44-2.96+1.48
Medical genetics
Base82.0061.0073.0066.0066.0046.0074.0063.0079.0059.0069.0053.0053.0078.0053.00
+ DCO+5.43+5.00+7.00+14.00+14.00+11.00+3.00+13.00-5.00+8.00+16.00+2.00-2.00+4.00+8.00
Appendix
Table 17: Accuracy under cross-domain settings using Qwen2.5-14B . The model is post-trained with data of ‘high school microeconomics’.
More in this topic

Cross-Lingual Consistency in Language Models

Cross-lingual consistency in language models studies whether equivalent inputs across languages produce comparable knowledge, reasoning, and safety behavior.

See all 71 papers

Explore similar work

CardsList

  1. A Systematic Evaluation of Cross-Lingual Consistency Enhancement Methods in Multilingual Language Models

    Sep 3, 2026Jirui Qi, Mingyang Wang, Hinrich Schütze +2Multilingual Language Model EvaluationMultilingual QA

    University of Groningen · LMU Munich · University of Amsterdam

  2. English is Not All You Need: Systematically Exploring the Role of Multilinguality in LLM Post-Training

    Apr 14, 2026Mehak Dhaliwal, Shashwat Chaurasia, Yao Qin +2Cross-Lingual Reasoning in Language ModelsLow-Resource Language Processing

    UC Santa Barbara · Amazon

  3. Mitigating Cross-Lingual Cultural Inconsistencies in LLMs via Consensus-Driven Preference Optimisation

    Apr 2, 2026Lucas Resck, Isabelle Augenstein, Anna KorhonenCross-Lingual Consistency in Language ModelsCross-Cultural Language Model Evaluation

    Language Technology Lab, University of Cambridge · University of Copenhagen