Untangling Input Language from Reasoning Language: A Diagnostic Framework for Cross-Lingual Moral Alignment in LLMs
Organizations: IDLab, Department of Electronics and Information Systems Ghent University, Belgium
Abstract
When LLMs judge moral dilemmas, do they reach different conclusions in different languages, and if so, why? Two factors could drive such differences: the language of the dilemma itself, or the language in which the model reasons. Standard evaluation conflates these by testing only matched conditions (e.g., English dilemma with English reasoning). We introduce a methodology that separately manipulates each factor, covering also mismatched conditions (e.g., English dilemma with Chinese reasoning), enabling decomposition of their contributions. To study \emph{what} changes, we propose an approach to interpret the moral judgments in terms of Moral Foundations Theory. As a side result, we identify evidence for splitting the Authority dimension into a family-related and an institutional dimension. Applying this methodology to English-Chinese moral judgment with 13 LLMs, we demonstrate its diagnostic power: (1) the framework isolates reasoning-language effects as contributing twice the variance of input-language effects; (2) it detects context-dependency in nearly half of models that standard evaluation misses; and (3) a diagnostic taxonomy translates these patterns into deployment guidance. We release our code and datasets at https://anonymous.4open.science/r/CrossCulturalMoralJudgement.
Figures & tables
| Metric | AITA | CMoral |
|---|---|---|
| Models below baseline | 12/13 (92%) | 13/13 (100%) |
| Mean model YTA rate | 39.6% | 29.7% |
| Human baseline | 53.6% | 50.0% |
| Binomial | 0.0017 | 0.0001 |
| Cohen’s | 1.64 | 2.06 |
| Model | Max Flip | AITA | CMoral | Pattern |
|---|---|---|---|---|
| Ernie | 36.5% | 1.15 (Bal) | 1.08 (Bal) | Consistent |
| Qwen | 23.4% | 0.92 (Bal) | 1.00 (Bal) | Consistent |
| Nemotron | 22.6% | 1.01 (Bal) | 0.93 (Bal) | Consistent |
| Magistral | 22.5% | 1.04 (Bal) | 0.83 (Bal) | Consistent |
| GPT-OSS † | 21.0% | 0.83 (Bal) | 0.58 (Story) | Changes |
| Claude † | 19.0% | 1.00 (Bal) | 0.59 (Story) | Changes |
Appendix figures & tables27 assets
Supplementary material from the paper’s appendix.
Appendix
| Metric | 6-dim | 7-dim |
|---|---|---|
| CV AUC (EN/EN) | 0.764 | 0.768 |
| CV AUC (CN/CN) | 0.778 | 0.782 |
| Cross-dataset variance | Higher | Lower |
| Model | AITA YTA% | Flip | MFQ | |||
|---|---|---|---|---|---|---|
| EN/EN | EN/CN | CN/EN | CN/CN | |||
| Ernie | 54.9 | 33.7 | 57.4 | 30.9 | 36.5% | 0.113 |
| Qwen | 37.1 | 34.6 | 38.8 | 24.1 | 23.4% | 0.048 |
| Nemotron | 27.5 | 27.4 | 28.6 | 21.5 | 22.6% | 0.107 |
| GPT-OSS | 33.0 | 47.2 | 39.3 | 28.8 | 21.0% | 0.089 |
| Magistral | 46.3 | 29.9 | 46.5 | 38.5 | 19.7% | 0.089 |
| Model | CMoral YTA% | Flip | MFQ | |||
|---|---|---|---|---|---|---|
| EN/EN | EN/CN | CN/EN | CN/CN | |||
| Ernie | 45.1 | 29.1 | 43.6 | 31.5 | 27.2% | 0.173 |
| Magistral | 35.6 | 29.3 | 34.8 | 31.9 | 22.5% | 0.119 |
| Claude | 34.5 | 30.3 | 35.0 | 32.1 | 19.0% | 0.042 |
| Qwen | 28.6 | 23.9 | 27.3 | 25.3 | 18.9% | 0.060 |
| Nemotron | 18.1 | 22.9 | 19.7 | 31.8 | 17.8% | 0.060 |
| Model | AITA | CMoral | ||
|---|---|---|---|---|
| Ratio | Pattern | Ratio | Pattern | |
| Claude | 1.00 | Balanced | 0.59 | Story-sens. |
| GPT-OSS | 0.83 | Balanced | 0.58 | Story-sens. |
| Grok | 0.82 | Balanced | 0.56 | Story-sens. |
| GLM | 0.84 | Balanced | 0.73 | Story-sens. |
| DeepSeek | 0.91 | Balanced | 0.87 | Balanced |
| Metric | AITA (EN CN) | CMoral (CN EN) |
|---|---|---|
| Semantic similarity | ||
| Mean embedding sim. | 0.7116 | 0.7897 |
| Median embedding sim. | 0.7213 | 0.8000 |
| % posts 0.7 | 60.0% | 88.9% |
| Moral-dimension preservation | ||
| Pearson | 0.890 | 0.825 |
| Model | Story pp | Reason. pp | Ratio | Pattern |
|---|---|---|---|---|
| Claude | 3.48 | 2.42 | 0.69 | Story-sens. |
| DeepSeek | 2.78 | 10.41 | 3.75 | Reason.-sens. |
| Ernie † | 6.71 | 27.88 | 4.16 | Reason.-sens. |
| GLM | 3.79 | 3.79 | 1.00 | Balanced |
| GPT-OSS | 2.65 | 2.77 | 1.04 | Balanced |
| Grok | 2.54 | 2.77 | 1.09 | Balanced |
| Type | Pair (EN CN paraphrase) | Cosine |
|---|---|---|
| Long sentence | “hard core watching YouTube clips … sent her a txt … only place in town” “on Little Red Book brush to explosion … one throw WeChat link to her … in-city only one shop still has goods” | 0.452 |
| Long sentence | “frenemy … make me the third person and isolate me” “plastic-sister friendship … hard-hard cut into me-and-bestie, isolate me on purpose, make me become electric light bulb” | 0.715 |
| Long sentence | “SIL … lives at home … MIL paying bills” “big aunt at sixty still lie-flat in mother-in-law home; mother-in-law pays all kinds of big-aunt bills” | 0.539 |
| Long sentence | “house in my name … his credit not good enough” “house-property certificate writes only my name … his credit too poor, cannot go onto loan … do not want receive certificate again” | 0.766 |
| Phrase | “frenemy” “plastic sister” ( suliao jiemei ) | 0.433 |
| Phrase | “YouTube clips” “Xiaohongshu videos” | 0.574 |
| Model | Story | Think | Observed | Shared |
|---|---|---|---|---|
| Nemotron | 22.9% | 23.0% | 22.6% | 80% |
| GLM | 15.8% | 13.3% | 16.2% | 67% |
| Claude | 13.3% | 13.3% | 15.6% | 59% |
| GPT-OSS | 19.8% | 16.5% | 21.0% | 57% |
| Magistral | 16.6% | 17.2% | 19.7% | 57% |
| Qwen | 21.2% | 19.4% | 23.4% | 56% |
| Model | AITA | CMoral | ||||
|---|---|---|---|---|---|---|
| Story | Think | Ratio | Story | Think | Ratio | |
| Claude | 13.3% | 13.3% | 1.00 | 17.1% | 10.1% | 0.59 |
| GPT-OSS | 19.8% | 16.5% | 0.83 | 13.2% | 7.7% | 0.58 |
| Grok | 14.8% | 12.0% | 0.82 | 14.9% | 8.3% | 0.56 |
| GLM | 15.8% | 13.3% | 0.84 | 11.8% | 8.7% | 0.73 |
| DeepSeek | 12.2% | 13.8% | 1.13 | 13.5% | 11.5% | 0.86 |
| Dimension | Story | Think | Ratio | Dominant |
|---|---|---|---|---|
| Intercept | 0.177 | 0.360 | 2.03 | Thinking |
| Care/Harm | 0.077 | 0.118 | 1.53 | Thinking |
| Loyalty | 0.104 | 0.084 | 0.81 | Balanced |
| Proportionality | 0.069 | 0.051 | 0.74 | Story |
| Equality | 0.118 | 0.081 | 0.69 | Story |
| Authority (Family) | 0.081 | 0.053 | 0.65 | Story |
| Dimension | Story | Think | Ratio | Dominant |
|---|---|---|---|---|
| Intercept | 0.232 | 0.392 | 1.69 | Thinking |
| Authority (Family) | 0.062 | 0.078 | 1.27 | Thinking |
| Proportionality | 0.074 | 0.079 | 1.06 | Balanced |
| Care/Harm | 0.065 | 0.066 | 1.01 | Balanced |
| Equality | 0.075 | 0.062 | 0.83 | Balanced |
| Authority (Society) | 0.125 | 0.083 | 0.66 | Story |
| Model | Flip Rate | 95% CI | Width |
|---|---|---|---|
| Claude | 17.7% | [16.0%, 19.5%] | 3.5pp |
| DeepSeek | 16.8% | [15.2%, 18.6%] | 3.4pp |
| Ernie | 31.8% | [29.8%, 33.9%] | 4.2pp |
| GLM | 15.9% | [14.2%, 17.6%] | 3.4pp |
| GPT-OSS | 17.6% | [15.8%, 19.3%] | 3.5pp |
| Magistral | 22.7% | [20.8%, 24.7%] | 3.9pp |
| Subset | Story Effect | Thinking Effect | Ratio |
|---|---|---|---|
| High compliance ( 90%) | 3.54pp | 5.17pp | 1.46 |
| Low compliance ( 90%) | 3.13pp | 10.71pp | 3.42 |
| All 9 models | 3.40pp | 7.02pp | 2.06 |
| Effect | AITA | CMoral | ||
|---|---|---|---|---|
| Coef | Coef | |||
| story_cn | 0.85 | 0.36 | ||
| think_cn | 0.001 | 0.001 | ||
| Interaction | 0.49 | 0.55 | ||
| Metric (AITA, 8 models) | Value |
|---|---|
| Matched pairs | 13,017 |
| Overall flip rate | 19.34% (2,518/13,017) |
| Top-quartile reasoning-token similarity flip rate | 14.47% (471/3,255) |
| Top-quartile reasoning-token + explanation similarity flip rate | 2.93% (32/1,092) |
| Flip direction among all flips | 67.9% Y N (1,710/2,518) |
| Predictor | Coefficient | -value |
|---|---|---|
| sim_reasoning | ||
| sim_explanation | ||
| length_diff |
| Dataset (EN+CN) | Within-1 | Exact | Direction | w/ median | |
|---|---|---|---|---|---|
| AITA | 95.37% | 73.53% | 82.08% | 0.797 | 0.632 |
| CMoral | 96.54% | 78.46% | 84.73% | 0.784 | 0.636 |
| Dimension | Interpretation | |
|---|---|---|
| Care/Harm | 0.72 | Acceptable |
| Authority | 0.59 | Fair |
| Proportionality | 0.53 | Fair |
| Loyalty | 0.50 | Fair |
| Equality | 0.50 | Fair |
| Purity | 0.44 | Fair |
| Human–Human (A vs. B) | Hmean vs. LLM-median | Hmean vs. LLM-median | |||||
|---|---|---|---|---|---|---|---|
| Dimension | weighted | MAE | weighted | MAE | within-1 | ||
| Care/Harm | 0.98 | 0.90 | 0.15 | 0.65 | 0.41 | 0.84 | 0.78 |
| Authority | 0.73 | 0.66 | 0.20 | 0.20 | 0.24 | 0.61 | 0.85 |
| Proportionality | 0.69 | 0.64 | 0.05 | 0.13 | 0.00 | 0.61 | 0.83 |
| Purity | 0.57 | 0.58 | 0.30 | 0.39 | 0.24 | 0.43 | 0.88 |
| Loyalty | 0.64 | 0.51 | 0.25 | 0.28 | 0.10 | 0.54 | 0.90 |
| Model | Flip B vs A | Flip C vs A | Reduction | Interpretation |
|---|---|---|---|---|
| Ernie | 50.0% | 36.7% | 27% | Mixed |
| Qwen | 28.6% | 25.0% | 12% | Genuine effect |
| Claude | 11.5% | 7.7% | 33% | Mixed |
| Model | Care/Harm | Purity | Auth_Fam |
|---|---|---|---|
| Ernie | 0.04 | 0.09 | +0.06 |
| DeepSeek | 0.11 | 0.21 | +0.05 |
| Qwen | +0.09 | 0.15 | +0.07 |
| Nemotron | +0.13 | 0.05 | +0.02 |
| Grok | 0.04 | 0.26 | 0.02 |
| Magistral | 0.09 | 0.18 | +0.16 |
| Model | EN/EN | EN/CN | CN/EN | CN/CN |
|---|---|---|---|---|
| Claude | 0.316 | 0.198 | 0.296 | 0.372 |
| DeepSeek | 0.281 | 1.038 | 1.060 | 0.664 |
| Ernie | +0.218 | 0.729 | +0.256 | 0.939 |
| GLM | 0.908 | 0.378 | 0.490 | 0.438 |
| GPT-OSS | 0.791 | 0.994 | 1.029 | 0.974 |
| Grok | 0.751 | 0.781 | 1.182 | 0.909 |
| Model | EN/EN | EN/CN | CN/EN | CN/CN |
|---|---|---|---|---|
| Claude | +0.188 | +0.178 | +0.192 | +0.162 |
| DeepSeek | +0.110 | +0.080 | +0.168 | +0.155 |
| Ernie | 0.015 | +0.023 | +0.001 | +0.044 |
| GLM | +0.260 | +0.065 | +0.215 | +0.103 |
| GPT-OSS | 0.151** | 0.122* | 0.115** | 0.031 |
| Grok | +0.114 | +0.100 | +0.183 | +0.095 |
| Model | EN/EN | EN/CN | CN/EN | CN/CN |
|---|---|---|---|---|
| Claude | 1.022 | 1.166 | 1.031 | 1.043 |
| DeepSeek | 1.141 | 1.281 | 1.073 | 1.251 |
| Ernie | 0.759 | 0.753 | 0.826 | 0.801 |
| GLM | 1.132 | 1.206 | 1.075 | 1.114 |
| GPT-OSS | 0.924 | 0.983 | 0.910 | 0.859 |
| Grok | 1.043 | 1.187 | 1.104 | 1.084 |
| Model | EN/EN | EN/CN | CN/EN | CN/CN |
|---|---|---|---|---|
| Claude | 0.098 | 0.170 | 0.181 | 0.241 |
| DeepSeek | 0.042 | 0.021 | 0.183 | 0.248 |
| Ernie | 0.107 | 0.116 | 0.190 | 0.192 |
| GLM | 0.030 | 0.100 | 0.252 | 0.207 |
| GPT-OSS | 0.114 | 0.102 | 0.204 | 0.179 |
| Grok | 0.066 | 0.081 | 0.195 | 0.321 |
| Model | Total | Active | Arch. |
|---|---|---|---|
| Qwen3-235B | 235B | 22B | MoE |
| DeepSeek-R1-0528 | 671B | 37B | MoE |
| Kimi-K2 | 1T | 32B | MoE |
| Llama-4-Maverick | 400B | 17B | MoE |
| Ernie-4.5-21B | 21B | 21B | Dense |
| GPT-OSS-20B | 20B | 20B | Dense |