Organizations: Leipzig University & ScaDS.AI Dresden/Leipzig · The Hong Kong Polytechnic University · CENTAL, UCLouvain & FNRS · Uppsala University · DZSKJ, University Medical Center Hamburg-Eppendorf · Georgetown University · Federal University of Juiz de Fora, FrameNet Brasil, CNPq
Languages use different morphosyntactic strategies to express the same meaning. Speakers of multiple languages are able to apply strategies from one language to another, either by mistake or intentionally, to produce a literal translation, or calque. Considering that large language models (LLMs) have been shown to generalize crosslingually, despite being predominantly trained on English, we investigate whether they overgeneralize constructional strategies from English to other languages. To test this, we create a dataset of 5583 sentence pairs across seven languages, matching literal translations of strategies from English into other languages, and literal translations of the other languages' strategies into English. We then quantify constructional bleeding by measuring how often strategies from English are assigned a higher probability than native strategies. We find that models often fail to assign higher probabilities to native strategies and tend to prefer English-derived strategies even when evaluating sentences in other languages. Our findings highlight the limitations of the current English-dominant training paradigm.
Figures & tables
Figure 1: English and French speakers describe playing the cello with distinct morphosyntactic strategies. We measure bleeding of English morphosyntax into French ( Δlogp(⋅) between French sentences) and additionally conduct a control experiment on English ( Δlogp(⋅) between English sentences).
Strategy Source Language
Target language
English
German
Das Wasser fühlt sich kalt an.
The water feels itself cold.
English
Das Wasser fühlt kalt.
The water feels cold.
Swedish
Jag bäddade sängen.
I bedded the bed.
English
Jag gjorde sängen.
I made the bed.
French
Je joue du violoncelle.
I play of the cello.
English
Je joue le violoncelle.
I play the cello.
Table 1: Examples of sentence pairs in each language.
Figure 2: Distribution of strategy preferences across languages for sentence pairs with strong native-speaker preferences ( ≫ ), measured by Δ SLOR. Following Zhong et al. (2025) , continually pretrained models (CPMs) are adapted from English-centric pretrained models through additional pretraining on target-language data, whereas balanced language models (BLMs) are pretrained from scratch with an emphasis on English and their respective focus languages. Positive values indicate a preference for the native strategy. Numbers in parentheses indicate sample sizes.
Figure 3: Failure rates (%) across languages, defined as the percentage of sentence pairs for which a model assigns a higher SLOR to the transferred strategy than to the native strategy.
Figure 4: Comparison of model preferences for native Japanese strategies over strategies transferred from English and other languages, measured by Δ SLOR. All comparisons are evaluated in Japanese, with matched semantic content and restricted to sentence pairs with strong preference on the native Japanese strategy ( ≫ ) over both the English-transferred and the corresponding non-English-transferred strategies. Positive values indicate a preference for the native Japanese strategy. Numbers in parentheses indicate sample sizes.
Figure 5: Standardized regression coefficients from multivariable linear regression models predicting Δ SLOR. Predictors include human preference annotations, model size, instruction tuning, and cross-linguistic structural features. Positive coefficients indicate a positive association with preference for the native strategy.
Figure 6: Pairwise Spearman rank correlations of Δ SLOR across language models, computed over matched sentence pairs. Higher correlations indicate greater similarity in strategy preference distributions between models.
Appendix figures & tables9 assets
Supplementary material from the paper’s appendix.
Appendix
Strategy
Body Part
Everyday Activities
Sensory Experience
Native
Transferred
≫
>
≫
>
≫
>
en
de
162
7
4
36
66
1
se
78
13
39
22
70
1
fr
188
3
182
1
39
14
pt
68
4
18
48
85
6
zh
60
5
122
7
15
11
Appendix
Table 2: Distribution of human preferences for native over transferred strategies across three construction groups. ≫ and > indicate strong and weak preferences for the native strategy, respectively. Japanese-control comparisons use Japanese as the native strategy and a non-English language as the source of the transferred strategy.
Category
Value
Count
Temporal
Past
236
Ongoing
133
Recurrent
121
Current state
77
Anterior
54
Generic
48
Appendix
Table 3: Dataset composition by temporal interpretation, subject type, and construction group. The distribution is identical across all evaluated languages.
Family
Model
Citation
Aya Expanse
CohereLabs/aya-expanse-8b
Dang et al. (2024)
CohereLabs/aya-expanse-32b
Tiny Aya
CohereLabs/tiny-aya-base
Salamanca et al. (2026)
CohereLabs/tiny-aya-global
CohereLabs/tiny-aya-earth
CohereLabs/tiny-aya-fire
Appendix
Table 4: Models evaluated, grouped by model family.
Figure 7: Distribution of strategy preferences across languages for sentence pairs with strong native-speaker preferences ( ≫ ), measured by Δ MeanLP. Positive values indicate a preference for the native strategy.
Figure 8: Comparison of model preferences for native Japanese strategies over strategies transferred from English and other languages, measured by Δ MeanLP. All comparisons are evaluated in Japanese, with matched semantic content and restricted to sentence pairs with a strong preference for the native Japanese strategy ( ≫ ) over both the English-transferred and the corresponding non-English-transferred strategies. Positive values indicate a preference for the native Japanese strategy.
Language
Candidate pairs
Groups
Threshold
German
4,879
157
31.07
French
5,674
148
38.34
Japanese
9,632
181
53.22
Portuguese
551
102
5.40
Swedish
575
116
4.96
Chinese
8,429
188
44.84
Appendix
Table 5: Candidate statistics after filtering. The threshold corresponds to the mean number of candidate pairs per group within each language.
≫ vs others
( ≫ or > ) vs others
Lang.
N
Agr.
κ
Agr.
κ
en
50
0.80
0.567
0.78
0.556
de
50
0.82
0.613
0.78
0.551
se
50
0.78
0.428
0.94
0.849
fr
50
0.82
0.526
0.98
0.929
pt
50
0.76
0.525
0.86
0.451
Appendix
Table 6: Inter-annotator agreement by target language (Lang). Strong preference distinguishes ≫ from all other labels, whereas preference direction distinguishes ≫ or > from all other labels. Agreement (Agr) denotes the proportion of matching annotations, and κ denotes Cohen’s kappa.
Lang.
Native strategy
A1
A2
Transferred strategy
Failed
de
Ich habe Zähne geputzt.
>
>
Ich habe meine Zähne geputzt.
27/28
Ich bin umgeknickt.
≫
≫
Ich habe meinen Knöchel umgeknickt.
26/28
Ich habe mir das Gesicht gewaschen.
>
>
Ich habe mein Gesicht gewaschen.
26/28
sv
Axeln gick ur led på mig.
≫
>
Jag dislokerade min axel.
28/28
Maten luktade vitlök.
>
>
Måltiden luktade som vitlök.
27/28
Jag borstade tänderna.
>
>
Jag borstade mina tänder.
25/28
Appendix
Table 7: High-failure-rate canonical sentence examples across 28 English-centric models in six languages. For each language, the three sentence pairs are shown, for which both annotators preferred the native strategy ( > or ≫ ). The A1 and A2 columns report the respective native-speaker preference annotations. The Failed column indicates the number of models assigning a lower SLOR to the native strategy than to the transferred strategy, out of 28 evaluated models.
Model
IT
EN
DE
SV
FR
PT
JA
ZH
Multilingual models
Tiny Aya
Base
6.6
35.5
44.3
22.0
59.6
52.6
31.8
Earth
✓
9.5
48.2
46.6
22.9
66.8
49.7
40.9
Fire
✓
8.1
44.2
48.9
23.5
67.3
50.9
40.3
Global
✓
10.2
48.7
46.1
21.8
66.6
49.7
41.2
Water
✓
10.0
47.5
47.3
22.2
66.6
53.2
40.3
Appendix
Table 8: Failure rates (%) across languages, defined as the percentage of sentence pairs for which a model assigns a higher SLOR to the transferred strategy than to the native strategy. Darker red cells indicate higher failure rates. ✓ in the IT column marks instruction-tuned models.