Lexical normalization rewrites the noisy, non-standard words that fill user-generated text (tmrw, u, gr8) into their standard forms. Because labelled data is scarce for most languages, a popular shortcut is to train a single model on many languages at once. We ask a simple question: how many languages should such a model be trained on? Using one fixed-capacity character-level model and twelve languages from a standard benchmark, we vary the number of jointly trained languages from one to twelve and measure per-language accuracy. We find a clear curse of multilinguality: accuracy is highest when a language is trained with only a few others, often just one to four, and then falls steadily and substantially, dropping by about forty percent as the rest are piled on. A control that holds the total amount of training data constant makes the decline arrive sooner and fall further, which points to competition among the languages for one fixed-size model rather than to how much data is available. We also test whether a language's typological distance from the others predicts its ideal number of co-training languages, and find no dependable rule: any apparent relationship rests on a couple of languages and does not hold up. For compact normalization models, less can be more: a few languages beat pooling everything into a single model.
Figures & tables
Lang
Train
Test
% changed
Danish
16,448
3,758
9.2
German
15,006
5,082
17.2
English
35,216
29,421
7.6
Spanish
7,189
6,635
7.7
Croatian
54,416
15,695
6.6
Indo.–Eng.
13,949
4,366
14.7
Table 1: The twelve MultiLexNorm datasets: training and test tokens, and the share of training tokens whose gold form differs from the input (“% changed”).
Setting
Value
Architecture
char Transformer (enc–dec)
Layers (enc / dec)
3 / 3
Model width
128
Attention heads
4
Feed-forward width
512
Parameters
1.49M
Table 2: Model and training settings, identical across all conditions.
Lang
k∗
ERR k=1
ERR k∗
ERR k=12
Danish
2
.450
.498
.285
German
1
.108
.108
.010
English
1
.384
.384
.088
Spanish
10
− .847
− .387
− .451
Croatian
4
.219
.246
.196
Indo.–Eng.
3
.300
.353
.245
Table 3: Per-language results (growing-data setting). k∗ is the optimal number of jointly trained languages; ERR is shown at one language, at the optimum, and at all twelve. Spanish and Italian have negative ERR throughout and, unlike every other language, improve as partners are added; they are excluded from the aggregate frontier and discussed in Section 4.1 . Among the other ten, six peak at an interior k∗ .
Figure 1: Mean ERR against the number of jointly trained languages k , averaged over the ten languages the model normalizes effectively. Both settings peak at k=2 and then decline steadily: the curse of multilinguality. Holding total data fixed (dashed) makes the drop steeper, so the decline is not a matter of data volume. Shaded bands are 95% confidence intervals across languages.
Figure 2: Per-language frontiers (growing-data setting). Dotted lines mark each language’s optimum k∗ . Languages well served alone (e.g. English, Dutch) gain nothing from partners; the rest benefit from a few before crowding hurts.
Figure 3: Typological isolation versus optimum k∗ for the ten effective languages. The positive correlation ( r=0.69 , p=0.04 ) rests entirely on the two most isolated languages (Turkish, Indonesian–English); the European languages cluster with no internal trend, and the correlation vanishes over all twelve ( r=0.25 ). We draw no reliable typological rule.