RILEC: Detection and Generation of L1 Russian Interference Errors in English Learner Texts

Darya Kharlamova; Irina Proskurina

cs.CLMar 7, 2026

RILEC: Detection and Generation of L1 Russian Interference Errors in English Learner Texts

Authors: Darya Kharlamova, Irina Proskurina

Organizations: Higher School of Economics · Université Claude Bernard Lyon 1, Université Lumière Lyon 2, ERIC

Abstract

Many errors in student essays can be explained by influence from the native language (L1). L1 interference refers to errors influenced by a speaker's first language, such as using stadion instead of stadium, reflecting lexical transliteration from Russian. In this work, we address the task of detecting such errors in English essays written by Russian-speaking learners. We introduce RILEC, a large-scale dataset of over 18,000 sentences, combining expert-annotated data from REALEC with synthetic examples generated through rule-based and neural augmentation. We propose a framework for generating L1-motivated errors using generative language models optimized with PPO, prompt-based control, and rule-based patterns. Models fine-tuned on RILEC achieve strong performance, particularly on word-level interference types such as transliteration and tense semantics. We find that the proposed augmentation pipeline leads to a significant performance improvement, making it a potentially valuable tool for learners and teachers to more effectively identify and address such errors.

Figures & tables

We will pay more attention to our health if we will have enough time. The straightforward approach is for people to pay attention on their nutrition and daily activity, and to refuse from using alcohol and tobacco. On one hand, most of parents want to shield the younger generation from another cureless diseases, including ads between TV shows or youtube’s videos. On the other hand, we have firm and organizations producing unhealthy goods, and these firms want to expand their base of consumers through advertising.

Table 1: An excerpt from a real essay parsed by the model fine-tuned on RILEC : Synonyms , Copying Expression , Tense Semantics , Word Form Transmission , and Transliteration .

Dataset	L1 Background	Sents.	Error Tags	CEFR Level	Task
FCE ( Yannakoudakis et al., 2018 )	NA, Mixed L1s	2,695	71	B1-B2	GEC/GED
KJ ( Heck and Meurers, 2023 )	JA	3,199	22	A1-A2?	GEC/GED
CoNLL-2014 ( Ng et al., 2014 )	NA, Mixed L1s	1,312	28	C1	GEC/GED
JFLEG ( Napoles et al., 2017 )	NA, Mixed L1s	747	-	A1-C2?	GEC
BEA-2019 ( Bryant et al., 2019 )	NA, Mixed L1s (NUS students; EN, ZH, MS, TA)	4,384	25	A1-Native	GEC/GED
ICNALE ( Ishikawa, 2023 )	10 Asian L1s (ZH, ID, JA, KO, TH, ZH-TW, HK-YUE, PK-UR, PH-TL, TA)	15,000 essays	-	A2-C1	GEC

Table 2: Comparison of English learner and GEC corpora by L1 background, dataset size, tag inventory, CEFR range, and task. Only RILEC includes explicit L1-specific interference tags. GEC = Grammatical Error Correction. GED = Grammatical Error Detection. NLI = Native Language Identification. L1 GED = L1-interference Error Detection. A question mark indicates unknown or approximate information.

Figure 1: Pairwise inter-annotator agreement (Cohen’s kappa) for the REALEC-L1 data annotation.

Dataset	Train	Test
REALEC-L1	3,882	2,204
GPT-based (PPO)	4,583	1,965
LLM-based	556	239
Rule-based (GECToR)	4,131	1,771
RILEC	12,652	6,179

Table 3: L1-error dataset splits by train and test size. The GPT-, LLM-, and rule-based datasets represent synthetic augmented subsets.

Error Type	Generation Examples
CopExp	Overall all these places are in our age . (PPO) This is extremely unlikely for everyone and every scientist and medical worker in influence on world. (GECToR) Males in 66-75 ages made a slight jump compared to 56-65 years old males. (LLM)
Synonyms	If such a person should enter this prison, they should be given what they had already done. (PPO) In the one hand, people should make their living, and they should take the necessary education. (PPO) This number is 1.5 times higher than the previous ones during the departed period. (GECToR)
WFT	Also, the global total increase has fallen, and nearly 2 billion people, making it number 1 billion billions. (PPO) This will be girls’ longest run we will see in nearly every post on this web. (GECToR)
TenSem	It can be seen that in Africa the rate of unemployment increases at the same rate as in 2000, and then in Africa it remains at a steady rate. (Rule-Based) There is no changes in this part of the population in the period in which 2000 was presented. (GECToR) The proportion of old individuals aged 66 and over decrease in periods 1950-1995 from 6% to 4% and start growing after that. (LLM)
Transliteration	The percent of investitsment came in Russia, where in 2015 it was 5 billion Euros and in 2018 it is 8 billion Euros. (Rule-Based) She is receiving lots of funat emails. (LLM) This can be deduced from the fact that the highest rate of unemployment among South Asia fabricks Latin America in 2014 and 2015 was observed. (GECToR)

Table 4: Examples of ungrammatical model generations from RILEC , grouped by L1 error type. Data augmentation methods are indicated in brackets: PPO (Proximal Policy Optimization with DistilGPT2), GECToR (dictionary-based), Rule-based (tense replacement or transliteration), and LLM (prompt-based generation). CopExp = Copying Expression; WFT = Word Form Transmission; TenSem = Tense Semantics.

Dataset	$F1_{train}$	$F1_{test}$ Tag-specific
Dataset	$F1_{train}$	CopExp	Syn	TenSem	Transl.	WFT	Avg.
Base	55.66	15.79	46.60	75.44	83.33	57.14	55.66
GPT-Based	83.76	21.62	79.25	82.14	92.31	83.72	71.81
Rule-Based	78.53	31.82	61.40	77.06	84.62	75.00	65.98
LLM-Based	44.78	41.03	72.16	78.85	92.31	71.43	71.16
RILEC	83.00	33.33	69.39	80.00	96.55	90.48	73.95

Table 5: L1-interference error classification results for RoBERTa models fine-tuned on augmented subsets: augmented with DistilGPT optimized with PPO (GPT-based; § 4.1 ), with prompting LLMs (LLM-based; § 4.3 ), and obtained via rule-based generation (§ 4.2 ). We report average F1-scores on the training dataset and tag-specific scores. CopExp = Copying Expression; WFT = Word Form Transmission; TenSem = Tense Semantics. The best score is in bold, and the second-best score is underlined.

Figure 2: Error distribution in RILEC , including synthetic (S) and real (R) data. CopExp = Copying Expression; WFT = Word Form Transmission; TenSem = Tense Semantics.

Model	Test 100		REALEC 70
Model	Acc.	Dist.	TP	TP–FP
Base	0.66	0	6	4
GPT-based	0.74	14	15	7
Rule-based	0.74	10	17	13
LLM-based	0.68	8	8	5
RILEC	0.73	10	19	10

Table 6: Manual evaluation. Test 100 : Acc. = accuracy, Dist. = distinct TPs. REALEC 70 : TP = true positives, TP–FP = net correct predictions.

Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

Pipeline	Self-BLEU $\downarrow$	MAUVE $\uparrow$	3-gram Novelty $\uparrow$
GPT-based	15.25	0.47	0.70
LLM-based	12.24	0.97	0.20
Rule-based	12.75	0.36	0.92
All Augmented	14.03	0.48	0.92
REALEC-L1	16.37	1.00	0.00

Appendix

Table 7: Diversity and distributional similarity of augmented datasets relative to REALEC-L1. Lower Self-BLEU indicates greater diversity, while higher MAUVE and 3-gram Novelty indicate closer distributional alignment and higher n-gram novelty.

Language Model Error Detection

Language model error detection identifies unreliable answers, reasoning steps, and actions so systems can flag, verify, or repair them.

See all 90 papers

Individual and Combined Effects of English as a Second Language and Typos on LLM Performance
Apr 6, 2026Serena Liu, Yutong Yang, Prisha Sheth +9Language Model Robustness LLM Evaluation
Harvard University, Boston, MA 02138, US
Edit-level Majority Voting Mitigates Over-Correction in LLM-based Grammatical Error Correction
May 13, 2026Takumi Goto, Yusuke Sakai, Taro WatanabeSelf-Consistency Decoding Grammatical Error Correction
Nara Institute of Science and Technology (NAIST)
Investigating the Impact of Learner Errors on Social Bias in Large Language Models
Oct 21, 2026Federico Marcuzzi, Martin Christoph, Iryna Gurevych +1Social Bias in Language Models Language Model Bias Evaluation
INSAIT, Sofia University “St. Kliment Ohridski”, Bulgaria · Ubiquitous Knowledge Processing Lab (UKP Lab), Department of Computer Science, TU Darmstadt and National Research Center for Applied Cybersecurity ATHENE, Germany · Department of Computer Science at Queens College, City University of New York (CUNY), United States

Apr 6, 2026J/K move · Enter open · S save

Individual and Combined Effects of English as a Second Language and Typos on LLM Performance

Serena Liu, Yutong Yang, Prisha Sheth, Weixuan Dong+8

Harvard University, Boston, MA 02138, US

Language Model Robustness LLM Evaluation

Open paper AskPDF arXiv

Large language models (LLMs) are used globally, and because much of their training data is in English, they typically perform best on English inputs. As a result, many non-native English speakers interact with them in English as a second language (ESL), and these inputs often contain typographical errors. Prior work has largely studied the effects of ESL variation and typographical errors separately, even though they often co-occur in real-world use. In this study, we use the Trans-EnV framework to transform standard English inputs into eight ESL variants and apply MulTypo to inject typos at three levels: low, moderate, and severe. We find that combining ESL variation and typos generally leads to larger performance drops than either factor alone, though the combined effect is not simply additive. This pattern is clearest on closed-ended tasks, where performance degradation can be characterized more consistently across ESL variants and typo levels, while results on open-ended tasks are more mixed. Overall, these findings suggest that evaluations on clean standard English may overestimate real-world model performance, and that evaluating ESL variation and typographical errors in isolation does not fully capture model behavior in realistic settings.

Abstract

Language Model Error Detection

Explore similar work

Individual and Combined Effects of English as a Second Language and Typos on LLM Performance

Edit-level Majority Voting Mitigates Over-Correction in LLM-based Grammatical Error Correction

Investigating the Impact of Learner Errors on Social Bias in Large Language Models