Simulating Student-Tutor Interactions with LLMs for Multilingual Math Feedback

Junior Cedric Tonga; KV Aditya Srivatsa; Kaushal Kumar Maurya; Fajri Koto; Ekaterina Kochmar

cs.CLJun 5, 2025

Simulating Student-Tutor Interactions with LLMs for Multilingual Math Feedback

Authors: Junior Cedric Tonga, KV Aditya Srivatsa, Kaushal Kumar Maurya, Fajri Koto, Ekaterina Kochmar

Organizations: Mohamed bin Zayed University of Artificial Intelligence

Abstract

Large language models (LLMs) have demonstrated the ability to generate formative feedback and instructional hints in English, making them increasingly relevant for AI-assisted education. However, their ability to provide effective instructional support across different languages, especially for mathematically grounded reasoning tasks, remains largely unexamined. In this work, we present the first large-scale simulation of multilingual student-tutor interactions using LLMs. A stronger model plays the role of the tutor, generating feedback in the form of hints, while a weaker model simulates the student. We explore 352 experimental settings across 11 typologically diverse languages, four state-of-the-art LLMs, and multiple prompting strategies to assess whether language-specific feedback leads to measurable learning gains. Our study examines how student input language, tutor feedback language, model choice, and language resource level jointly influence performance. Results show that English hints are generally preferred, even by multilingual student LLMs, with few exceptions that lead to improved student learning outcomes. These findings offer practical insights for developing multilingual, LLM-based educational tools that are both effective and inclusive.

Figures & tables

Figure 1: Overview of the student-tutor interaction flow.

		En→En	En→En→L	En→L	L→L
Multilingual Setup: Student input prompt in native language
HRLs	Mistral-7B	11.30	10.90	9.50	11.00
HRLs	Aya-8B	4.00	3.60	3.60	3.90
LRLs	Mistral-7B	12.10	6.60	5.60	14.90
LRLs	Aya-8B	27.60	16.90	17.40	20.50
Avg. HRLs		7.65	7.25	6.55	7.45

Table 1: Mean student gains (%) across two experimental setups using LLaMA-3.3-70B as the tutor model. The best average values and second-best values are highlighted.

\Delta

rows indicate performance differences between Mistral-7B and Aya-8B within each language resource category (HRLs and LRLs).

		En→En		En→En→L		En→L		L→L
	Models	L-3.1	L-3.3	L-3.1	L-3.3	L-3.1	L-3.3	L-3.1	L-3.3
HRLs	Mistral-7B	0.51	0.80	0.51	0.80	0.51	0.74	1.43	0.63
HRLs	Aya-8B	0.46	0.51	0.46	0.51	0.17	0.29	0.57	0.06
LRLs	Mistral-7B	0.70	1.90	0.70	1.90	0.50	1.60	4.70	2.20
LRLs	Aya-8B	1.10	1.80	1.10	1.80	1.20	1.30	8.30	1.60

Table 2: Answer leakage proportions (%) across LLaMa-3.1-8B (as L-3.1) and Llama-3.3-70B (as L-3.3).

		En→En		En→En→L		En→L		L→L
	Models	Aya-8B	Mistral-7B	Aya-8B	Mistral-7B	Aya-8B	Mistral-7B	Aya-8B	Mistral-7B
HRLs	L-3.1	99.99	99.88	99.88	99.88	99.31	97.62	99.87	99.92
HRLs	L-3.3	100.00	99.77	99.94	99.88	99.94	99.71	99.94	99.94
LRLs	L-3.1	99.64	98.90	97.92	96.90	98.40	97.85	98.15	98.80
LRLs	L-3.3	100.00	99.80	98.95	99.17	99.40	99.37	99.47	99.40

Table 3: Mean language identification accuracy for hints with LLaMA-3.3-70B (as L.3.3) and LLaMA-3.1-8B (as L.3.1). The lowest number is bold.

		En→En	En→En→L	En→L	L→L
Initial solution
HRLs	Mistral-7b	99.8	99.8	99.8	99.8
HRLs	Aya	100.0	100.0	100.0	100.0
LRLs	Mistral-7b	96.8	96.8	96.7	96.8
LRLs	Aya	98.6	98.6	98.6	98.6
Revised solution

Table 4: Mean language identification accuracy (in %) of student models’ initial and revised solutions. Lowest value is bold for both initial and revised solutions.

Figure 2: Student Gain scores as a function of the mean number of hints required per language.

		En→En		En→En→L		En→L		L→L
	Models	Aya-8B	Mistral-7B	Aya-8B	Mistral-7B	Aya-8B	Mistral-7B	Aya-8B	Mistral-7B
HRLs	L-3.1	99.99	99.88	99.88	99.88	99.31	97.62	99.87	99.92
HRLs	L-3.3	100.00	99.77	99.94	99.88	99.94	99.71	99.94	99.94
LRLs	L-3.1	99.64	98.90	97.92	96.90	98.40	97.85	98.15	98.80
LRLs	L-3.3	100.00	99.80	98.95	99.17	99.40	99.37	99.47	99.40

Table 5: Mean language identification accuracy for hints with LLaMA-3.3-70B (as L.3.3) and LLaMA-3.1-8B (as L.3.1). The lowest number is bold.

		En→En	En→En→L	En→L	L→L
Initial solution
HRLs	Mistral-7b	99.8	99.8	99.8	99.8
HRLs	Aya	100.0	100.0	100.0	100.0
LRLs	Mistral-7b	96.8	96.8	96.7	96.8
LRLs	Aya	98.6	98.6	98.6	98.6
Revised solution

Table 6: Mean language identification accuracy (in %) of student models’ initial and revised solutions. Lowest value is bold for both initial and revised solutions.

Appendix figures & tables14 assets

Supplementary material from the paper’s appendix.

Appendix

Panel of Figure 3 — Figure 3: Relative improvement (Student gain G) in multilingual student-tutor interaction.

Feature	Mean Feature Value		T-Statistic	p-value
Feature	GSM8K	MGSM	T-Statistic	p-value
Gx_mean_numerical_word_rank	28638.85	28661.13	-0.413	0.680
Gx_word_arg_count	0.64	0.68	-0.407	0.684
Gx_world_knowledge	1.07	1.06	0.140	0.889
Qx_constituency_tree_depth	10.97	10.96	0.044	0.965
Qx_flesch_kinkaid_grade	4.19	4.18	0.071	0.943

Appendix

Table 7: Pairwise T-Test results between feature level mean values for GSM8K and MGSM.

English only prompting: when student input prompt is in English language
Languages	Llama 3.1-8B	Aya-8b	Mistral-7B	Llama-3.3-70B	Deepseek-R1-Llama-distill-70B
English	85.2	77.2	60.8	93.2	85.2
Spanish	76.8	72.0	46.0	86.0	80.8
French	74.8	68.0	44.8	86.4	78.4
German	74.8	68.0	40.0	86.8	82.4
Russian	72.4	76.8	43.6	90.8	80.8

Appendix

Table 8: Zero-shot accuracy (%) of language models across languages. These zero-shot scores serve as the baseline for each model. The table shows results for both English-only prompting (top) and multilingual prompting (bottom) setups. Llama-3.3-70B consistently achieves the highest accuracy across most languages, demonstrating superior cross-lingual capabilities in zero-shot settings.

Model	Reproduction Link
Llama-3.3-70B	https://www.together.ai/models/llama-3-3-70b-free
Mistral-7B	https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.3
Llama-3.1-8B	https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct
Aya-8b	https://huggingface.co/CohereLabs/aya-expanse-8b
GPT-4o	gpt-4o-2024-08-06

Appendix

Table 9: Models used in our experiments with their corresponding links for reproducibility.

Figure 4: Prompt for generating a candidate solution.

Figure 5: Prompt for revising the initial candidate solution

Figure 6: Prompt used by the tutor model to generate hints.

Figure 7: Prompt employed by GPT-4o to assess candidate answer correctness.

Figure 8: Prompt employed by GPT-4o to assess the correctness of the revised solution and categorize hints.

Axis	Values	Count
Languages	en, bn, de, es, fr, ja,	11
Languages	ru, sw, te, th, zh	11
Student Prompts	English-only, Multilingual	2
Student Models	Mistral-7B , Aya-8B	2
Hint Strategies	EN $\rightarrow$ EN, EN $\rightarrow$ EN $\rightarrow$ L,	4
Hint Strategies	L $\rightarrow$ L, EN $\rightarrow$ L	4

Appendix

Table 10: Overview of the experimental space.

Figure 9: Leakage ratio per language; numbers above bars indicate total helpful hints.

Language	BLEU	Language	BLEU
Spanish	62.7	Chinese	40.1
French	54.7	Japanese	45.6
German	46.7	Thai	40.0
Russian	49.3	Swahili	45.5
Bengali	43.0	Telugu	39.5

Appendix

Table 11: BLEU scores by language

Table 12: Student gains G (%) of Mistral-7B after receiving a hint from LlaMA-3.3-70B across English-only and multilingual prompting setups.

Table 13: Student gain G (%) of Aya-8b after receiving a hint from LlaMA-3.3-70B in the two setups.

Educational Feedback Generation

See all 19 papers

Large Language Models for Math Education in Low-Resource Languages: A Study in Sinhala and Tamil
Feb 16, 2026Sukumar Kishanthan, Kumar Thushalika, Buddhi Jayasekara +1Numerical Reasoning in Language Models Cross-Lingual Reasoning in Language Models
Department of Computer Science and Engineering University of Moratuwa Moratuwa, Sri Lanka · Department of Electrical and Information Engineering University of Ruhuna Galle, Sri Lanka · Faculty of Information Technology and Communication Sciences Tampere University Tampere, Finland +1
LANG: Reinforcement Learning for Multilingual Reasoning with Language-Adaptive Hint Guidance
May 21, 2026Yuchun Fan, Bei Li, Peiguang Li +9Multilingual Language Models Reinforcement Learning
NLP Lab, School of Computer Science and Engineering, Northeastern University, Shenyang, China · Meituan Inc. 3NiuTrans Research, Shenyang, China
LLM Parameters for Math Across Languages: Shared or Separate?
Jun 16, 2026Behzad Shomali, Luisa Victor, Tim Selbach +5Multilingual Language Models Cross-Lingual Reasoning in Language Models
Lamarr Institute · University of Bonn · Fraunhofer IAIS

Feb 16, 2026J/K move · Enter open · S save

Large Language Models for Math Education in Low-Resource Languages: A Study in Sinhala and Tamil

Sukumar Kishanthan, Kumar Thushalika, Buddhi Jayasekara, Asela Hevapathige

Department of Computer Science and Engineering University of Moratuwa Moratuwa, Sri Lanka · Department of Electrical and Information Engineering University of Ruhuna Galle, Sri Lanka · Faculty of Information Technology and Communication Sciences Tampere University Tampere, Finland+1

Numerical Reasoning in Language Models Cross-Lingual Reasoning in Language Models Multilingual Language Model Evaluation AI in Education Low-Resource Language Processing

Open paper AskPDF arXiv

Large language models (LLMs) have achieved strong results in mathematical reasoning, and are increasingly deployed as tutoring and learning support tools in educational settings. However, their reliability for students working in non-English languages, especially low-resource languages, remains poorly understood. We examine this gap by evaluating mathematical reasoning in Sinhala and Tamil -- two languages widely used in South Asian schools but underrepresented in artificial intelligence (AI) research. Using a taxonomy of six math problem types, from basic arithmetic to complex unit conflict and optimization problems, we evaluate four prominent large language models. To avoid translation artifacts that confound language ability with translation quality, we construct a parallel dataset in which each problem is independently authored in Sinhala and Tamil by native speakers, and in English by fluent speakers, all with strong mathematical backgrounds. Our analysis demonstrates that while basic arithmetic reasoning transfers robustly across languages, complex reasoning tasks show significant degradation in Tamil and Sinhala. The pattern of failures varies by model and problem type, suggesting that strong performance in English does not guarantee reliable performance across languages. These findings have direct implications for the deployment of AI tools in multilingual classrooms, and highlight the need for language-specific evaluation before adopting large language models as math tutoring aids in non-English educational contexts.

Abstract

Educational Feedback Generation

Explore similar work

Large Language Models for Math Education in Low-Resource Languages: A Study in Sinhala and Tamil

LANG: Reinforcement Learning for Multilingual Reasoning with Language-Adaptive Hint Guidance

LLM Parameters for Math Across Languages: Shared or Separate?