Can AI be a Teaching Partner? Evaluating ChatGPT, Gemini, and DeepSeek across Three Teaching Strategies

Talita de Paula Cypriano de Souza; Shruti Mehta; Matheus Arataque Uema; Luciano Bernardes de Paula; Seiji Isotani

cs.CYFeb 24, 2026

Can AI be a Teaching Partner? Evaluating ChatGPT, Gemini, and DeepSeek across Three Teaching Strategies

Authors: Talita de Paula Cypriano de Souza, Shruti Mehta, Matheus Arataque Uema, Luciano Bernardes de Paula, Seiji Isotani

Organizations: Federal Institute of São Paulo (IFSP), Major Fernando Valle Avenue, 2013 Braganca Paulista, 12903-000, SP, Brazil · Institute of Mathematics and Computer Sciences of University of São Paulo (USP), Trabalhador São-Carlense Avenue, 400 Sao Carlos, 13566-590, Sao Paulo, Brazil · Graduate School of Education of University of Pennsylvania (UPENN), 3700 Walnut Street Philadelphia, PA 19104 Philadelphia, 610101, Pennsylvania, United States · Nucleus of Excellence in Social Technologies (NEES) of Federal University of Alagoas (UFAL), Lourival Melo Mota Avenue, S/N Maceió, 57072-900, Alagoas, Brazil

Abstract

There are growing promises that Large Language Models (LLMs) can support students' learning by providing explanations, feedback, and guidance. However, despite their rapid adoption and widespread attention, there is still limited empirical evidence regarding the pedagogical skills of LLMs. This article presents a comparative study of popular LLMs, namely, ChatGPT, DeepSeek, and Gemini, acting as teaching agents. An evaluation protocol was developed, focusing on three pedagogical strategies: Examples, Explanations and Analogies, and the Socratic Method. Six human judges conducted the evaluations in the context of teaching the C programming language to beginners. The results indicate that LLM models exhibited similar interaction patterns in the pedagogical strategies of Examples and Explanations and Analogies. In contrast, for the Socratic Method, the models showed greater sensitivity to the pedagogical strategy and the initial prompt. Overall, ChatGPT and Gemini received higher scores, whereas DeepSeek obtained lower scores across the criteria, indicating differences in pedagogical performance across models.

Figures & tables

Reference	Domain	Human Ass.	Multi- turn	Interaction	Approach
Xu et al. (2025)	General	$\sim$	✓	Simulation	Benchmark
Maurya et al. (2025)	Math	✓	✗	Dataset	Benchmark
Macina et al. (2025)	Math	✗	✗	Dataset	Benchmark
Shi et al. (2025)	General	$\sim$	✓	Simulation	Framework
Joshi et al. (2025)	STEM/non STEM	$\sim$	✗	Dataset	Benchmark
Wang et al. (2025a)	Coding	$\sim$	✓	Simulation	Workflow

Table 1: Comparison of key features of related works and the proposed experiment.

Figure 1: Experimental Workflow of LLM Interaction and Evaluation

Figure 2: Process of interaction with LLM adopted by each judge

Criteria	ChatGPT			DeepSeek			Gemini
Criteria	M	Md	SD	M	Md	SD	M	Md	SD
Final Average	4.180	4.400	0.710	4.035	4.200	0.928	4.159	4.400	0.784
Relevance	4.607	5.000	0.919	4.280	5.000	1.243	4.360	5.000	0.822
Correctness	4.927	5.000	0.286	4.867	5.000	0.564	4.733	5.000	0.808
Level of detail	4.493	5.000	0.849	4.427	5.000	0.951	4.220	5.000	1.110
Variety	3.273	3.000	1.395	3.093	3.000	1.688	3.327	4.000	1.657

Table 2: Descriptive statistics of the criteria evaluated for ChatGPT, DeepSeek, and Gemini in the Examples Approach.

Figure 3: Distribution of evaluation grades for the Examples approach based on the criteria: (A) “Relevance” and (B) “Abstract-concrete connections”. Blue dots represent individual observations; boxes indicate the data distribution (quartiles), and the red line marks the median of each group — the value dividing the dataset into two equal halves and serving as a robust measure of central tendency. Black lines with asterisks denote statistically significant differences between models, identified by Dunn’s post hoc test ( $p<0.05$ ) following the Kruskal–Wallis test.

Criterion	H statistic	p-value
Final Average	0.8268	0.6614
Relevance ∗	16.3636	0.0003
Correctness	5.7260	0.0571
Level of detail	6.1636	0.0459
Variety	1.7964	0.4073
Abstract-concrete connections ∗	13.6923	0.0011

Table 3: Kruskal–Wallis Test Results for Models in Examples Approach (df = 2; n = 150)

Figure 4: “Judges’ perception” of Examples approach

Figure 5: “Provided immediate solution (non desirable)” criterion of Examples approach

Figure 6: Interaction in Examples approach: (A) ChatGPT 4.1-mini; (B) Gemini 2.5 flash.

Criteria	ChatGPT			DeepSeek			Gemini
Criteria	M	Md	SD	M	Md	SD	M	Md	SD
Final average	4.591	4.667	0.413	4.364	4.667	0.645	4.347	4.667	1.016
Clarity, consistency,
and ease	4.607	5.000	0.612	4.340	5.000	0.866	4.387	5.000	1.157
Critical parts focus	4.880	5.000	0.383	4.273	5.000	0.969	4.540	5.000	1.156
Correctness	4.893	5.000	0.369	4.840	5.000	0.465	4.693	5.000	1.042

Table 4: Descriptive statistics of the criteria evaluated for ChatGPT, DeepSeek, and Gemini in Explanations and Analogies Approach

Figure 7: Distribution of evaluation grades for Explanations and Analogies approach based on the criteria: (A)“Final average”; (B) “Clarity, consistency, and ease”; (C) “Critical parts focus”; (D) “Level adaptation”; (E) “Usefulness”. Blue dots represent individual observations; boxes indicate the data distribution (quartiles), and the red line marks the median of each group — the value dividing the dataset into two equal halves and serving as a robust measure of central tendency. Black lines with asterisks denote statistically significant differences between models, identified by Dunn’s post hoc test ( $p<0.05$ ) following the Kruskal–Wallis test.

Criterion	H statistic	p-value
Final average ∗	7.0449	0.0295
Clarity, consistency, and ease ∗	7.6037	0.0223
Critical parts focus ∗	53.9816	$<0.001$
Correctness	1.3195	0.5170
Level adaptation ∗	13.8414	0.0010
Usefulness ∗	26.8662	$<0.001$

Table 5: Kruskal–Wallis Test Results for Models in Explanations and Analogies Approach (df = 2; n = 150)

Figure 8: Comparison of ChatGPT, DeepSeek, and Gemini Percentages by Presence of Analogy and Explanation

Figure 9: “Judges’ perception” of Explanations and Analogies approach

Figure 10: “Provided immediate solution (non desirable)” criterion of Explanations and Analogies approach

Figure 11: Answers of Gemini 2.5 flash in Examples Approach and Explanations and Analogies Approach for the same exercise

Figure 12: ChatGPT 4.1-mini in Explanations and Analogies Approach

Criteria	ChatGPT			DeepSeek			Gemini
Criteria	M	Md	SD	M	Md	SD	M	Md	SD
Final average	4.631	4.600	0.423	4.051	4.200	0.839	4.427	4.600	0.524
Initial question	4.620	5.000	0.825	4.587	5.000	0.813	4.413	5.000	0.845
Counterexamples	4.413	5.000	0.906	3.313	4.000	1.777	4.160	5.000	1.124
Questions only	4.800	5.000	0.543	4.200	5.000	1.074	4.653	5.000	0.655
Well-formulated

Table 6: Descriptive statistics of the criteria evaluated for ChatGPT, DeepSeek, and Gemini in the Socratic Method Approach

Figure 13: Distribution of evaluation grades for Socratic Method approach for the criteria: (A) “Final average”; (B) “Initial question”; (C) “Counterexamples”; (D) “Questions only”; (E) “Well-formulated questions” criterion; (F) “Critical thinking promotion”. Blue dots represent individual observations; boxes indicate the data distribution (quartiles), and the red line marks the median of each group — the value dividing the dataset into two equal halves and serving as a robust measure of central tendency. Black lines with asterisks denote statistically significant differences between models, identified by Dunn’s post hoc test ( $p<0.05$ ) following the Kruskal–Wallis test.

Criterion	H statistic	p-value
Final average ∗	52.3954	$<0.001$
Initial question ∗	8.5192	0.0141
Counterexamples ∗	37.2175	$<0.001$
Questions only ∗	40.9931	$<0.001$
Well-formulated questions ∗	24.0971	$<0.001$
Critical thinking promotion ∗	49.3344	$<0.001$

Table 7: Kruskal–Wallis Test Results for Models in Socratic Method Approach (df = 2; n = 150)

Figure 14: “Judges’ perception” of Socratic Method approach

Figure 15: “Provided immediate solution (non desirable)” criterion of Socratic Method approach

Figure 16: The three LLMs in Socratic Method Approach

Figure 17: Gemini 2.5 flash in Socratic Method Approach

Figure 18: Evaluation of Pedagogical Skills Using the Examples approach. Asterisks indicate variables that showed a statistically significant difference in the statistical analysis.

Figure 19: Evaluation of Pedagogical Skills Using the Explanations and Analogies approach. Asterisks indicate variables that showed a statistically significant difference in the statistical analysis.

Figure 20: Evaluation of Pedagogical Skills Using the Socratic Method approach. Asterisks indicate variables that showed a statistically significant difference in the statistical analysis.

Appendix figures & tables22 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 21: DeepSeek V3 answer in Examples approach

Figure 22: DeepSeek V3 answer in Examples approach

Pairwise	p value	$p_{bonferroni}$
ChatGPT vs DeepSeek	0.0017	0.0051
ChatGPT vs Gemini	0.0002	0.0005
DeepSeek vs Gemini	0.5203	1.0000

Appendix

Table 8: Dunn’s with Bonferroni correction for “Relevance” criterion

Pairwise	p value	$p_{bonferroni}$
ChatGPT vs DeepSeek	0.7935	1.0000
ChatGPT vs Gemini	0.0233	0.0698
DeepSeek vs Gemini	0.0447	0.1342

Appendix

Table 9: Dunn’s with Bonferroni correction for “Level of detail” criterion

Pairwise	p value	$p_{bonferroni}$
ChatGPT vs DeepSeek	0.1257	0.3771
ChatGPT vs Gemini	0.0314	0.0943
DeepSeek vs Gemini	0.0002	0.0007

Appendix

Table 10: Dunn’s with Bonferroni correction for “Abstract-concrete connections” criterion

Pairwise	$\chi^{2}$	p value	$p_{bonferroni}$
ChatGPT vs DeepSeek	23.4354	$<0.001$	$<0.001$
ChatGPT vs Gemini	1.7284	0.4214	1.0000
DeepSeek vs Gemini	20.2985	$<0.001$	0.0001

Appendix

Table 11: Dunn’s with Bonferroni correction for “Judges’ perception” criterion

Pairwise	$\chi^{2}$	p value	$p_{bonferroni}$
ChatGPT vs DeepSeek	81.9751	$<0.001$	$<0.001$
ChatGPT vs Gemini	74.6148	$<0.001$	$<0.001$
DeepSeek vs Gemini	212.9071	$<0.001$	$<0.001$

Appendix

Table 12: Chi-square with Bonferroni correction for “Provided immediate solution (non desirable)” criterion

Pairwise	p value	$p_{bonferroni}$
ChatGPT vs DeepSeek	0.0079	0.0238
ChatGPT vs Gemini	0.1866	0.5599
DeepSeek vs Gemini	0.1823	0.5470

Appendix

Table 13: Dunn’s with Bonferroni correction for “Final average” criterion

Pairwise	p value	$p_{bonferroni}$
ChatGPT vs DeepSeek	0.0087	0.0262
ChatGPT vs Gemini	0.5659	1.0000
DeepSeek vs Gemini	0.0405	0.1215

Appendix

Table 14: Dunn’s with Bonferroni correction for “Clarity, consistence and ease” criterion

Pairwise	p value	$p_{bonferroni}$
ChatGPT vs DeepSeek	$<0.001$	$<0.001$
ChatGPT vs Gemini	$<0.001$	0.0985
DeepSeek vs Gemini	$<0.001$	$<0.001$

Appendix

Table 15: Dunn’s with Bonferroni correction for “Critical parts focus” criterion

Pairwise	p value	$p_{bonferroni}$
ChatGPT vs DeepSeek	0.0019	0.0057
ChatGPT vs Gemini	0.8214	1.0000
DeepSeek vs Gemini	0.0009	0.0026

Appendix

Table 16: Dunn’s with Bonferroni correction for “Level adaptation” criterion

Pairwise	p value	$p_{bonferroni}$
ChatGPT vs DeepSeek	0.0002	0.0005
ChatGPT vs Gemini	$<0.001$	$<0.001$
DeepSeek vs Gemini	$<0.001$	0.7126

Appendix

Table 17: Dunn’s with Bonferroni correction for “Usefulness” criterion

Pairwise	$\chi^{2}$	p value	$p_{bonferroni}$
ChatGPT vs DeepSeek	37.2355	$<0.001$	$<0.001$
ChatGPT vs Gemini	2.0493	0.3589	1.0000
DeepSeek vs Gemini	25.8560	$<0.001$	$<0.001$

Appendix

Table 18: Dunn’s with Bonferroni correction for “Judges’ perception” criterion

Pairwise	$\chi^{2}$	p value	$p_{bonferroni}$
ChatGPT vs DeepSeek	150.6215	$<0.001$	$<0.001$
ChatGPT vs Gemini	4.8572	0.0882	0.2645
DeepSeek vs Gemini	172.3212	$<0.001$	$<0.001$

Appendix

Table 19: Chi-square with Bonferroni correction for “Provided immediate solution (non desirable)” criterion

Pairwise	p value	$p_{bonferroni}$
ChatGPT vs DeepSeek	$<0.001$	$<0.001$
ChatGPT vs Gemini	0.0005	0.0017
DeepSeek vs Gemini	$<0.001$	$<0.001$

Appendix

Table 20: Dunn’s with Bonferroni correction for “Final average” criterion

Pairwise	p value	$p_{bonferroni}$
ChatGPT vs DeepSeek	0.3322	0.9967
ChatGPT vs Gemini	0.0041	0.0124
DeepSeek vs Gemini	0.0575	0.1726

Appendix

Table 21: Dunn’s with Bonferroni correction for “Initial question” criterion

Pairwise	p value	$p_{bonferroni}$
ChatGPT vs DeepSeek	$<0.001$	$<0.001$
ChatGPT vs Gemini	0.0678	0.2035
DeepSeek vs Gemini	$<0.001$	$<0.001$

Appendix

Table 22: Dunn’s with Bonferroni correction for “Counterexamples” criterion

Pairwise	p value	$p_{bonferroni}$
ChatGPT vs DeepSeek	$<0.001$	$<0.001$
ChatGPT vs Gemini	0.0408	0.1224
DeepSeek vs Gemini	$<0.001$	$<0.001$

Appendix

Table 23: Dunn’s with Bonferroni correction for “Questions only” criterion

Pairwise	p value	$p_{bonferroni}$
ChatGPT vs DeepSeek	$<0.001$	$<0.001$
ChatGPT vs Gemini	0.0986	0.2957
DeepSeek vs Gemini	0.0015	0.0045

Appendix

Table 24: Dunn’s with Bonferroni correction for “Well-formulated questions” criterion

Pairwise	p value	$p_{bonferroni}$
ChatGPT vs DeepSeek	$<0.001$	$<0.001$
ChatGPT vs Gemini	0.0058	0.0175
DeepSeek vs Gemini	$<0.001$	0.0001

Appendix

Table 25: Dunn’s with Bonferroni correction for “Critical thinking promotion” criterion

Pairwise	$\chi^{2}$	p value	$p_{bonferroni}$
ChatGPT vs DeepSeek	23.9315	$<0.001$	$<0.001$
ChatGPT vs Gemini	7.4427	0.0242	0.0726
DeepSeek vs Gemini	6.7810	0.0337	0.1011

Appendix

Table 26: Dunn’s with Bonferroni correction for “Judges’ perception” criterion

Pairwise	$\chi^{2}$	p value	$p_{bonferroni}$
ChatGPT vs DeepSeek	111.7237	$<0.001$	$<0.001$
ChatGPT vs Gemini	2.2307	0.3278	0.9834
DeepSeek vs Gemini	101.6538	$<0.001$	$<0.001$

Appendix

Table 27: Chi-square with Bonferroni correction for “Provided immediate solution (non desirable)” criterion

AI in Education

AI in Education studies how artificial intelligence can support teaching, learning, assessment, and educational decision-making.

See all 261 papers

Do Large Language Models Mentalize When They Teach?
Apr 2, 2026Sevan K. Harootonian, Mark K. Ho, Thomas L. Griffiths +2Theory of Mind
Department of Psychology, Princeton University · Department of Psychology, New York University · Department of Computer Science, Department of Psychology, Princeton University +1
Sherpa: Teaching LLMs to Teach Adaptively
Oct 6, 2026Weixian Xu, Yanzhe Zhang, Zora Zhiruo Wang +2RL for Language Models Intelligent Tutoring Systems
Stanford University · Georgia Tech · Carnegie Mellon University
Clause Encounters of the Third Kind: Can LLMs Replace Language Teachers?
Aug 17, 2026Kristina Šekrst, Ana KovačićEducational Technology LLM Evaluation
University of Zagreb

Apr 2, 2026J/K move · Enter open · S save

Do Large Language Models Mentalize When They Teach?

Sevan K. Harootonian, Mark K. Ho, Thomas L. Griffiths, Yael Niv+1

Department of Psychology, Princeton University · Department of Psychology, New York University · Department of Computer Science, Department of Psychology, Princeton University+1

Theory of Mind

Open paper AskPDF arXiv

How do LLMs decide what to teach next: by reasoning about a learner's knowledge, or by using simpler rules of thumb? We test this in a controlled task previously used to study human teaching strategies. On each trial, a teacher LLM sees a hypothetical learner's trajectory through a reward-annotated directed graph and must reveal a single edge so the learner would choose a better path if they replanned. We run a range of LLMs as simulated teachers and fit their trial-by-trial choices with the same cognitive models used for humans: a Bayes-Optimal teacher that infers which transitions the learner is missing (inverse planning), weaker Bayesian variants, heuristic baselines (e.g., reward based), and non-mentalizing utility models. In a baseline experiment matched to the stimuli presented to human subjects, most LLMs perform well, show little change in strategy over trials, and their graph-by-graph performance is similar to that of humans. Model comparison (BIC) shows that Bayes-Optimal teaching best explains most models' choices. When given a scaffolding intervention, models follow auxiliary inference- or reward-focused prompts, but these scaffolds do not reliably improve later teaching on heuristic-incongruent test graphs and can sometimes reduce performance. Overall, cognitive model fits provide insight into LLM tutoring policies and show that prompt compliance does not guarantee better teaching decisions.

Abstract

AI in Education

Explore similar work

Do Large Language Models Mentalize When They Teach?

Sherpa: Teaching LLMs to Teach Adaptively

Clause Encounters of the Third Kind: Can LLMs Replace Language Teachers?