SLATE: Are AI-Generated Slides Educationally Effective? A Benchmark for Language Teaching Quality and Learner Knowledge Acquisition

Jingzhuo Wu; Jiajun Zhang; Liu Yi; Leqi Zheng; Yuheng Jing; Xinyuan Zhou; Quan yang

cs.CLSep 5, 2026

SLATE: Are AI-Generated Slides Educationally Effective? A Benchmark for Language Teaching Quality and Learner Knowledge Acquisition

Authors: Jingzhuo Wu, Jiajun Zhang, Liu Yi, Leqi Zheng, Yuheng Jing, Xinyuan Zhou, Quan yang

Organizations: Beijing Normal University · University of Science and Technology of China · Institute of Automation, Chinese Academy of Sciences · Beijing Language and Culture University ·

Abstract

LLMs have achieved remarkable capabilities in generating language teaching slides. However, a critical mismatch persists between visual polish and actual instructional effectiveness. To address this gap, we introduce SLATE (Slide-based Learning Assessment for Teaching Effectiveness), the first benchmark that evaluates AI-generated language teaching slides through instructional effectiveness and learner knowledge acquisition. SLATE transforms linguistics olympiad puzzles from low-resource languages with negligible web presence into 90 standardized instructional units comprising 1,133 assessable items, paired with a structured course outline and matched near- and far-transfer test sets. This pretest-posttest design eliminates pretrained knowledge leakage, ensuring gains reflect learning rather than prior recall. Using VLMs as scalable learner proxies and directionally supported by a three-system human pilot, our results show that content validity exhibits a weak association with learning gain, while pedagogical design exhibits a robust positive association. Moreover, most systems show a significant gap between near- and far-transfer accuracy, and even frontier models can produce negative learning gains. SLATE reveals a dissociation between artifact quality and instructional effectiveness, calling for a paradigm shift in how generative teaching systems are built, evaluated, and deployed.

Figures & tables

Figure 1: AI-generated slides can appear visually polished and pedagogically plausible, yet still fail to help learners acquire target concepts or solve novel problems after study.

	Scope		Evaluation
Benchmark	Slide	Lang.	Artifact	Pedagog.	Learner	Transfer
	gen.	teaching	quality	grounding	outcomes	eval.
PPTEval ( Zheng et al., 2025b )	✓	✗	✓	✗	✗	✗
SlidesGen-Bench ( Yang et al., 2026 )	✓	✗	✓	✗	✗	✗
PresentBench ( Chen et al., 2026 )	✓	✗	✓	✗	✗	✗
TutorBench ( Srinivasa et al., 2025 )	✗	✓	✗	✓	✗	✗

Table 1: Comparison of SLATE with existing slide generation and educational benchmarks.

Figure 2: Overview of SLATE . Source linguistics olympiad puzzles are transformed into instructional units; generation models produce teaching decks; D1–D4 evaluate content, pedagogy, visual support, and learner transfer.

Property	Value
Instructional units	90
Assessment sub-items	1,133
Task types	7
Difficulty bands	3
Basic / Intermediate / Challenge	2 / 70 / 18
Avg. teachable rules per unit	5.8

Table 2: SLATE benchmark statistics.

System	Artifact Quality		D3	D4: Learning & Transfer
System	D1	D2	VA%	Gain (%)	Post	Near	Far
Claude Opus 4.7 ( Anthropic, 2026a )	4.19	4.42	99	$\uparrow$ 5.46	29.25	41.67	12.22
Qwen3.6-Max ( Qwen Team, 2026 )	3.33	3.95	99	$\uparrow$ 5.14	28.92	39.89	12.36
MiniMax-M2.7 ( MiniMax, 2026 )	3.77	3.91	99	$\uparrow$ 3.27	27.06	38.33	15.00
GPT-5.4 ( OpenAI, 2026 )	4.20	4.24	99	$\uparrow$ 2.61	26.40	35.00	13.33
Claude Sonnet 4.6 ( Anthropic, 2026b )	4.17	3.72	98	$\uparrow$ 1.85	25.63	41.67	12.22

Table 3: Main results on SLATE (sorted by D4 gain). D1/D2 are multi-agent judge averages (1–5 scale; 5 runs

\times

3 judges). VA%: visual aid completeness (full D3 breakdown in Appendix). D4 gain is measured against the pretest baseline of 23.78%. Bold : best; underline : second best.

		Student Model
Cond.	Setting	Q25-3B	Q3-4B	Q25-7B	Q3-8B
A	Ctx only	14.6	19.9	16.9	22.9
B	Ctx + slides	13.1	24.2	20.9	28.0
V1	No input	9.0	9.2	8.0	10.4
V2	Slides only	13.1	17.7	16.1	20.4
V3	Ctx + 3-frame	13.3	20.8	17.7	24.4

Table 4: Ablation study across 4 student VLMs and 6 conditions using Claude Opus 4.7 slides. Upper block: accuracy (%); lower block: effect sizes (%).

Figure 3: Ablation effect decomposition across 4 student VLMs. Context is the strongest factor; slide effects scale with model capacity; full-frame consistently outperforms 3-frame sampling.

Figure 4: Near- vs. far-transfer accuracy for each system (color = posttest gain, size = $|$ gain $|$ ). Systems cluster in a high-near / low-far region, indicating a systematic gap between pattern reuse and conceptual generalization.

System	Parad.	Trans.	Morph.	Other
	(n=37)	(n=31)	(n=10)	(n=12)
Claude Opus 4.7	$\uparrow$ 8.5	$\uparrow$ 0.8	$\uparrow$ 2.2	$\uparrow$ 11.1
Qwen3.6-Max	$\uparrow$ 9.4	$\uparrow$ 0.5	$\uparrow$ 2.1	$\uparrow$ 6.2
MiniMax-M2.7	$\uparrow$ 7.9	$\downarrow$ 0.9	$\downarrow$ 2.2	$\uparrow$ 4.4
GPT-5.4	$\uparrow$ 4.5	$\downarrow$ 0.4	$\uparrow$ 2.3	$\uparrow$ 4.9
Claude Sonnet 4.6	$\uparrow$ 4.7	$\downarrow$ 0.8	$\downarrow$ 2.3	$\uparrow$ 3.3

Table 5: D4 gain (%) by task type. Paradigm completion is the most teachable; translation tasks show weakest gains.

Figure 5: D1/D2 scores vs. D4 gain. D2 (pedagogical design) shows a stronger positive association with learning gain (

\rho=0.72

) than D1 (knowledge validity;

\rho=0.38

), though

N=10

limits statistical power. Green = positive gain; red = negative gain.

Metric	H–H	H–M
	ICC	Pearson $r$
D1.1 Rule coverage	0.82	0.81
D1.2 Data fidelity	0.78	0.79
D1.3 Metalinguistic acc.	0.71	0.75
D1.4 Answer non-leakage	0.85	0.83
D1.5 Scope approp.	0.76	0.77

Table 6: D1/D2 reliability. H–H: human–human ICC; H–M: human–model Pearson

r

Appendix figures & tables17 assets

Supplementary material from the paper’s appendix.

Appendix

Factor	0 points	1 point	2 points
Rule complexity	Single rule with direct application	Two conditions or a limited rule variant	Interacting conditions, chained rules, or multiple dependent transformations
Exception density	No explicit exceptions	One or two regularized exceptions	Three or more exceptions, or exceptions that themselves require rule learning
Representation density	Compact material, low symbolic load	Moderate data length or moderate symbolic complexity	Large representations, dense glossing, IPA / tone-heavy notation, or high symbolic compression
Transfer abstraction	Transfer closely resembles source application	Transfer requires modest recombination or cross-example abstraction	Transfer requires abstraction beyond surface format or combination of multiple rules
Distractor strength	No meaningful distractors	Limited surface-similar distractors	Multiple strong distractors that align with plausible learner misconceptions

Appendix

Table 7: Instructional difficulty rubric used to assign Basic / Intermediate / Challenge bands.

Theory	Primary dimensions	Metric implications
Cognitive Load Theory	D2, D3	Supports metrics for information density, load control, and reduction of extraneous complexity
Multimedia Learning Theory	D2, D3	Supports signaling, segmentation, layout coherence, and alignment between explanation and example
Dual Coding Theory	D3	Supports verbal–visual pairing, rule visualization, and interpretable non-verbal encoding
Scaffolding / ZPD	D2	Supports progressive sequencing, guided explanation, and the transition from supported to more independent application
Transfer Theory	D4	Supports the distinction between near and far transfer and motivates abstraction-sensitive evaluation
Signaling / Communication Theory	D3, D2	Supports consistent use of emphasis, highlighting, grouping, and other cues that guide interpretation

Appendix

Table 8: High-level mapping from learning-science theory to the four benchmark dimensions.

Dimension	Sub-dimension	ICC	Agreement level
D1	D1.1 Rule coverage	0.82	Excellent
D1	D1.2 Data fidelity	0.78	Excellent
D1	D1.3 Metalinguistic accuracy	0.71	Good
D1	D1.4 Answer non-leakage	0.85	Excellent
D1	D1.5 Scope appropriateness	0.76	Good
D1	D1 average	0.80	Excellent

Appendix

Table 9: Human–human agreement for D1 and D2 using intraclass correlation coefficients.

Dimension	Sub-dimension	Pearson $r$	MAE
D1	D1.1 Rule coverage	0.81	0.32
D1	D1.2 Data fidelity	0.79	0.35
D1	D1.3 Metalinguistic accuracy	0.75	0.41
D1	D1.4 Answer non-leakage	0.83	0.28
D1	D1.5 Scope appropriateness	0.77	0.38
D1	D1 average	0.80	0.34

Appendix

Table 10: Human–model agreement for D1 and D2, comparing model scores with mean expert scores.

System	Slides	WPS	Tables	Colors	VA%	Exer.%
Claude Opus 4.7	13.2	72.6	7.0	17.0	99	100
Qwen3.6-Max	12.3	76.5	4.9	19.9	99	99
MiniMax-M2.7	12.0	83.5	6.7	25.8	99	100
GPT-5.4	12.3	72.1	6.7	16.3	99	100
Claude Sonnet 4.6	6.0	95.4	3.2	36.7	98	96
Qwen3.6-Plus	11.6	71.4	4.4	17.5	99	100

Appendix

Table 11: Full D3 visual design metrics. Slides: mean slide count per deck; WPS: words per slide; Tables: mean table count; Colors: distinct CSS colors; VA%: visual aid completeness; Exer.%: exercise inclusion rate.

System	Basic	Interm.	Challenge
Claude Opus 4.7	$\downarrow$ 4.17	$\uparrow$ 0.11	$\uparrow$ 3.63
Qwen3.6-Max	$\uparrow$ 12.50	$\uparrow$ 2.43	$\uparrow$ 2.26
MiniMax-M2.7	$\uparrow$ 4.17	$\downarrow$ 2.33	$\uparrow$ 7.68
GPT-5.4	$\uparrow$ 12.50	$\downarrow$ 4.82	$\uparrow$ 1.80
Claude Sonnet 4.6	$\uparrow$ 12.50	$\downarrow$ 6.65	$\downarrow$ 0.64
Qwen3.6-Plus	$\downarrow$ 4.17	$\downarrow$ 7.14	$\downarrow$ 2.31

Appendix

Table 12: Posttest gain (%) by difficulty band. Basic contains only 2 units and is highly unstable.

Condition	Absolute Scores (10-point scale)				Drop vs. Original
Condition	D1	D2	D3	D4	$\Delta$ D1	$\Delta$ D2	$\Delta$ D3	$\Delta$ D4
Original deck	8.5	8.2	8.8	8.0	–	–	–	–
Knowledge-corrupted	4.0	7.8	8.5	5.0	-4.5	-0.4	-0.3	-3.0
Pedagogically-corrupted	8.1	3.8	8.4	4.8	-0.4	-4.4	-0.4	-3.2
Visually-corrupted	8.2	7.9	3.5	5.5	-0.3	-0.3	-5.3	-2.5
Assessment-integrity-corrupted	7.0	7.9	8.6	7.8	-1.5	-0.3	-0.2	-0.2

Appendix

Table 13: Main error injection study results (mean scores over 10 reference decks). Absolute scores and score drops are reported relative to the original deck.

Condition	Pre	Post	Gain	Near	Far
Original deck	5.0	8.5	+3.5	8.0	7.5
Knowledge-corrupted	5.0	6.0	+1.0	5.5	3.0
Pedagogically-corrupted	5.0	6.2	+1.2	4.0	3.5
Visually-corrupted	5.0	6.8	+1.8	5.0	6.0
Assess.-integrity-corrupted	5.0	9.0	+4.0	7.8	2.5

Appendix

Table 14: D4-specific learning outcomes under controlled perturbations (mean over 10 reference decks; all scores on a 10-point scale).

Figure 6: Representative slides from the highest-gain deck (P187, Claude Opus 4.7, $\uparrow$ 68%). The deck uses color-coded tables, explicit rule statements, and worked examples to teach Wik-Mungkan compound noun formation.

Figure 7: Representative slides from the largest-negative-gain deck (P131, Gemini-3.1-Pro, $\downarrow$ 50%). Despite clear category labeling and color coding, the deck disrupts the student model’s existing priors through rigid re-categorization, high phonological similarity among verb forms (e.g., cŋs vs. cŋæs vs. crŋir), and cognitive-path mismatch, resulting in severe negative transfer after study.

Figure 8: Representative slides from the artifact-quality gap case (P144, GPT-5.4, $\downarrow$ 30% despite D1=4.20). The deck features professional two-column layouts and color-coded morpheme tables, yet posttest accuracy drops while near-transfer remains perfect (100%), consistent with format-induced response shift and prior-knowledge disruption.

Case	System	Pre	Post	Gain	Near	Far
P187 Success	Opus 4.7	4	72	$\uparrow$ 68	100	0
P131 Failure	Gem-3.1	72	22	$\downarrow$ 50	100	0
P144 Gap	GPT-5.4	70	40	$\downarrow$ 30	100	50

Appendix

Table 15: Summary of three case study decks. All values are percentages. The success case achieves the benchmark’s largest gain; the failure case has the largest negative gain; the gap case illustrates high artifact quality with negative learning outcomes.

Abbreviation	Full Model Name
Q25-3B	Qwen2.5-VL-3B-Instruct
Q3-4B	Qwen3-VL-4B-Instruct
Q25-7B	Qwen2.5-VL-7B-Instruct
Q3-8B	Qwen3-VL-8B-Instruct

Appendix

Table 16: Mapping of abbreviated student model identifiers to full model names.

Learner	System	Pre	Post	Gain	Near	Far
Human	Claude Opus 4.7 ( $n$ =10)	16.2	31.8	$\uparrow$ 15.6	41.5	18.5
	Claude Sonnet 4.6 ( $n$ =10)	16.5	28.3	$\uparrow$ 11.8	38.3	15.0
	Gemini-3.1-Pro ( $n$ =10)	17.1	14.4	$\downarrow$ 2.7	28.3	11.7
	Overall ( $n$ =30)	16.6	24.8	$\uparrow$ 8.2	36.0	15.1
VLM (Q3-8B)	Claude Opus 4.7	23.8	29.2	$\uparrow$ 5.5	41.7	12.2
	Claude Sonnet 4.6	23.8	25.6	$\uparrow$ 1.9	41.7	12.2

Appendix

Table 17: Human vs. VLM performance (%) across three systems. Key finding : System-level rankings are perfectly concordant between human and VLM learners (Spearman

\rho=1.0

Table 18: Per-participant results (%, averaged across each participant’s 3 puzzles) by system condition.

Comparison	Statistic	$p$ -value	Sig.
Opus 4.7 vs. Sonnet 4.6	$t(18)=2.27$	0.032
Sonnet 4.6 vs. Gemini-3.1-Pro	$t(18)=7.92$	$<0.001$
Opus 4.7 vs. Gemini-3.1-Pro	$t(18)=10.19$	$<0.001$

Appendix

Table 19: Post-hoc Tukey HSD tests for cross-system gain differences. *

p<0.05

, ***

p<0.001

Task	Opus	Sonnet	Gemini	Agree
Paradigm Completion	$\uparrow$ 18.2	$\uparrow$ 13.5	$\downarrow$ 1.8	$\boldsymbol{\checkmark}$
Translation	$\uparrow$ 1.2	$\uparrow$ 0.3	$\downarrow$ 4.1	$\boldsymbol{\checkmark}$
Morphophonology	$\uparrow$ 2.1	$\uparrow$ 1.5	$\downarrow$ 3.2	$\boldsymbol{\checkmark}$
Other	$\uparrow$ 3.5	$\uparrow$ 2.8	$\downarrow$ 1.9	$\boldsymbol{\checkmark}$

Appendix

Table 20: Human learning gains (%) stratified by task type. All show perfect human-VLM ranking consistency.

Generative AI in Education

Generative AI in education uses language and multimodal models to support teaching, learning, feedback, assessment, and educational design.

See all 282 papers

AI-Generated Slides: Are They Good? Can Students Tell?
May 13, 2026Juho Leinonen, Lisa Zhang, Arto HellasGenerative AI in Education Generative AI Evaluation
Aalto University · Espoo, Finland · University of Toronto Mississauga +1
Small, Private Language Models as Teammates for Educational Assessment Design
May 14, 2026Chris Davis Jaldi, Anmol Saini, Shan Zhang +3LLM Evaluation LLM-as-a-Judge
Wright State University, USA · University of Florida, USA · TIB – Leibniz Information Centre for Science and Technology, Germany
X+Slides: Benchmarking Audience-Conditioned Slide Generation
Jun 17, 2026Haodong Chen, Xuanhe Zhou, Wei Zhou +6Evidence-Grounded Generation Conditional Generation
Shanghai Jiao Tong University · Harbin Institute of Technology · SenseTime

May 13, 2026J/K move · Enter open · S save

AI-Generated Slides: Are They Good? Can Students Tell?

Juho Leinonen, Lisa Zhang, Arto Hellas

Aalto University · Espoo, Finland · University of Toronto Mississauga+1

Generative AI in Education Generative AI Evaluation

Open paper AskPDF arXiv

As generative AI (GenAI) tools become easily accessible, there is promise in using such tools to support instructors. To that end, this paper examines using GenAI to help generate slides from instructor authored course notes, emphasizing instructor and student perceptions. We examine an end-to-end education tool (NotebookLM), two general-purpose LLMs (Claude, M365 Copilot), and two coding assistants (Cursor, Claude Code). We first analyze whether GenAI generated slides are

good'' via narrative assessment by educators. We choose the best slides to use (with some modification) in a real course setting, and compare the student perception of human vs. AI generated slides. We find that coding assistant tools produce slides that were most accurate, complete, and pedagogically sound. Additionally, students rate GenAI slides to be of similar quality as instructor-created slides, and cannot reliably identify which slides are AI-generated. Additionally, we find a negative correlation between a high quality rating and a high

AI-generated'' rating, suggesting students associate poor quality with the source of the slides being AI. These findings highlight promising opportunities for integrating GenAI into instructional design workflows and call for further research on how educators can best harness such tools responsibly and effectively.

Abstract

Generative AI in Education

Explore similar work

AI-Generated Slides: Are They Good? Can Students Tell?

Small, Private Language Models as Teammates for Educational Assessment Design

X+Slides: Benchmarking Audience-Conditioned Slide Generation