When AI Meets Early Childhood Education: Large Language Models as Assessment Teammates in Chinese Preschools

Xingming Li; Runke Huang; Yanan Bao; Yuye Jin; Yuru Jiao; Qingyong Hu

cs.CLMar 25, 2026

When AI Meets Early Childhood Education: Large Language Models as Assessment Teammates in Chinese Preschools

Authors: Xingming Li, Runke Huang, Yanan Bao, Yuye Jin, Yuru Jiao, Qingyong Hu

Organizations: National University of Defense Technology, Changsha, China · The Chinese University of Hong Kong, Shenzhen, China · University of Oxford, Oxford, United Kingdom

Abstract

High-quality teacher-child interaction (TCI) is fundamental to early childhood development, yet traditional expert-based assessment faces a critical scalability challenge. In large systems like China's-serving 36 million children across 250,000+ kindergartens-the cost and time requirements of manual observation make continuous quality monitoring infeasible, relegating assessment to infrequent episodic audits that limit timely intervention and improvement tracking. In this paper, we investigate whether AI can serve as a scalable assessment teammate by extracting structured quality indicators and validating their alignment with human expert judgments. Our contributions include: (1) TEPE-TCI-370h (Tracing Effective Preschool Education), the first large-scale dataset of naturalistic teacher-child interactions in Chinese preschools (370 hours, 105 classrooms) with standardized ECQRS-EC and SSTEW annotations; (2) We develop Interaction2Eval, a specialized LLM-based framework addressing domain-specific challenges-child speech recognition, Mandarin homophone disambiguation, and rubric-based reasoning-achieving up to 88% agreement; (3) Deployment validation across 43 classrooms demonstrating an 18x efficiency gain in the assessment workflow, highlighting its potential for shifting from annual expert audits to monthly AI-assisted monitoring with targeted human oversight. This work not only demonstrates the technical feasibility of scalable, AI-augmented quality assessment but also lays the foundation for a new paradigm in early childhood education-one where continuous, inclusive, AI-assisted evaluation becomes the engine of systemic improvement and equitable growth.

Figures & tables

Figure 1: Overview of the TEPE-TCI dataset, comprising over 370 hours of teacher–child interaction audio from 41 public preschools ( 105 classrooms). Data were ethically approved, expert-annotated, and analyzed across levels.

Corpus	Language	Age Range	# Speakers	Hours	Style	Diar.	Quality FW	Year
CHIEDE [ 12 ]	Spanish	3-6	59	$\sim$ 8	Conversation	Partial	-	2008
CSLU Kid’s Speech Corpus [ 29 ]	English	K-G10	1,100	-	Read+Spont.	N	-	2007
CMU Kids Corpus [ 9 ]	English	6-11	76	-	Read speech	N	-	1997
PF-STAR Children’s Speech [ 2 ]	English	4-14	158	14.5	Read Speech	N	-	2005
MyST Corpus [ 26 ]	English	G3-G5	1,371	393	Conversation	Y	-	2024
TalkMoves [ 32 ]	English	K-12	-	-	Classroom	Y	-	2022

Table 1: Summary of children’s speech and educational interaction datasets. K denotes kindergarten, G denotes grade. Diar. indicates speaker diarization and Quality FW indicates quality assessment framework.

Figure 2: Overview of the Interaction2Eval pipeline.

Model	Raw CER (%)	After Refinement (%)	$\Delta$ (%) $\downarrow$
Whisper-large [ 28 ]	35.1	23.2	-11.9 ( $\downarrow$ 33.4%)
FunASR Paraformer [ 11 ]	9.9	4.3	-5.6 ( $\downarrow$ 56.6%)

Table 2: CER results.

\Delta

\downarrow

: absolute/relative reduction.

Figure 3: Frequently misrecognized terms in raw ASR outputs (term size reflects error frequency).

		Models
		GPT-5 [ 22 ]		Gemini-2.5-pro [ 7 ]		DeepSeek-v3.1 [ 17 ]		Qwen3-Max [ 39 ]
Scale	Dimension	$\kappa$	%Agr.	$\kappa$	%Agr.	$\kappa$	%Agr.	$\kappa$	%Agr.
ECQRS-EC	Literacy	0.678	0.844	0.736	0.871	0.705	0.886	0.764	0.882
	Mathematics	0.631	0.836	0.659	0.845	0.721	0.876	0.656	0.853
	Science	0.605	0.823	0.591	0.852	0.704	0.858	0.611	0.836
	Overall Mean	0.638	0.834	0.662	0.856	0.710	0.873	0.677	0.857

Table 3: Indicator-level agreement between LLM predictions and human expert annotations for ECQRS-EC [ 34 ] and SSTEW [ 30 ] scales, measured by percentage agreement (%Agr.) and Cohen’s Kappa (

\kappa

) which quantifies true agreement beyond chance following [ 6 ] .

Figure 4: Overview of the pilot deployment study and key outcomes.

Educational Assessment

Educational assessment measures what learners know and can do, using tests, rubrics, and learning evidence to guide decisions.

See all 151 papers

I code or AI code: A comparative evaluation of AI-rated scores in classroom observations
Sep 16, 2026Y. Fong, J. Xiang, T. Y. D. Chan +2LLM-as-a-Judge Educational Assessment
Department of Early Childhood Education, The Education University of Hong Kong, 10 Lo Ping Road, Tai Po, Hong Kong SAR, China · Yew Chung College of Early Childhood Education, 2 Tin Wan Hill Road, Tin Wan, Hong Kong SAR, China
Small, Private Language Models as Teammates for Educational Assessment Design
May 14, 2026Chris Davis Jaldi, Anmol Saini, Shan Zhang +3LLM Evaluation LLM-as-a-Judge
Wright State University, USA · University of Florida, USA · TIB – Leibniz Information Centre for Science and Technology, Germany
Knowledge Distillation for Automated AI Tutor Evaluation
Jul 12, 2026Tahmid Al Hannan, Diego Garcia, Alex Njoroge +2LLM-as-a-Judge AI in Education
Folsom Lake College Folsom, CA, USA

Sep 16, 2026J/K move · Enter open · S save

I code or AI code: A comparative evaluation of AI-rated scores in classroom observations

Y. Fong, J. Xiang, T. Y. D. Chan, K. Lee+1

Department of Early Childhood Education, The Education University of Hong Kong, 10 Lo Ping Road, Tai Po, Hong Kong SAR, China · Yew Chung College of Early Childhood Education, 2 Tin Wan Hill Road, Tin Wan, Hong Kong SAR, China

LLM-as-a-Judge Educational Assessment AI in Education Automated Evaluation Generative AI Evaluation

Open paper AskPDF arXiv

Classroom observations are widely recognized as a key tool for establishing benchmarks of education quality and guiding pedagogical improvement, yet they remain resource-intensive and dependent on trained observers. This study evaluated the feasibility of using a LLM (GPT-5 model) to score teacher-child interactions in early childhood classrooms, benchmarked against human raters. The study analyzed 87 video-recorded observations from 38 classrooms across 30 kindergartens in Hong Kong. Using observation transcripts, the AI model was configured to apply the full Classroom Assessment Scoring System (CLASS) framework. AI-rated scores were then compared with human ratings by examining correlations and differences in mean scores of the CLASS domains and dimensions. The results showed greater convergence between AI and raters for the Emotional Support domain and, in particular, the Quality of Feedback dimension, which captures how teachers use feedback to extend children's learning. Greater divergence emerged for interactions that were more procedural or context-dependent, particularly within the Classroom Organization and Instructional Support domains. These findings suggest that transcript-based AI scoring may capture some of the relative variation in teacher-child interactions but cannot yet reproduce calibrated human judgements consistently across the full CLASS framework. AI-assisted observation may therefore be more appropriate as a preliminary screening tool rather than as a replacement for trained observers, providing teachers with evidence for reflection rather than high-stakes evaluation. Future research should examine whether domain-specific training and incorporation of contextual and visual information can improve alignment between AI and human rated scores.

Abstract

Educational Assessment

Explore similar work

I code or AI code: A comparative evaluation of AI-rated scores in classroom observations

Small, Private Language Models as Teammates for Educational Assessment Design

Knowledge Distillation for Automated AI Tutor Evaluation