Learner-item cognitive modeling plays a central role in the web-based online intelligent education system by enabling cognitive diagnosis (CD) across diverse online educational scenarios. Although ID embedding remains the mainstream approach in cognitive modeling due to its effectiveness and flexibility, recent advances in language models (LMs) have introduced new possibilities for incorporating rich semantic representations to enhance CD performance. This highlights the need for a comprehensive analysis of how LMs enhance embeddings through semantic integration across mainstream CD tasks. This paper identifies two key challenges in fully leveraging LMs in existing work: Misalignment between the training objectives of LMs and CD models creates a distribution gap in feature spaces; A unified framework is essential for integrating textual embeddings across varied CD tasks while preserving the strengths of existing cognitive modeling paradigms to ensure the robustness of embedding enhancement. To address these challenges, this paper introduces EduEmbed, a unified embedding enhancement framework that leverages fine-tuned LMs to enrich learner-item cognitive modeling across diverse CD tasks. EduEmbed operates in two stages. In the first stage, we fine-tune LMs based on role-specific representations and an interaction diagnoser to bridge the semantic gap of CD models. In the second stage, we employ a textual adapter to extract task-relevant semantics and integrate them with existing modeling paradigms to improve generalization. We evaluate the proposed framework on four CD tasks and computerized adaptive testing (CAT) task, achieving robust performance. Further analysis reveals the impact of semantic information across diverse tasks, offering key insights for future research on the application of LMs in CD for online intelligent education systems.
Figures & tables
Figure 1. (a) Motivation study. (b) The comparison of our proposed EduEmbed with best-performing baseline methods on SLP.
Figure 2. The overall framework of the proposed EduEmbed. Stage 1: Role-aware Interaction Fine-tuning (RaIF). Stage 2: Adapter-aware Representation Integration (AaRI).
Datasets
SLP-Math
SLP-Chi
NeurIPS20
EDM
MOOC
# Students
1080
562
4918
2699
3000
# Exercises
609
510
948
1479
1967
# Knowledge Concepts
32
17
86
319
2278
# Response Logs
52100
28686
1382727
116156
333602
Average Correct Rate
0.506
0.623
0.545
0.628
0.812
Q Density
1.000
1.000
4.017
1.000
2.284
Table 1. Statistics of the real-world datasets.
Datasets
SLP-Math
NeurIPS20
EDM
Scenarios
Method
AUC
ACC
DOA
AUC
ACC
DOA
AUC
ACC
DOA
Transductive CD
MIRT
82.03 ± 0.01
74.81 ± 0.09
–
78.68 ± 0.01
71.77 ± 0.02
–
78.98 ± 0.03
74.36 ± 0.04
–
KaNCD
82.12 ± 0.13
74.67 ± 0.11
77.81 ± 0.13
78.57 ± 0.03
71.73 ± 0.04
66.61 ± 1.92
79.92 ± 0.13
74.40 ± 0.23
78.78 ∗ ± 0.12
ORCDF
82.37 ± 0.01
74.48 ± 0.13
78.24 ∗ ± 0.08
78.70 ± 0.03
71.79 ± 0.03
73.58 ± 0.04
82.63 ± 0.07
76.88 ± 0.03
77.84 ± 0.16
EduEmbed
82.23 ± 0.05
74.45 ± 0.11
77.85 ± 0.09
78.55 ± 0.01
71.75 ± 0.02
73.60 ± 0.01
82.59 ± 0.05
76.75 ± 0.02
77.65 ± 0.11
Inductive CD
ICDM
74.54 ± 0.03
68.83 ± 0.01
60.49 ± 0.02
71.72 ± 0.00
65.63 ± 0.01
59.00 ± 0.00
74.18 ± 0.01
70.54 ± 0.01
65.38 ± 0.01
Table 2. The overall performance of EduEmbed compared with the baseline methods in four CD tasks. Within each method, the highest mean performance is highlighted in bold. The value following “ ± ” denotes the standard deviation of the model’s performance. If a mean value is significantly higher than the second-best result according to a t -test with a significance level of 0.05, it is marked with “*”.
Method
AUC
ACC
DOA
LRCD
80.56 ± 0.12
72.59 ± 0.32
76.87 ± 0.04
EduEmbed
81.20 ∗ ± 0.21
73.69 ∗ ± 0.42
77.11 ∗ ± 0.08
Table 3. The performance of cross-subject CD on SLP. Other details are as same as Table 2 .
Dataset
SLP-Math
Metric
AUC / ACC (%)
Strategy
step
IRT
NCD
OL
EduEmbed
OL
EduEmbed
RANDOM
5
74.61 / 68.03
75.23 ∗ / 69.42 ∗
73.38 / 67.38
74.01 ∗ / 68.02 ∗
10
77.15 / 70.16
78.56 ∗ / 71.48 ∗
76.47 / 69.59
78.20 ∗ / 71.22 ∗
15
78.44 / 71.34
80.24 ∗ / 72.02 ∗
78.33 / 70.78
79.28 ∗ / 72.09 ∗
Table 4. The overall performance of EduEmbed with five CAT selection strategies on SLP-Math. “OL” stands for the original method under ID embedding paradigm.
CD Scenario
Metric
EduEmbed w/o RaIF
EduEmbed w/o RsR
EduEmbed w/o TA
EduEmbed
Transductive CD
AUC
82.27
82.24
82.06
82.23
ACC
74.40
74.40
74.38
74.45
DOA
77.75
77.44
76.78
77.85
Inductive CD
AUC
81.04
81.59
81.62
81.68
ACC
73.75
73.63
73.97
73.78
DOA
78.60
77.33
78.79
78.61
Table 5. Ablation study in four CD tasks on SLP-Math.
Figure 3. The performance of EduEmbed under varying LMs types and scales on SLP-Math.
Appendix figures & tables9 assets
Supplementary material from the paper’s appendix.
Appendix
Metric
AUC
ACC
DOA
TechCD
57.96
56.44
48.8
ZeroCD
61.77
59.07
50.81
LRCD
78.56
72.01
74.96
EduEmbed
78.74 ∗
72.32 ∗
75.30 ∗
Appendix
Table 6. The performance of EduEmbed with overlapping students. Other details are as same as Table 2 .
Metric
EduEmbed w/o IDI
EduEmbed w/o IDC
EduEmbed
AUC
82.21
82.05
82.23
ACC
74.40
74.27
74.45
DOA
77.50
77.59
77.85
Appendix
Table 7. Ablation study in transductive CD.
Metric
AUC / ACC (%)
CD Model
Step
EduEmbed w/o RaIF
EduEmbed w/o RsR
EduEmbed w/o TA
EduEmbed w/o IDI
EduEmbed w/o IDC
EduEmbed
IRT
5
67.72 / 61.83
73.71 / 58.49
73.21 / 66.91
76.69 / 69.30
76.29 / 67.87
77.45 / 70.40
10
73.91 / 65.54
76.00 / 69.53
73.78 / 67.34
79.15 / 71.67
78.20 / 70.64
79.02 / 71.48
15
74.95 / 68.72
80.05 / 71.94
74.64 / 67.81
81.27 / 73.50
78.92 / 69.87
81.33 / 73.38
NCD
5
62.26 / 57.20
72.25 / 65.64
70.65 / 64.41
69.83 / 62.57
73.37 / 64.07
72.36 / 65.74
10
65.30 / 61.62
75.69 / 67.12
77.35 / 69.80
75.92 / 69.31
73.48 / 68.05
77.26 / 69.57
Appendix
Table 8. Ablation study in CAT.
Figure 4. Visualization of students’ mastery levels on SLP-Math.
Figure 5. Comparison of LMs types in four CD tasks.
Figure 6. Comparison of LMs scales in four CD tasks.
Figure 7. Effect of text selection on MOOC. “OL” refer to baselines, specifically denoting ID embedding in transductive CD and IDCD for inductive CD.
Figure 8. Hyperparameter analysis on SLP-Math.
Metric
Text-Only
EduEmbed
AUC
75.53
82.23
ACC
68.93
74.45
DOA
76.60
77.85
Appendix
Table 9. Comparison of EduEmbed and “Text-Only” on SLP-Math in transductive CD.
Cognitive Diagnosis Models (CDMs) play a pivotal role in personalized online learning. Traditional CDMs rely on discrete, ID-based embeddings to represent students, exercises, and concepts. This paradigm diverges from the nature of learner cognition, where knowledge is not stored and retrieved as isolated symbols. As a result, CDMs suffer from semantic limitations when new exercises or concepts appear. In this paper, we propose a Process-aware Language Cognitive Diagnosis (PLCD) framework that uses language-derived structures as cognitive priors and response records to calibrate student posterior states. PLCD leverages large language models (LLMs) to construct concept schemas and cognitive process graphs, and uses target-conditioned semantic memory to retrieve historical responses that are relevant to each target exercise. A process-grounded Language-to-Cognition Mapper with DA-MoE experts and process-level contrastive learning then maps the textual evidence into a unified cognitive space. Experimental results show that PLCD not only outperforms traditional baselines in predicting student performance but also exhibits strong cognitive transfer capabilities. These results connect the computational power of LLMs with the psychometric goal of measuring latent knowledge states, suggesting that structured language priors calibrated by response records can improve cold-start robustness and cognitive grounding.
Minghang Liu, Yuanzhuo Wang, Qiang Qiu +2
State Key Laboratory of AI Safety, Institute of Computing Technology, CAS · University of Chinese Academy of Sciences
Knowledge Tracing (KT) models students' evolving knowledge states to predict future performance, serving as a foundation for personalized education. While traditional deep learning models achieve high accuracy, they often lack interpretability. Large Language Models (LLMs) offer strong reasoning capabilities but struggle with limited context windows and hallucinations. Furthermore, existing LLM-based methods typically require expensive fine-tuning, limiting scalability and adaptability to new data. We propose MERIT (Memory-Enhanced Retrieval for Interpretable Knowledge Tracing), a training-free framework combining frozen LLM reasoning with structured pedagogical memory. Rather than updating parameters, MERIT transforms raw interaction logs into an interpretable memory bank. The framework uses semantic denoising to categorize students into latent cognitive schemas and constructs a paradigm bank where representative error patterns are analyzed offline to generate explicit Chain-of-Thought (CoT) rationales. During inference, a hierarchical routing mechanism retrieves relevant contexts, while a logic-augmented module applies semantic constraints to calibrate predictions. By grounding the LLM in interpretable memory, MERIT achieves state-of-the-art performance on real-world datasets without gradient updates. This approach reduces computational costs and supports dynamic knowledge updates, improving the accessibility and transparency of educational diagnosis.
Runze Li, Kedi Chen, Guwei Feng +3
School of Computer Science and Technology, East China Normal University, China · Shanghai Innovation Institute, China · WeChat AI, Tencent, China
Large language models (LLMs) have been widely explored for embedding generation. While recent studies show that in-context learning (ICL) effectively enhances the representational capability of LLMs by prepending a few task-related demonstrations, it causes substantial token overhead due to the increased sequence length. In this work, we propose EPIC, a novel embedding-based in-context prompt training strategy that leverages ICL to generate high-quality embeddings while reducing computational burden during both training and inference. This approach replaces discrete text demonstrations with their corresponding continuous embeddings, which not only encourages the LLM to align semantically-related text pairs during contrastive learning, but also requires the model to interpret demonstration embeddings as part of the in-context prompt. Consequently, EPIC-trained models achieve excellent embedding performance both with or without in-context prompts at inference time. Comprehensive experiments demonstrate that our method establishes new state-of-the-art results on the MTEB benchmark, surpassing frontier models trained solely on publicly available retrieval data. Extensive ablation studies further validate the effectiveness and necessity of our mechanism.