MERIT: Memory-Enhanced Retrieval for Interpretable Knowledge Tracing

Runze Li; Kedi Chen; Guwei Feng; Mo Yu; Jun Wang; Wei Zhang

cs.CLMar 3, 2026

MERIT: Memory-Enhanced Retrieval for Interpretable Knowledge Tracing

Authors: Runze Li, Kedi Chen, Guwei Feng, Mo Yu, Jun Wang, Wei Zhang

Organizations: School of Computer Science and Technology, East China Normal University, China · Shanghai Innovation Institute, China · WeChat AI, Tencent, China

Abstract

Knowledge Tracing (KT) models students' evolving knowledge states to predict future performance, serving as a foundation for personalized education. While traditional deep learning models achieve high accuracy, they often lack interpretability. Large Language Models (LLMs) offer strong reasoning capabilities but struggle with limited context windows and hallucinations. Furthermore, existing LLM-based methods typically require expensive fine-tuning, limiting scalability and adaptability to new data. We propose MERIT (Memory-Enhanced Retrieval for Interpretable Knowledge Tracing), a training-free framework combining frozen LLM reasoning with structured pedagogical memory. Rather than updating parameters, MERIT transforms raw interaction logs into an interpretable memory bank. The framework uses semantic denoising to categorize students into latent cognitive schemas and constructs a paradigm bank where representative error patterns are analyzed offline to generate explicit Chain-of-Thought (CoT) rationales. During inference, a hierarchical routing mechanism retrieves relevant contexts, while a logic-augmented module applies semantic constraints to calibrate predictions. By grounding the LLM in interpretable memory, MERIT achieves state-of-the-art performance on real-world datasets without gradient updates. This approach reduces computational costs and supports dynamic knowledge updates, improving the accessibility and transparency of educational diagnosis.

Figures & tables

Figure 1. Overview of the MERIT framework. The architecture integrates a frozen LLM with an external cognitive memory across four stages. Cognitive Schema Discovery first discretizes the student space using semantic denoising and density-based clustering. Interpretative Memory Bank Construction then transforms expert reasoning traces into a static retrieval database. Subsequently, Hierarchical Cognitive Retrieval applies global centroid routing and local hybrid search to maintain domain consistency. Finally, Logic-Augmented Reasoning incorporates semantic difficulty calibration and explicit boundary constraints to regulate the prediction.

Dataset	ASSIST2009	ASSIST2012	Eedi	BePKT
# Responses	0.4m	2.7m	17.8m	23.9k
# Sequences	8.3k	67.1k	475.4k	1.9k
# Questions	6.9k	53.1k	2.7k	0.5k
# Concepts	200	265	386	100

Table 1. Statistics of the preprocessed datasets.

Method	Category	ASSIST09			ASSIST12			Eedi			BePKT
Method	Category	AUC	ACC	F1	AUC	ACC	F1	AUC	ACC	F1	AUC	ACC	F1
DKT	DL-based	0.7485	0.7315	0.7985	0.7512	0.7385	0.7993	0.7286	0.7065	0.8061	0.6985	0.6852	0.5647
AKT	DL-based	0.7684	0.7185	0.7925	0.7657	0.7374	0.7925	0.7388	0.7132	0.7994	0.6975	0.6845	0.5591
SAKT	DL-based	0.7582	0.7365	0.8005	0.7616	0.7361	0.8027	0.7411	0.7165	0.8082	0.7042	0.6904	0.5712
LPKT	DL-based	0.7533	0.7317	0.8038	0.7485	0.7248	0.8044	0.7457	0.7222	0.8106	0.6958	0.6807	0.5730
IKT	DL-based	0.7439	0.7194	0.7937	0.7583	0.7386	0.7795	0.7338	0.7135	0.7859	0.6885	0.6730	0.5614

Table 2. Overall performance comparison on ASSIST09, ASSIST12, Eedi, and BePKT. The best results are bolded , and the second best are underlined . Methods marked with * are training-free.

Method	ASSIST09			ASSIST12			Eedi			BePKT
Method	AUC	ACC	F1	AUC	ACC	F1	AUC	ACC	F1	AUC	ACC	F1
Base (w/o Retrieval)	0.6708	0.4794	0.3131	0.6749	0.4383	0.3601	0.7333	0.6855	0.7458	0.7392	0.6875	0.2038
w/o Global Routing	0.6715	0.4782	0.3297	0.6804	0.4500	0.3838	0.7312	0.6815	0.7433	0.7461	0.6925	0.2166
w/o Interpretative Traces	0.6709	0.4879	0.3482	0.6792	0.4487	0.3822	0.7295	0.6850	0.7460	0.7573	0.6800	0.1688
w/o Logic Constraints	0.7015	0.5012	0.3622	0.6887	0.4613	0.4035	0.7329	0.6935	0.7675	0.7617	0.6950	0.2278
MERIT ${}_{\textnormal{Gemini-2.5-Flash}}$	0.8244	0.7554	0.8054	0.7778	0.7441	0.8122	0.7969	0.7603	0.8139	0.7827	0.7850	0.6228

Table 3. Comprehensive ablation study results. The shaded rows denote the full MERIT framework, distinguishing it from the baseline (w/o Retrieval) and ablated variants.

Figure 2. Parameter sensitivity analysis on ASSISTments 2009 using Gemini-2.5-Flash. (Left) Impact of retrieval volume

k

. Performance peaks at

k=3

, suggesting that a compact context window limits noise from irrelevant samples. (Right) Sensitivity to hybrid search weight

\alpha

. A weight of

0.7

optimally balances latent semantic features with symbolic keyword matching.

Figure 3. UMAP visualization of cognitive schema discovery. (a) Embeddings from raw logs overlap significantly, suggesting statistical artifacts (e.g., IDs, scores) obscure underlying patterns. (b) Conversely, Semantic Denoising yields distinct, compact clusters, confirming effective separation of cognitive signals from noise.

Appendix figures & tables2 assets

Supplementary material from the paper’s appendix.

Appendix

Problem Definition
Symbol	Description
$\mathcal{U},\mathcal{E}$	Set of students and set of exercises, respectively.
$\mathcal{S}_{u}$	Interaction sequence for student $u$ : $\mathcal{S}_{u}=\{s_{1},s_{2},\dots,s_{t}\}$ .
$s_{t}=(e_{t},r_{t},d_{t})$	Interaction tuple at step $t$ , consisting of exercise content $e_{t}$ , response $r_{t}$ , and difficulty $d_{t}$ .
$r_{t}\in\{0,1\}$	Binary response correctness (0 indicates Incorrect, 1 indicates Correct).
$d_{t}\in[0,1]$	Continuous difficulty scalar for exercise $e_{t}$ .

Appendix

Table 4. Summary of key notations.

Phase 1: Input Analysis (The Trap)
Pipeline Stage	System Execution & Analysis
Target Student History	Sequence: [("Range", 0.0, Correct)] $\times$ 5 Observed Pattern: Perfect Correct Streak on minimal difficulty items.
Target Task	Concept: Median Difficulty: 1.0 (Max) Risk: Extreme Difficulty Spike .
Baseline Error	DKT predicts High Probability (0.85) due to strong positive momentum bias.
Phase 2: MERIT Evidence Retrieval (The Reality Check)
Retrieved Paradigm	Source: Memory Bank Paradigm #5 (Cluster #5) Historical Context: "Student had perfect streak on trivial items: Stem/Leaf Plot ( $d=0.0$ ), Median ( $d=0.0$ )…" Outcome: False on next question (Venn Diagram, $d=1.0$ ). Retrieved Analysis: $\hookrightarrow$ Key Pattern: "Difficulty Spike Failure" $\hookrightarrow$ Reasoning: "The student’s perfect streak on trivial (0.0 difficulty) questions created a misleading sense of momentum . The failure was caused by an extreme and simultaneous spike in concept difficulty (0.0 $\to$ 1.0)."

Appendix

Table 5. Detailed execution trace for the case study (Student ID: 712). The table contrasts the surface-level observation that misled the baseline model with the deep, memory-augmented reasoning process of MERIT. By retrieving a structurally similar paradigm (Paradigm #5), MERIT identifies the "Difficulty Spike" pattern and corrects the prediction.

Knowledge Tracing

Knowledge tracing models how learners’ mastery of skills changes over time from their interactions with educational tasks.

See all 31 papers

Explainable Knowledge Tracing via Probabilistic Embeddings and Pattern-based Reasoning
May 10, 2026Siyu Wu, Cong Xu, Wei ZhangRepresentation Learning Interpretable ML
1Shanghai Institute of AI Education, East China Normal University, Shanghai, 200241, China · 2Shool of Computer Sicence and Technology, East China Normal University, Shanghai, 200241, China
RAG-KT: Cross-platform Explainable Knowledge Tracing with Multi-view Fusion Retrieval Generation
Apr 13, 2026Zhiyi Duan, Hongyu Yuan, Rui LiuKnowledge Tracing AI in Education
Inner Mongolia University, Hohhot, China
Interpretable Difficulty-Aware Knowledge Tracing in Tutor-Student Dialogues
May 1, 2026Shuyan Huang, Alexander Scarlatos, Jaewook Lee +1Intelligent Tutoring Systems Interpretable ML
University of Massachusetts Amherst

May 10, 2026J/K move · Enter open · S save

Explainable Knowledge Tracing via Probabilistic Embeddings and Pattern-based Reasoning

Siyu Wu, Cong Xu, Wei Zhang

1Shanghai Institute of AI Education, East China Normal University, Shanghai, 200241, China · 2Shool of Computer Sicence and Technology, East China Normal University, Shanghai, 200241, China

Representation Learning Interpretable ML Knowledge Tracing

Open paper AskPDF arXiv

Knowledge Tracing (KT) models students' knowledge states based on learning interactions to predict performance. While deep learning-based KT models have boosted predictive accuracy, most models rely on deterministic vector embeddings and opaque latent state transitions, limiting interpretability regarding how specific past behaviors influence predictions. To address this limitation, we propose Probabilistic Logical Knowledge Tracing (PLKT), an interpretable KT framework that formulates prediction as a goal-conditioned evidence reasoning process over historical learning behaviors. Instead of representing knowledge states as deterministic vector embeddings, PLKT employs robust Beta-distributed probabilistic embeddings to represent student knowledge states. This probabilistic foundation allows us to model the uncertainty of historical behaviors and perform explicit logical operations (e.g., conjunction), constructing transparent reasoning paths that reveal how specific past interactions contribute to the prediction. Extensive experiments show that PLKT outperforms state-of-the-art KT methods while achieving superior interpretability. Our code is available at https://anonymous.4open.science/r/PLKT-D3CE/.

Abstract

Knowledge Tracing

Explore similar work

Explainable Knowledge Tracing via Probabilistic Embeddings and Pattern-based Reasoning

RAG-KT: Cross-platform Explainable Knowledge Tracing with Multi-view Fusion Retrieval Generation

Interpretable Difficulty-Aware Knowledge Tracing in Tutor-Student Dialogues