CaRe-KT: Continuity-Aware Retrieval-Augmented Reasoning for Knowledge Tracing
Authors: Mehrnoush Mohammadi, Kamal Berahmand, Mohammad Morsali, Shazia Sadiq, Wojtek Tomaszewski, Hassan Khosravi
Organizations: School of Electrical Engineering and Computer Science, The University of Queensland, Brisbane, Australia. · School of Computing Technologies, RMIT University, Melbourne, Australia. · School of Telecommunications and Aerospace Engineering , Universitat Polit‘ecnica de Catalunya, Spain. · Australian Research Council Centre of Excellence for Children and Families over the Life Course, The University of Queensland, Brisbane, Australia. · Institute for Teaching and Learning Innovation, The University of Queensland, Brisbane, Australia.
Knowledge tracing (KT) estimates learners' evolving knowledge states from interaction histories to support adaptive learning. Retrieval-augmented KT improves these estimates using cross-learner behavioral evidence, but existing approaches typically separate evidence retrieval from downstream prediction. This creates a retrieval-inference alignment gap because evidence relevance varies with the learner's evolving state and prediction context. We propose CaRe-KT, a query-adaptive framework that maintains relevance continuity between evidence acquisition and inference without equating retrieval relevance with predictive utility. CaRe-KT introduces an Evidence-Graph Cache (EGC) that combines persistent question-level relational compatibility with interaction-level temporal validity to construct query-specific evidence subgraphs. Historical learning trajectories retrieved through these subgraphs are encoded alongside the target learner's history in a shared representation space using outcome-supervised contrastive learning. An attention mechanism combines acquisition relevance with learned query-evidence compatibility, allowing relevance to inform, rather than determine, each trajectory's predictive influence. Experiments across five benchmarks show that CaRe-KT outperforms strong KT baselines and remains robust under sparse-history and cold-start conditions. These findings support relevance continuity as a design principle for retrieval-augmented knowledge tracing.
Figures & tables
Fig. 1: Retrieval–inference alignment in retrieval-augmented KT. (a) Existing pipelines admit a fixed number of evidence items ① by time-agnostic relevance ② and discard that relevance before inference ③, so stale evidence can dominate. (b) CaRe-KT admits a query-adaptive evidence set ①, refines relevance by temporal validity ②, and carries it into inference ③, where it informs but does not dictate influence. Values are illustrative.
Symbol
Description
S,Q,C
Student, question, and concept sets
xt=(st,qt,τt),rt
Query interaction and response outcome
Cq,eqsem
Concept set and semantic representation of q
H<τts
Chronological history of student s before τt
M<τt
Global interaction memory before τt
Rt,Γt
Retrieved behavioral evidence and its relevance structure
TABLE I: Summary of Main Notation.
Fig. 2: Overall framework of CaRe-KT, which realizes relevance continuity from evidence acquisition to knowledge-state inference. ① The Evidence-Graph Cache refines persistent question-level compatibility through query-adaptive candidate screening and temporally valid interaction-anchor selection, yielding a query-specific evidence subgraph whose edge weights preserve acquisition relevance. ② Selected anchors guide temporally grounded trajectory retrieval, while a shared encoder and complementary regularizers establish a behaviorally structured representation space for the retrieved trajectories and target history. ③ The preserved relevance enters attention alongside learned query-evidence compatibility (orange arrow), informing the influence of each trajectory without determining it. ④ The fused representation yields r^t , and the observed response updates the cache and memory. Numerical values are illustrative.
Dataset
Students
Questions
Concepts
Interactions
DBE-KT22
1.3k
212
93
162.0k
ASSISTment09
4.2k
26.7k
110
401.8k
XES3G5M
18.1k
7.7k
865
5,549.6k
Slepemapy.cz
91.3k
4.5k
1,458
10,087.3k
Junyi
247.6k
0.7k
39
25,926.0k
TABLE II: Statistical Properties of the Evaluation Datasets.
Model
DBE-KT22
ASSISTment09
Slepemapy.cz
Junyi
XES3G5M
↑ AUC
↑ AP
↓ RMSE
↑ AUC
↑ AP
↓ RMSE
↑ AUC
↑ AP
↓ RMSE
↑ AUC
↑ AP
↓ RMSE
↑ AUC
↑ AP
↓ RMSE
DKT
76.55
89.72
0.4031
69.45
84.08
0.444
64.62
83.94
0.3861
69.01
90.70
0.3612
65.14
88.85
0.4007
AKT
76.53
89.75
0.4080
77.43
88.31
0.423
67.39
85.59
0.3802
73.10
91.72
0.3549
69.12
89.42
0.3871
IEKT
79.11
91.14
0.3894
76.97
88.74
0.477
69.87
87.00
0.3795
77.09
92.76
0.3514
72.49
91.09
0.3655
DIMKT
75.56
89.62
0.4174
75.53
88.69
0.452
68.09
84.03
0.3793
72.51
91.02
0.3556
72.75
90.70
0.3624
QIKT
79.59
91.14
0.3912
71.98
85.54
0.434
69.79
86.97
0.378
75.38
92.09
0.3575
66.65
88.88
0.4026
TABLE III: Overall performance on five datasets. The best result in each column is shaded dark blue and in bold, while the second best is shaded light blue and underlined. Arrows indicate the preferred direction ( ↑ higher is better, ↓ lower is better)
Model
DBE-KT22
ASSISTment09
Slepemapy.cz
Junyi
XES3G5M
↑ AUC
↑ AP
↓ RMSE
↑ AUC
↑ AP
↓ RMSE
↑ AUC
↑ AP
↓ RMSE
↑ AUC
↑ AP
↓ RMSE
↑ AUC
↑ AP
↓ RMSE
DKT
75.63
85.71
0.4347
58.67
69.99
0.478
59.99
82.98
0.422
57.33
81.93
0.483
69.83
89.15
0.3894
AKT
74.77
87.39
0.4474
62.99
73.23
0.479
59.34
83.09
0.440
57.39
82.51
0.432
70.52
89.08
0.3954
IEKT
73.98
87.52
0.4447
63.24
73.48
0.469
62.22
83.94
0.418
57.47
82.31
0.421
71.74
89.88
0.3823
DIMKT
59.05
81.69
0.4913
64.84
74.30
0.460
54.78
79.78
0.418
53.08
80.28
0.409
70.36
89.02
0.3909
QIKT
70.76
82.71
0.4359
66.24
73.50
0.474
62.58
84.56
0.418
58.56
83.03
0.406
71.14
89.26
0.3886
TABLE IV: Question-level cold-start performance on five datasets. The best result in each column is shaded dark blue and in bold, while the second best is shaded light blue and underlined. Arrows indicate the preferred direction ( ↑ higher is better, ↓ lower is better)
Fig. 3: Performance under restricted learner history. AUC (top) and RMSE (bottom) of CaRe-KT and five baselines on DBE-KT22 (a, d), XES3G5M (b, e), and Slepemapy.cz (c, f) when each target learner’s history is limited to its most recent L interactions, with other learners’ data unchanged.
DBE-KT22
XES3G5M
Slepemapy.cz
Variant
↑ AUC
↓ RMSE
↑ AUC
↓ RMSE
↑ AUC
↓ RMSE
CaRe-KT (full)
79.98
0.3901
78.50
0.3565
80.55
0.3714
RQ3: relevance continuity
w/o Acq. Relevance
79.68
0.3912
76.55
0.3633
79.50
0.3748
w/o Compatibility
79.80
0.3908
77.75
0.3591
79.85
0.3732
Learned- γ
79.92
0.3902
78.40
0.3569
80.20
0.3722
TABLE V: Ablation results for CaRe-KT variants on DBE-KT22, XES3G5M, and Slepemapy.cz, evaluated by AUC and RMSE.
Fig. 4: Sensitivity of CaRe-KT to (a, d) η , (b, e) λ , and (c, f) the regularization weight, as the change in AUC ↑ and RMSE ↓ relative to the full model of Table V , marked by the dashed lines. Mean ± one standard deviation over three seeds on DBE-KT22 and one seed on XES3G5M. The regularization axis is the effective ratio of Lreg to Lpred .
Knowledge Tracing (KT) infers a student's knowledge state from past interactions to predict future performance. Conventional Deep Learning (DL)-based KT models are typically tied to platform-specific identifiers and latent representations, making them hard to transfer and interpret. Large Language Model (LLM)-based methods can be either ungrounded under prompting or overly domain-dependent under fine-tuning. In addition, most existing KT methods are developed and evaluated under a same-distribution assumption. In real deployments, educational data often arise from heterogeneous platforms with substantial distribution shift, which often degrades generalization. To this end, we propose RAG-KT, a retrieval-augmented paradigm that frames cross-platform KT as reliable context constrained inference with LLMs. It builds a unified multi-source structured context with cross-source alignment via Question Group abstractions and retrieves complementary rich and reliable context for each prediction, enabling grounded prediction and interpretable diagnosis. Experiments on three public KT benchmarks demonstrate consistent gains in accuracy and robustness, including strong performance under cross-platform conditions.
Knowledge tracing (KT) aims to predict students' future performance by modeling their evolving knowledge states from historical interactions. Existing KT methods usually treat the raw interaction sequence as a unified behavioral process, overlooking the phase-specific nature of learning behaviors. Our preliminary observations show that students are more likely to correctly answer previously failed knowledge concepts after sufficient practice, suggesting a transition from ability-building to proficiency-oriented learning. Motivated by this, we propose Phase-Aware Knowledge Tracing (PAKT), a KT framework that decomposes student interactions into ability and proficiency phases based on the tailored decomposition mechanism. To effectively exploit the decomposed sequences, we design a multi-branch Transformer with a type-aware readout module to jointly capture phase-specific and holistic knowledge states. We further provide a causal analysis to reveal the confounding bias caused by entangling complex learning behaviors in phase-agnostic KT models. Extensive experiments on six public benchmarks demonstrate that our method consistently outperforms representative baselines, with a maximum AUC gain of 1.33% and an average gain of 0.82%.
Duantengchuan Li, Yingqian Bi, Jinsong Chen +2
School of the Information Management, Wuhan University, Wuhan 430072, China · School of Computer Science and Artificial Intelligence, Hubei University of Technology, Wuhan 430068, China · Faculty of Artificial Intelligence in Education, Central China Normal University, Wuhan 430074, China
Knowledge Tracing (KT) models students' knowledge states based on learning interactions to predict performance. While deep learning-based KT models have boosted predictive accuracy, most models rely on deterministic vector embeddings and opaque latent state transitions, limiting interpretability regarding how specific past behaviors influence predictions. To address this limitation, we propose Probabilistic Logical Knowledge Tracing (PLKT), an interpretable KT framework that formulates prediction as a goal-conditioned evidence reasoning process over historical learning behaviors. Instead of representing knowledge states as deterministic vector embeddings, PLKT employs robust Beta-distributed probabilistic embeddings to represent student knowledge states. This probabilistic foundation allows us to model the uncertainty of historical behaviors and perform explicit logical operations (e.g., conjunction), constructing transparent reasoning paths that reveal how specific past interactions contribute to the prediction. Extensive experiments show that PLKT outperforms state-of-the-art KT methods while achieving superior interpretability. Our code is available at https://anonymous.4open.science/r/PLKT-D3CE/.
Siyu Wu, Cong Xu, Wei Zhang
1Shanghai Institute of AI Education, East China Normal University, Shanghai, 200241, China · 2Shool of Computer Sicence and Technology, East China Normal University, Shanghai, 200241, China