Knowledge Tracing (KT) aims to predict learners' future performance from past interactions. While recent KT approaches have improved via learning item representations aligned with Knowledge Components, they overlook the procedural dynamics of problem solving. We propose Behavior-Aware Item Modeling (BAIM), a framework that enriches item representations by integrating dynamic procedural solution information. BAIM leverages a reasoning language model to decompose each item's solution into four problem-solving stages (i.e., understand, plan, carry out, and look back), pedagogically grounded in Polya's framework. Specifically, it derives stage-level representations from per-stage embedding trajectories, capturing latent signals beyond surface features. To reflect learner heterogeneity, BAIM adaptively routes these stage-wise representations, introducing a context-conditioned mechanism within a KT backbone, allowing different procedural stages to be emphasized for different learners. Experiments on XES3G5M and NIPS34 show that BAIM consistently outperforms strong pretraining-based baselines, achieving particularly large gains under repeated learner interactions.
Figures & tables
Figure 1: Comparison between conventional item modeling and BAIM. (a) Prior KT models rely on static item embeddings derived from item–KC structures. (b) BAIM instead generates procedural, context-aware item representations via Polya-based reasoning simulation.
Figure 2: Overview of the proposed BAIM framework. Left: A solver RLM simulates Polya-based procedural problem solving for each item and extracts stage-wise latent representations. Center: Stage-aware procedural embeddings yield procedural solution vectors, and a learner context encoder encodes the learner’s interaction history; both serve as inputs to the gating network. Right: A context-conditioned stage routing module adaptively selects the most relevant problem-solving stage to produce a learner-conditioned item representation for KT.
Figure 3: Example of a staircase-shaped geometry problem from the XES3G5M dataset and its stage-wise solution description.
Attribute
Datasets
XES3G5M
NIPS34
# Students
18.07K
4.92K
# Questions
7.65K
948
# KCs
865
57
# Interactions
5.55M
1.38M
Question Meta
Text + Image (CN)
Image-only (EN)
Table 1: Dataset statistics and metadata availability. Question Meta indicates the original format of question content, and Analysis Meta denotes whether analytical explanations are provided.
Table 2: Performance comparison (mean ± std) of KT backbones with different item representation methods. Values in parentheses denote absolute improvements over the Default setting; The best-performing method in terms of mean AUC is highlighted in bold .
Figure 4: Visualization of BAIM’s stage-level dynamic routing behavior and its impact on prediction outcomes under repeated interactions. Rows 0–3 : routing weights over problem-solving stages; Rows 4–6 : prediction outcomes of BAIM and baseline methods; Row 7 : ground-truth (GT) student performances.
Figure 5: AUC comparison across repeated student interactions. Adaptive routing enables procedural focus shifts in BAIM.
Figure 6: AUC (%, mean) of five KT backbones trained with varying numbers of students on XES3G5M. The red horizontal line indicates the performance of the Default representation trained on the full dataset (100%).
Figure 7: Performance comparison of aggregation strategies across multiple KT backbones on XES3G5M, highlighting the consistent advantage of BAIM’s routing-based aggregation over fixed schemes.
Figure 8: Impact of different representation extraction strategies. Leveraging the embedding trajectory of the solver RLM yields more effective item representations.
Appendix figures & tables6 assets
Supplementary material from the paper’s appendix.
Appendix
Notation
Shape
Dinput
768
Dkt
256
Dhistory
64
fproj(⋅)
Linear (4Dinput→Dkt) + ReLU + Dropout
Win
∈RDhistory×(2Dkt)
Wgate
∈R4×(Dkt+Dhistory)
Appendix
Table 3: Notations and tensor shapes used in the BAIM architecture.
Table 4: Performance comparison (mean ± std) of BAIM-equipped KT backbones using different solver RLMs. The best-performing solver in terms of mean AUC is highlighted in bold .
Stage
Mean
Std Dev
Min
Max
Qwen3-VL-32B-Thinking
Thinking Process
1,191.78
1,107.05
206
11,704
Stage 1: Understand
50.05
15.29
16
149
Stage 2: Plan
43.39
14.23
9
140
Stage 3: Carry Out
76.80
40.49
7
375
Stage 4: Look Back
36.52
25.20
9
1,773
Appendix
Table 5: Token usage statistics for solver RLM generation across model variants on the XES3G5M.
λ
AUC
Stage 0
Stage 1
Stage 2
Stage 3
0
83.18±0.06
0.48
0.44
0.08
0.01
0.01
83.21±0.10
0.24
0.29
0.23
0.24
0.1
83.16±0.19
0.28
0.23
0.31
0.18
Appendix
Table 6: Effect of the load-balancing coefficient λ on XES3G5M with sparseKT. We report test AUC and aggregated Top-1 routing proportions over the four procedural stages.
Figure 9: Prompt used to generate structured metadata from question images using Gemini-2.5-Pro.
Figure 10: System prompt used to elicit Polya-style four-stage reasoning trajectories from the solver RLM.
Knowledge tracing (KT) aims to predict students' future performance by modeling their evolving knowledge states from historical interactions. Existing KT methods usually treat the raw interaction sequence as a unified behavioral process, overlooking the phase-specific nature of learning behaviors. Our preliminary observations show that students are more likely to correctly answer previously failed knowledge concepts after sufficient practice, suggesting a transition from ability-building to proficiency-oriented learning. Motivated by this, we propose Phase-Aware Knowledge Tracing (PAKT), a KT framework that decomposes student interactions into ability and proficiency phases based on the tailored decomposition mechanism. To effectively exploit the decomposed sequences, we design a multi-branch Transformer with a type-aware readout module to jointly capture phase-specific and holistic knowledge states. We further provide a causal analysis to reveal the confounding bias caused by entangling complex learning behaviors in phase-agnostic KT models. Extensive experiments on six public benchmarks demonstrate that our method consistently outperforms representative baselines, with a maximum AUC gain of 1.33% and an average gain of 0.82%.
Duantengchuan Li, Yingqian Bi, Jinsong Chen +2
School of the Information Management, Wuhan University, Wuhan 430072, China · School of Computer Science and Artificial Intelligence, Hubei University of Technology, Wuhan 430068, China · Faculty of Artificial Intelligence in Education, Central China Normal University, Wuhan 430074, China
Knowledge Tracing (KT) models students' knowledge states based on learning interactions to predict performance. While deep learning-based KT models have boosted predictive accuracy, most models rely on deterministic vector embeddings and opaque latent state transitions, limiting interpretability regarding how specific past behaviors influence predictions. To address this limitation, we propose Probabilistic Logical Knowledge Tracing (PLKT), an interpretable KT framework that formulates prediction as a goal-conditioned evidence reasoning process over historical learning behaviors. Instead of representing knowledge states as deterministic vector embeddings, PLKT employs robust Beta-distributed probabilistic embeddings to represent student knowledge states. This probabilistic foundation allows us to model the uncertainty of historical behaviors and perform explicit logical operations (e.g., conjunction), constructing transparent reasoning paths that reveal how specific past interactions contribute to the prediction. Extensive experiments show that PLKT outperforms state-of-the-art KT methods while achieving superior interpretability. Our code is available at https://anonymous.4open.science/r/PLKT-D3CE/.
Siyu Wu, Cong Xu, Wei Zhang
1Shanghai Institute of AI Education, East China Normal University, Shanghai, 200241, China · 2Shool of Computer Sicence and Technology, East China Normal University, Shanghai, 200241, China
Knowledge Tracing (KT) aims to model student's knowledge state and predict future performance to enable personalized learning in Intelligent Tutoring Systems. However, traditional KT methods face fundamental limitations in explainability, as they rely solely on the response correctness, neglecting the rich information embedded in students' problem-solving processes. To address this gap, we propose Knowledge Tracing Leveraging Problem-Solving Process (KT-PSP), which incorporates students' problem-solving processes to capture the multidimensional aspects of mathematical proficiency. We also introduce KT-PSP-25, a new dataset specifically designed for KT-PSP. Building on this, we present StatusKT, a KT framework that employs a teacher-student-teacher three-stage LLM pipeline to extract students' Mathematical Proficiency (MP) as intermediate representation. In this pipeline, the teacher LLM first extracts problem-specific proficiency indicators, then a student LLM generates responses based on the student's solution process, and a teacher LLM evaluates these responses to determine mastery of each indicator. The experimental results on KT-PSP-25 demonstrate that StatusKT improves the prediction performance of existing KT methods. Moreover, StatusKT provides interpretable explanations for its predictions by explicitly modeling students' mathematical proficiency. Code is available here.
Jungyang Park, Suho Kang, Jaewoo Park +4
1Yonsei University · 2Mathpresso · 3Seoul National University