Behavior-Aware Item Modeling via Dynamic Procedural Solution Representations for Knowledge Tracing

Jun Seo; Sangwon Ryu; Heejin Do; Hyounghun Kim; Gary Geunbae Lee

cs.CLApr 9, 2026

Behavior-Aware Item Modeling via Dynamic Procedural Solution Representations for Knowledge Tracing

Authors: Jun Seo, Sangwon Ryu, Heejin Do, Hyounghun Kim, Gary Geunbae Lee

Organizations: GSAI, POSTECH · ETH Zurich, ETH AI Center · CSE, POSTECH

Abstract

Knowledge Tracing (KT) aims to predict learners' future performance from past interactions. While recent KT approaches have improved via learning item representations aligned with Knowledge Components, they overlook the procedural dynamics of problem solving. We propose Behavior-Aware Item Modeling (BAIM), a framework that enriches item representations by integrating dynamic procedural solution information. BAIM leverages a reasoning language model to decompose each item's solution into four problem-solving stages (i.e., understand, plan, carry out, and look back), pedagogically grounded in Polya's framework. Specifically, it derives stage-level representations from per-stage embedding trajectories, capturing latent signals beyond surface features. To reflect learner heterogeneity, BAIM adaptively routes these stage-wise representations, introducing a context-conditioned mechanism within a KT backbone, allowing different procedural stages to be emphasized for different learners. Experiments on XES3G5M and NIPS34 show that BAIM consistently outperforms strong pretraining-based baselines, achieving particularly large gains under repeated learner interactions.

Figures & tables

Figure 1: Comparison between conventional item modeling and BAIM. (a) Prior KT models rely on static item embeddings derived from item–KC structures. (b) BAIM instead generates procedural, context-aware item representations via Polya-based reasoning simulation.

Figure 2: Overview of the proposed BAIM framework. Left: A solver RLM simulates Polya-based procedural problem solving for each item and extracts stage-wise latent representations. Center: Stage-aware procedural embeddings yield procedural solution vectors, and a learner context encoder encodes the learner’s interaction history; both serve as inputs to the gating network. Right: A context-conditioned stage routing module adaptively selects the most relevant problem-solving stage to produce a learner-conditioned item representation for KT.

Figure 3: Example of a staircase-shaped geometry problem from the XES3G5M dataset and its stage-wise solution description.

Attribute	Datasets
Attribute	XES3G5M	NIPS34
# Students	18.07K	4.92K
# Questions	7.65K	948
# KCs	865	57
# Interactions	5.55M	1.38M
Question Meta	Text + Image (CN)	Image-only (EN)

Table 1: Dataset statistics and metadata availability. Question Meta indicates the original format of question content, and Analysis Meta denotes whether analytical explanations are provided.

Panel of Table 2 — Table 2: Performance comparison (mean $\pm$ std) of KT backbones with different item representation methods. Values in parentheses denote absolute improvements over the Default setting; The best-performing method in terms of mean AUC is highlighted in bold .

Figure 4: Visualization of BAIM’s stage-level dynamic routing behavior and its impact on prediction outcomes under repeated interactions. Rows 0–3 : routing weights over problem-solving stages; Rows 4–6 : prediction outcomes of BAIM and baseline methods; Row 7 : ground-truth (GT) student performances.

Figure 5: AUC comparison across repeated student interactions. Adaptive routing enables procedural focus shifts in BAIM.

Figure 6: AUC (%, mean) of five KT backbones trained with varying numbers of students on XES3G5M. The red horizontal line indicates the performance of the Default representation trained on the full dataset (100%).

Figure 7: Performance comparison of aggregation strategies across multiple KT backbones on XES3G5M, highlighting the consistent advantage of BAIM’s routing-based aggregation over fixed schemes.

Figure 8: Impact of different representation extraction strategies. Leveraging the embedding trajectory of the solver RLM yields more effective item representations.

Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

Notation	Shape
$D_{\text{input}}$	$768$
$D_{\text{kt}}$	$256$
$D_{\text{history}}$	$64$
$f_{\text{proj}}(\cdot)$	Linear $(4D_{\text{input}}\rightarrow D_{\text{kt}})$ + ReLU + Dropout
$\mathbf{W}_{\text{in}}$	$\in\mathbb{R}^{D_{\text{history}}\times(2D_{\text{kt}})}$
$\mathbf{W}_{\text{gate}}$	$\in\mathbb{R}^{4\times(D_{\text{kt}}+D_{\text{history}})}$

Appendix

Table 3: Notations and tensor shapes used in the BAIM architecture.

Panel of Table 4 — Table 4: Performance comparison (mean $\pm$ std) of BAIM-equipped KT backbones using different solver RLMs. The best-performing solver in terms of mean AUC is highlighted in bold .

Stage	Mean	Std Dev	Min	Max
Qwen3-VL-32B-Thinking
Thinking Process	1,191.78	1,107.05	206	11,704
Stage 1: Understand	50.05	15.29	16	149
Stage 2: Plan	43.39	14.23	9	140
Stage 3: Carry Out	76.80	40.49	7	375
Stage 4: Look Back	36.52	25.20	9	1,773

Appendix

Table 5: Token usage statistics for solver RLM generation across model variants on the XES3G5M.

$\lambda$	AUC	Stage 0	Stage 1	Stage 2	Stage 3
0	$83.18\pm 0.06$	0.48	0.44	0.08	0.01
0.01	$\mathbf{83.21\pm 0.10}$	0.24	0.29	0.23	0.24
0.1	$83.16\pm 0.19$	0.28	0.23	0.31	0.18

Appendix

Table 6: Effect of the load-balancing coefficient

\lambda

on XES3G5M with sparseKT. We report test AUC and aggregated Top-1 routing proportions over the four procedural stages.

Panel of Figure 9 — Figure 9: Prompt used to generate structured metadata from question images using Gemini-2.5-Pro.

Panel of Figure 10 — Figure 10: System prompt used to elicit Polya-style four-stage reasoning trajectories from the solver RLM.

Knowledge Tracing

Knowledge tracing models how learners’ mastery of skills changes over time from their interactions with educational tasks.

See all 31 papers

Disentangling Knowledge States with Ability and Proficiency Modeling for Knowledge Tracing
Jul 14, 2026Duantengchuan Li, Yingqian Bi, Jinsong Chen +2Knowledge Tracing
School of the Information Management, Wuhan University, Wuhan 430072, China · School of Computer Science and Artificial Intelligence, Hubei University of Technology, Wuhan 430068, China · Faculty of Artificial Intelligence in Education, Central China Normal University, Wuhan 430074, China
Explainable Knowledge Tracing via Probabilistic Embeddings and Pattern-based Reasoning
May 10, 2026Siyu Wu, Cong Xu, Wei ZhangRepresentation Learning Interpretable ML
1Shanghai Institute of AI Education, East China Normal University, Shanghai, 200241, China · 2Shool of Computer Sicence and Technology, East China Normal University, Shanghai, 200241, China
Tracing Mathematical Proficiency Through Problem-Solving Processes
Nov 29, 2025Jungyang Park, Suho Kang, Jaewoo Park +4Intelligent Tutoring Systems Knowledge Tracing
1Yonsei University · 2Mathpresso · 3Seoul National University

Jul 14, 2026J/K move · Enter open · S save

Disentangling Knowledge States with Ability and Proficiency Modeling for Knowledge Tracing

Duantengchuan Li, Yingqian Bi, Jinsong Chen, Rui Zhang+1

School of the Information Management, Wuhan University, Wuhan 430072, China · School of Computer Science and Artificial Intelligence, Hubei University of Technology, Wuhan 430068, China · Faculty of Artificial Intelligence in Education, Central China Normal University, Wuhan 430074, China

Knowledge Tracing

Open paper AskPDF arXiv

Knowledge tracing (KT) aims to predict students' future performance by modeling their evolving knowledge states from historical interactions. Existing KT methods usually treat the raw interaction sequence as a unified behavioral process, overlooking the phase-specific nature of learning behaviors. Our preliminary observations show that students are more likely to correctly answer previously failed knowledge concepts after sufficient practice, suggesting a transition from ability-building to proficiency-oriented learning. Motivated by this, we propose Phase-Aware Knowledge Tracing (PAKT), a KT framework that decomposes student interactions into ability and proficiency phases based on the tailored decomposition mechanism. To effectively exploit the decomposed sequences, we design a multi-branch Transformer with a type-aware readout module to jointly capture phase-specific and holistic knowledge states. We further provide a causal analysis to reveal the confounding bias caused by entangling complex learning behaviors in phase-agnostic KT models. Extensive experiments on six public benchmarks demonstrate that our method consistently outperforms representative baselines, with a maximum AUC gain of 1.33% and an average gain of 0.82%.

Abstract

Knowledge Tracing

Explore similar work

Disentangling Knowledge States with Ability and Proficiency Modeling for Knowledge Tracing

Explainable Knowledge Tracing via Probabilistic Embeddings and Pattern-based Reasoning

Tracing Mathematical Proficiency Through Problem-Solving Processes