Modeling the latent, multi-step reasoning processes of learners is a core challenge for adaptive tutoring systems, which often rely on simple right/wrong answers rather than understanding how students think. Instantiated here for grammatical reasoning, this study introduces a diagnostic framework that makes students' step-by-step problem-solving visible by analyzing their action sequences in an interactive learning game. The framework computes five interpretable strategy dimensions (e.g., systematic error correction, random guessing), estimates coordination demand from task structure and behavioral patterns, and classifies reasoning into four distinct states. Analyzing 36,234 interactions from 234 students working on 42 morphosyntactic exercises, we found that reasoning strategies vary primarily by context rather than stable traits (intraclass correlation = 0.14), and students show high persistence in unproductive states; specifically, 74.6% of failures driven by high coordination demand (rather than simple knowledge gaps) are followed by another failure without intervention. Our early warning model predicts validation success with strong accuracy (AUC = 0.826 using strategy patterns alone; 0.902 with additional sequential context) while avoiding circular reliance on current error counts. These findings demonstrate that making latent reasoning processes visible enables earlier, more targeted scaffolding that responds to how students are thinking rather than simply whether they succeed. While demonstrated in language learning, the framework provides a generalizable, interpretable architecture for diagnosing complex, structured reasoning in any domain.
Figures & tables
Figure 1: High-level architecture of the diagnostic framework. Raw interaction sequences (slider movements and validations) are processed through four integrated components to produce interpretable reasoning states and early warning predictions that inform adaptive scaffolding decisions.
Number of sliders
Number of exercises
2
3
3
7
4
2
5
1
6
2
7
4
Table 1: Distribution of exercises per number of sliders
Number of solutions
Number of exercises
1
1
2
6
3
4
4
7
5
2
6
8
Table 2: Distribution of exercises per number of solutions
Component
Description
Count
Total events
All logged interactions
36,234
Unique students
Distinct learner identifiers
234
Total sessions
Student-exercise instances
2,144
Validation events
validate actions
10,046
Tirette actions
tirette (drag) actions
26,188
Exercises
Distinct task types
42
Table 3: Basic dataset composition
Feature Category
Description
Coverage
Strategy Dimensions
Sverb(val)
Verb-First targeting
6,257 (62.3%)
Schain(val)
Chain-Aware progression
8,730 (86.9%)
Srand(val)
Randomness
8,730 (86.9%)
Sfocus(val)
Error-Focus
8,730 (86.9%)
Sstrat(val)
Strategic Commitment
8,712 (86.7%)
Table 4: Feature coverage and availability (validation-level)
Table 6: Average strategy evolution slope per dimension (per-student linear regression)
Figure 2: Visual guide: Vertical bars represent the four reasoning states; flowing bands represent the proportion of students transitioning from validation t to t+1 . The most prominent visual features are the large, curved self-loops (bands exiting and re-entering the same bar). Key findings: The massive red self-loop on High-Demand Failure (74.6%) and the large orange self-loop on Low-Demand Failure (62.3%) visually dominate, confirming that students rarely escape unproductive states without intervention. Conversely, the blue Fragile Success state shows a strong outward flow toward the green Sound Reasoning state (52.4%).
Figure 3: Top predictors of validation success. Standardized logistic regression coefficients shown; bar length reflects magnitude, sign indicates direction of effect. Error-Focus ( Sfocus(val) ) is the strongest strategy predictor in both models; Hamming trend emerges as the overall strongest predictor in the full sequential model.
Figure 4: Model calibration curves. Predicted probabilities (x-axis) vs. actual validation success rates (y-axis) across decile bins. Dashed line indicates perfect calibration. Strong alignment in the 0.3–0.7 range for both models confirms reliable risk estimates for scaffolding triggers.
Interaction
β
p (raw)
Effect
Chain × Rand
−1.475
<0.001
Antagonistic
Chain × Strat
+1.363
<0.001
Synergistic
Rand × Strat
+0.930
<0.001
Synergistic
Focus × Strat
+0.291
<0.001
Mild synergy
Rand × Focus
+0.561
<0.001
Synergistic
Verb × Chain
+0.607
<0.001
Synergistic
Table 7: Pairwise strategy interactions in early warning model. All reported effects survive Bonferroni correction for 10 comparisons at α=0.05 .
Check
Strategy-Only
Full Sequential
CV AUC
0.826 [0.818, 0.839]
0.902 [0.894, 0.909]
Held-out AUC
0.837 [0.815, 0.858]
0.912 [0.897, 0.926]
Student-level GroupKFold
0.823 ± 0.013
0.870 ± 0.013
Cross-exercise (25% holdout)
0.837
0.873
Temporal stability (Early/Late)
0.849 / 0.801
0.880 / 0.857
Noise resilience ( σ =0.15)
0.823
0.899
Table 8: Robustness summary: strategy-only vs. full sequential
Finding
Implication for Scaffolding
Strategy use is state-dominant (mean ICC = 0.14); global profiling fails
Preserve moderate exploration; intervene only when configurations become persistently unproductive
Table 9: Core empirical contributions and their implications for adaptive scaffolding
Figure 5: Abstract diagnostic pipeline and cross-domain instantiations. Top row: the generalizable architecture from raw actions to interpretable strategy dimensions, CDI estimation, and quadrant-based triggers. Lower panels: instantiations for (a) grammatical reasoning (this paper), (b) algebra equation solving, and (c) programming debugging. In all domains, the framework preserves full interpretability by grounding every diagnostic signal in theoretically defined, auditable reasoning configurations.
Supporting students in developing diagnostic reasoning is a key challenge across educational domains. Novices often face cognitive biases such as premature closure and over-reliance on heuristics, and they struggle to transfer diagnostic strategies to new cases. Scenario-based learning (SBL) enhanced by Learning Analytics (LA) and large language models (LLM) offers a promising approach by combining realistic case experiences with personalized scaffolding. Yet, how different scaffolding approaches shape reasoning processes remains insufficiently explored. This study introduces PharmaSim Switch, an SBL environment for pharmacy technician training, extended with an LA- and LLM-powered pharmacist agent that implements pedagogical conversations rooted in two theory-driven scaffolding approaches: structuring and problematizing, as well as a student learning trajectory. In a between-groups experiment, 63 vocational students completed a learning scenario, a near-transfer scenario, and a far-transfer scenario under one of the two scaffolding conditions. Results indicate that both scaffolding approaches were effective in supporting the use of diagnostic strategies. Performance outcomes were primarily influenced by scenario complexity rather than students' prior knowledge or the scaffolding approach used. The structuring approach was associated with more accurate Active and Interactive participation, whereas problematizing elicited more Constructive engagement. These findings underscore the value of combining scaffolding approaches when designing LA- and LLM-based systems to effectively foster diagnostic reasoning.
While test-time scaling improves the problem-solving ability of large reasoning models (LRMs) through additional inference-time computation, it can also exacerbate overthinking and underthinking, which we formulate as reasoning state--action mismatch. Resolving this mismatch requires reliable reasoning state diagnosis, yet single-signal monitors provide ambiguous evidence, while steering-based controllers often rely on outcome-labeled supervision or model-specific calibration. We introduce the Uncertainty--Progress Alignment Hypothesis, which posits that the relative transition timing of proxy answer uncertainty and latent reasoning progress distinguishes healthy, stagnant, and ready states that warrant different subsequent actions. Building on this insight, we propose UPAIR, a training-free framework that couples lightweight uncertainty monitoring with event-triggered joint diagnosis and maps the resulting state to native continuation, selective strategy switching, or verification-guided stopping. Across three LRMs and five cross-domain benchmarks, the stagnation diagnosis detects 64.3% of natural errors while flagging only 5.4% of correct samples, revealing a dynamic reasoning regularity shared across models and tasks. End to end, UPAIR improves accuracy by up to 16.67 percentage points and reduces generated tokens by up to 29.64%, demonstrating the effectiveness of its integrated diagnosis and intervention, while online diagnosis costs less than 1% of natural-generation time.
Cheng Yan, Zhijun Fan, Guangyang Ye +4
University of Science and Technology of China · AWS Agentic AI
This study investigates grammatical reasoning in primary school learners through a sequence-based learning analytics approach, leveraging fine-grained action sequences from an interactive game targeting morphosyntactic agreement in French. Unlike traditional assessments that rely on final answers, we treat each slider movement as a hypothesis-testing action, capturing real-time cognitive strategies during sentence construction. Analyzing 597 gameplay sessions (9,783 actions) from 100 students aged 8-11 in authentic classroom settings, we introduce Hamming distance to quantify proximity to valid grammatical solutions and examine convergence patterns across exercises with varying levels of difficulty. Results reveal that determiners and verbs are key sites of difficulty, with action sequences deviating from left-to-right usual treatment. This suggests learners often fix the verb first and adjust preceding elements. Exercises with fewer solutions exhibit slower and more erratic convergence, while changes in the closest valid solution indicate dynamic hypothesis revision. Our findings demonstrate how sequence-based analytics can uncover hidden dimensions of linguistic reasoning, offering a foundation for real-time scaffolding and teacher-facing tools in linguistically diverse classrooms.
Thierry Geoffre, Trystan Geoffre
University of Luxembourg FHSE - Department of Education and Social Work Esch-sur-Alzette, Luxembourg · Université de Fribourg Institut de Plurilinguisme Fribourg, Switzerland