AI-Generated Traces for Novice Programmers: Learning Effects and Learner Differences in a Multi-Institutional Study

Yuri Noviello; Naaz Sibia; Anastasiia Birillo; Thomas Overklift Vaupel Klein; Michael Liut; Gosia Migut

cs.CYJun 2, 2026

AI-Generated Traces for Novice Programmers: Learning Effects and Learner Differences in a Multi-Institutional Study

Authors: Yuri Noviello, Naaz Sibia, Anastasiia Birillo, Thomas Overklift Vaupel Klein, Michael Liut, Gosia Migut

Organizations: Delft University of Technology Delft, Netherlands · University of Toronto Toronto, Canada · JetBrains Research Belgrade, Serbia · University of Toronto Mississauga Mississauga, Canada

Abstract

Introductory programming (CS1) courses often struggle to support students' understanding of program execution. While visualizations can make execution processes explicit, their effectiveness depends on design and context, and empirical evidence for AI-generated visualizations remains limited. We propose Generated Animated Traces (GATs), AI-generated, analogy-based, narrated animations that coordinate source code, execution state, and conceptual analogies. We conduct a study at two institutions in CS1 courses (Python, N=961; Java N=151) comparing GATs to textual explanations. We measure immediate learning performance and experience, end-of-course engagement and exam performance. Results show that GATs can yield selective benefits for immediate learning, but benefits are context-dependent and short-term. We observe that GATs' influence on performance is moderated by learner engagement profiles. This finding underscores the importance of personalized approaches.

Explore similar work

AI-Assisted Help-Seeking Trajectories in Programming Education from an SRL-Informed Perspective
Jun 22, 2026Boxuan Ma, Huiyong Li, Gen Li +3Problem Solving Capability Exploratory Study
Beyond Perspectives: A Trio-Ethnography of Interpretation Evolution in LLM-Supported Programming Education
Jul 24, 2026Jennie Ren, Jordan H. McDowell, Kyrie Zhixuan ZhouArtificial Intelligence Literacy Adaptive Testing
Choose Your Own Adventure: Non-Linear AI-Assisted Programming with EvoGraph
Apr 20, 2026Vassilios Exarhakos, Jinghui Cheng, Jin L. C. GuoAi-Ran Graph Representations

Jun 22, 2026 · cs.AIJ/K move · Enter open · S save

AI-Assisted Help-Seeking Trajectories in Programming Education from an SRL-Informed Perspective

Boxuan Ma, Huiyong Li, Gen Li, Li Chen+2

Faculty of Arts and Science, Kyushu University, Fukuoka, Japan. · Research Institute for Information Technology, Kyushu University, Fukuoka, Japan. · Faculty of Information Science and Electrical Engineering, Kyushu University, Fukuoka, Japan. · Division of Math, Sciences, and Information Technology in Education, Osaka Kyoiku University, Osaka, Japan.

Generative AI tools provide novice programmers with instant, personalized support, but also raise concerns about whether AI use supports or bypasses students' regulation of problem-solving. Existing work has largely focused on correctness, usability, or overall usage frequency, with less attention to how student--AI help-seeking unfolds. This study addresses this gap by analyzing AI-assisted help-seeking trajectories in university-level programming. Using an SRL-informed analytical framework that links prompt-level help-seeking codes to conceptual, implementation, debugging, and reflective forms of support, we analyzed 1,290 task-specific student prompts linked to 17,190 code submissions from 71 students in introductory Python programming courses. Specifically, we examined how help-seeking interactions were structured across turns and attempts, and how trajectory patterns related to task scores and the number of code submissions. Results indicate that many students primarily used AI for reactive troubleshooting rather than for planned, self-regulated problem-solving. Although trajectory patterns were not associated with significant differences in task scores, they differed substantially in the number of code submissions required. These findings suggest that the educational significance of AI support lies not only in whether students use AI, but in how their help-seeking trajectories develop during programming problem-solving.

Problem Solving Capability Exploratory Study Trajectory Prediction

Open paper Ask about it PDF

Abstract

Explore similar work

AI-Assisted Help-Seeking Trajectories in Programming Education from an SRL-Informed Perspective

Beyond Perspectives: A Trio-Ethnography of Interpretation Evolution in LLM-Supported Programming Education

Choose Your Own Adventure: Non-Linear AI-Assisted Programming with EvoGraph