cs.SEFeb 22, 2026

Mitigating "Epistemic Debt" in Generative AI-Scaffolded Novice Programming using Metacognitive Scripts

Authors: Sreecharan Sankaranarayanan

Organizations: Extuitive Inc. (Flagship Pioneering) Cambridge, Massachusetts, United States of America

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

The democratization of Large Language Models has given rise to vibe coding, where novice programmers prioritize semantic intent over syntactic implementation. Without pedagogical guardrails, we argue this is fundamentally misaligned with cognitive skill acquisition. Drawing on Kirschner's distinction between cognitive offloading and outsourcing, unrestricted AI encourages novices to outsource the intrinsic cognitive load required for schema formation rather than merely offloading extraneous load. This accumulation of epistemic debt creates fragile experts: developers whose high functional utility masks critically low corrective competence. To quantify and mitigate this debt, we conducted a between-subjects experiment (N=78) using a custom Cursor IDE plugin backed by Claude 3.5 Sonnet. Participants were recruited via Prolific and UserInterviews.com to represent AI-native learners. We compared three conditions: manual (control), unrestricted AI (outsourcing), and scaffolded AI (offloading). The scaffolded condition employed a novel Explanation Gate -- a real-time LLM-as-a-Judge framework enforcing a teach-back protocol before generated code could be integrated. Results reveal a collapse of competence: both AI groups significantly outperformed the manual control on functional utility (p < .001) and did not differ from each other (p = .64), yet unrestricted AI users suffered a 77% failure rate on a subsequent 30-minute AI-blackout maintenance task, vs. only 39% in the scaffolded group. Qualitative analysis suggests successful vibe coders naturally self-scaffold, treating AI as a consultant rather than a contractor. We discuss implications for AI-generated software maintainability and propose that future learning systems must enforce metacognitive friction to prevent mass production of unmaintainable code. Replication package: https://github.com/sreecharansankaranarayanan/vibecheck

Figures & tables

Figure 1. System Architecture of the VibeCheck Plugin. The diagram illustrates how the Rule Enforcement Engine intercepts the “Apply” signal from the LLM Sidebar and conditionally routes it through the Explanation Gate Manager, enforcing a metacognitive friction loop via the external Judge API. System architecture flowchart showing input from Sidebar and Keystrokes being processed by Rule Enforcement Engine and potentially gated by the Explanation Gate Manager.
ScoreSOLO LevelStudent Explanation Example
1Pre-structural“I added a button so that when you click it the course is added to the list and it updates.”
2Unistructural“It uses the enrollCourse function to send the ID to the database and then changes the UI.”
3Relational“The await keyword ensures the app waits for the DB confirmation before updating the local courses state, preventing UI desync.”
4Ext. Abstract (partial)“The try/catch around the fetch prevents the UI from showing stale data if the server call fails, but I’m not sure how it handles a timeout.”
5Ext. Abstract“It implements optimistic updates by updating the state first but uses a try/catch to rollback the UI if the fetch request fails, ensuring eventual consistency.”
Table 1. SOLO Taxonomy Calibration for Automated Judge Evaluation
Figure 2. Example Judge API feedback for sub-threshold explanations. The modal remains locked until a SOLO Level 3 (Relational) response is submitted. Two screenshots of the Judge API feedback displayed inside the VibeCheck Explanation Gate modal. Left: a SOLO Level 1 (Pre-structural) response receives feedback pointing out that the explanation describes only the visible outcome without addressing the underlying mechanism. Right: a SOLO Level 2 (Unistructural) response receives feedback noting that the explanation identifies a single component but does not explain the causal relationship between the fetch call and the state update.
Figure 3. The VibeCheck Explanation Gate enforcement interface (Group C). Left: the blocking modal. Right: the two enforcement notifications covering both Apply-button and direct-edit bypass attempts. Three screenshots of the VibeCheck plugin interface. Left: a blocking modal overlay in the Cursor IDE prompting the learner to explain the causal logic of AI-generated code before the Apply button is enabled. Top right: a notification banner reading VibeCheck: Changes blocked — explain the code to apply it.'' Bottom right: a second notification reading VibeCheck: Edit reverted — explain the code in the…'' showing that a direct editor bypass was undone.
IDAssertion Description
A-01User can log in using mock credentials and reach the dashboard.
A-02Course catalog successfully fetches and renders 20+ items from JSON.
A-03Course list displays correct metadata (Instructor, Credits, Time).
A-04Clicking “Enroll” adds a course to the user’s sidebar state.
A-05Duplicate enrollment in the same course is blocked by state logic.
A-06The system identifies time-overlaps between two distinct courses.
Table 2. 12 Component Assertions of the Functional Utility Score for a Total of 100 Points
Original (Functional)Injected Regression (Bug)
const res = await fetch(’/enroll’);const res = fetch(’/enroll’);
if (res.ok) { updateUI(); }updateUI(); // Optimistic
else { rollback(); }// No rollback logic
Table 3. The Phase 2 “Logic Bomb” Regression
ConditionMean Utility ScoreSD
Manual (Group A)65.2%14.3
Unrestricted AI (Group B)92.4%8.1
Scaffolded AI (Group C)89.1%9.5
Table 4. Phase 1: Functional Utility Scores
ConditionTotal Time Mean ( SDSD )Active Thinking Mean ( SDSD )Progress (%)
Manual (A)88.4 (2.1)88.4 (2.1)42%
Unrestricted (B)48.2 (10.4)18.5 (4.2)100%
Scaffolded (C)64.6 (11.2)36.3 (5.8)100%
Table 5. Phase 1: Time on Task and Cognitive Allocation
IDConditionUtility (Ph 1)Repair Result (Ph 2)Fix TimeDominant Stance (Ph 1)
P-12Manual72%Success11m 45sManual Reasoning
P-23Unrestricted95%FailTimeoutContractor (Directive)
P-35Unrestricted98%Success21m 10sConsultant (Explanatory)
P-48Scaffolded90%Success13m 30sEnforced Consultant
P-55Scaffolded82%FailTimeoutTautological (Low SOLO)
Table 6. Representative Individual Results showing the divergence in Phase 2. Table showing Phase 1 utility scores against Phase 2 repair results and time to fix across different conditions and participant IDs.
Figure 4. Distribution of the number of attempts per Explanation Gate encounter ( n=172n=172 encounters, Group C). The modal value of two attempts indicates that a single round of Socratic feedback was sufficient for most learners to reach SOLO Level 3. Distribution of the number of attempts per Explanation Gate encounter across 172 gate encounters in Group C.
Figure 5. A SOLO Level 3 (Relational) explanation that satisfies the Judge API threshold (Score 5/5). The learner explains the causal chain connecting state, memoization, and re-rendering: the kind of encoding the Gate is designed to produce. Screenshot of the VibeCheck Explanation Gate at the moment a passing explanation is accepted. The feedback panel reads ``SCORE 5/5 — Great explanation! Code is being applied…'' The learner's explanation describes how courseMatchesQuery filters courses by name and instructor, how visibleCourses is a memoized filtered result, and how typing in the search input updates the filter state, triggers the memo, and re-renders only the matching rows.
More in this topic

Generative AI in Education

Generative AI in education uses language and multimodal models to support teaching, learning, feedback, assessment, and educational design.

See all 282 papers

Explore similar work

CardsList
Jun 15, 2026

Vibe Coding Ate My Homework: An evaluation of AI approaches to greenfield software engineering and programming

Thanks to rapid developments in generative AI, we are in the midst of a paradigm shift that may change how we interact with computers forever. We have observed a growth in the use of natural language prompts to build applications and coding infrastructures without underlying knowledge of the field, and this practice has been dubbed `vibe coding.' It arguably represents what the field of programming has been building towards since the beginning, with every higher level of abstraction that is conceived. Vibe coding promises to be the endpoint for the meta of high-level programming as far as method of input is concerned: eliminating a human's use of code syntax entirely in favour of programming in their mother tongue. This paper aims to evaluate the viability of vibe coding for greenfield software engineering tasks, as well as analyse the benchmarks that have been used to measure its software engineering prowess. To this end, we have developed an evaluation suite for analysing an LLM's proficiency in carrying out simple, isolated greenfield programming tasks in Python to provide scoped insight on the matter.

Callum Barbour

Software EngineeringCode Generation EvaluationCode Generation Benchmarks
AskPDF
Apr 29, 2026

Unpacking Vibe Coding: Help-Seeking Processes in Student-AI Interactions While Programming

Generative AI is reshaping higher education programming through vibe coding, where students collaborate with AI via natural language rather than writing code line-by-line. We conceptualize this practice as help-seeking, analyzing 19,418 interaction turns from 110 undergraduate students. Using inductive coding and Heterogeneous Transition Network Analysis, we examined interaction sequences to compare top- and low-performing students. Results reveal that top performers engaged in instrumental help-seeking -- inquiry and exploration -- eliciting tutor-like AI responses. In contrast, low performers relied on executive help-seeking, frequently delegating tasks and prompting the AI to assume an executor role focused on ready-made solutions. These findings indicate that currently generative AI mirrors student intent (whether productive or passive) rather than optimizing for learning. To evolve from tools to teammates, AI systems must move beyond passive compliance. We argue for pedagogically aligned design that detect unproductive delegation and adaptively steer educational interactions toward inquiry, ensuring student-AI partnerships augment rather than replace cognitive effort.

Daiana Rinja, Eduardo Araujo Oliveira, Sonsoles López-Pernas +3

CATALPA, FernUniversität in Hagen, 58097 Hagen, Germany · School of Computing and Information Systems, University of Melbourne, Parkville VIC 3010, Australia · School of Computing, University of Eastern Finland, 80110 Joensuu, Finland

AI in EducationProgramming EducationGenerative AI in EducationHuman-AI Interaction
AskPDF
Jul 28, 2026

TraceCoder: Explainable and Auditable Code Generation with Position-Key Snippet Versioning

Contemporary LLM-based coding agents produce code as black-box outputs: the rationale behind each line is hidden, the evolution of the code through benchmark-driven repair is ephemeral, and post-hoc auditing is impossible. We present a code generation concept that addresses these shortcomings through three complementary mechanisms: (i) a relational snippet-history schema that records, per repair event, the benchmark reference, round number, failure text, and LLM explanation, enabling full provenance queries; (ii) a browser-based visualisation tool that renders this history as heat-mapped, hover-annotated source code; and (iii) a competitive fractional position-key indexing scheme with tree-node delimiters that assigns stable, lexicographically-ordered identifiers to each code snippet, enabling fine-grained tracking without disrupting surrounding lines. We evaluate TraceCoder on 30 algorithmic programming tasks spanning string processing, mathematical computation, and data-structure manipulation, across two provider configurations. Of these, 10 exhaust the 6-iteration budget on tasks with subtle edge-case behaviour. Mean Chg% reaches 30%, three in ten code snippets carry a traceable repair-event row, compared to 21% when using Gemini 2.0 Flash as sole provider on a 20-task subset. Three detailed case studies demonstrate how the system explains which specific benchmark failures shaped each line of the final program. The proposed mechanism makes the internal "narrative" of automated code generation auditable and replayable, a property essential for trust and accountability in production deployments.

Rwaida Alssadi, Muntaser Syed, Balaji Kasula +6

Florida Institute of Technology

Data ProvenanceLLM AuditingCode Generation
AskPDF