Argus: Academic Integrity in the Era of Generative AI

The rapid emergence of Large Language Models (LLMs) presents both opportunities and challenges for programming education. While students increasingly use generative AI tools, direct access often hinders the learning process by providing complete solutions rather than pedagogical hints. Concurrently, educators face significant workload and scalability challenges when providing timely, personalized feedback. This study investigates the capabilities of LLMs to safely and effectively assist educators in answering student questions within a CS1 programming course. To achieve this, we established a rigorous, reproducible evaluation process by curating a benchmark dataset of 170 authentic student questions from a learning management system, paired with ground-truth responses authored by subject matter experts. Because traditional text-matching metrics are insufficient for evaluating open-ended educational responses, we developed and validated a custom LLM-as-a-Judge metric optimized for assessing pedagogical accuracy. Our findings demonstrate that models, such as Gemini 3 flash, can surpass the quality baseline of typical educator responses, achieving high alignment with expert pedagogical standards. To mitigate persistent risks like hallucination and ensure alignment with course-specific context, we advocate for a "teacher-in-the-loop" implementation. Finally, we abstract our methodology into a task-agnostic evaluation framework, advocating for a shift in the development of educational LLM tools from ad-hoc, post-deployment testing to a quantifiable, pre-deployment validation process.

Students increasingly use the assistance of large language models (LLMs) in their academic writing. While slight assistance (e.g., grammar and style correction, as well as feedback) is permitted under most institutional policies, it is usually forbidden to offload entire writing tasks to LLMs. Unfortunately, current approaches to LLM-generated text detection predominantly assume a binary distinction between human-written and LLM-generated text, ignoring the breadth of realistic human-AI collaboration practices and limiting the validity of detection systems for educational assessment. In this paper, we propose a contribution-aware evaluation framework for LLM-based detection systems in education. We introduce a scale of eight student contribution levels that model realistic writing scenarios ranging from fully human-written texts to LLM-assisted revisions to fully LLM-generated and adversarially humanized texts. Institutional policies regarding LLM use can then be translated to thresholds of acceptable LLM assistance on this scale. We further present Generative Essay Detection in Education (GEDE), a novel benchmark dataset comprising more than 900 human-written and over 12,500 generated essays across 886 tasks and all contribution levels. Using this benchmark, we conduct a systematic evaluation of four state-of-the-art zero-shot and supervised detection methods across policy boundaries, contribution levels, generative models, out-of-distribution data, and text length. We show that most detectors struggle to accurately classify texts at intermediate student contribution levels, in particular LLM-assisted revisions of human-written texts. Such errors pose a substantial risk of false accusations, indicating that current text detection systems are (still) unsuitable to reliably support the enforcement of institutional policies regarding LLM assistance in education.

The rapid adoption of generative AI has made final artifacts unreliable evidence of student learning, and AI detectors that examine only the finished product are inaccurate and ethically contentious. Process data offers an alternative, but prior work covers only English essay writing. We ask whether AI assistance carries a temporal signature, whether it generalizes from writing to programming, and whether it distinguishes ordinary collaboration from wholesale delegation. We analyze three public corpora: CoAuthor (1,447 keystroke-level co-writing sessions), RealHumanEval (editor telemetry from 243 programmer records), and a pre-LLM CS1 corpus (5.1 million keystrokes) as a human-only baseline, comparing minimal-AI work, collaborative AI use, and simulated wholesale delegation. Three findings emerge. First, the signature generalizes: AI contributions arrive in bursts far outside the author's own baseline in both mediums (paired d_z = 1.13 and 3.54). Second, engagement diverges by medium: 93% of AI-inserted characters survived to writers' final documents, while only 14% of accepted code suggestions survived intact. Third, classifiers using only observable temporal features separate simulated delegation from authentic work nearly perfectly (F1 $\geq$ 0.997; at most 0.5% of real work misclassified), while ordinary collaboration remains hard to distinguish from unassisted work. Temporal evidence flags wholesale delegation rather than assistance, positioning process visibility as a candidate evidentiary basis for academic integrity, pending validation in authentic coursework.