Grading in large undergraduate STEM courses often yields minimal feedback due to heavy instructional workloads. We present a large-scale empirical study of AI grading on real, handwritten single-variable calculus work from UC Irvine. Using OCR-conditioned large language models with structured, rubric-guided prompting, our system produces scores and formative feedback for thousands of free-response quiz submissions from nearly 800 students. In a setting with no single ground-truth label, we evaluate performance against official teaching-assistant grades, student surveys, and independent human review, finding strong alignment with TA scoring and a large majority of AI-generated feedback rated as correct or acceptable across quizzes. Beyond calculus, this setting highlights core challenges in OCR-conditioned mathematical reasoning and partial-credit assessment. We analyze key failure modes, propose practical rubric- and prompt-design principles, and introduce a multi-perspective evaluation protocol for reliable, real-course deployment. Building on the dataset and evaluation framework developed here, we outline a standardized benchmark for AI grading of handwritten mathematics to support reproducible comparison and future research.
Figures & tables
Figure 1: Overview of the proposed AI grading pipeline: The core module converts handwritten solutions to LaTeX and evaluates them against reference solutions and structured rubrics, producing detailed scores and feedback.
Figure 2: Bars correspond to the flexible rubric, fixed rubric and max-rule, respectively (left to right within each group): A-Math 2A, B-Math 2B
Figure 4: Quiz-level summary: mean gap and within-1 percentage.
Data Analytics Lab, Vrije Universiteit Brussel, 1050 Brussel, Belgium · School of Engineering and Applied Sciences, Harvard University, Cambridge, Massachusetts 02138, USA