Automated Grading

Follow this topic Ask about this topic

Automated grading studies how to assess student work consistently, from short answers and essays to mathematical reasoning, handwritten exams, and programming assignments. Approaches include fine-tuned language models and encoders, rubric-guided LLM judges, and vision-language systems that read handwritten work; many combine scores with explanations or route uncertain cases to human graders. Systems are commonly evaluated against expert marks using agreement and error measures, with benchmarks testing transfer across subjects, questions, and scoring rubrics. A central challenge is handling partial or ambiguous answers reliably while avoiding bias, prompt sensitivity, and overconfidence, so human oversight remains important, especially for high-stakes decisions.

Latest papers

89 papers6 papers in the last four weeks, down 14% on the four weeks before. 0.0% of all new papers.

Can LLMs Reliably Grade Olympiad Proofs? A Study of Mathematical Verification with LLMs
Oct 27, 2026Azim Ospanov, Zijin Feng, Ding Ding +5Automated Grading Language Model Reasoning Evaluation
Huawei Foundation Model Department · The Chinese University of Hong Kong · The Hong Kong University of Science and Technology
SoK: Grading Open-Text Legal Answers, from Law School Exams to Trustworthy Rubric Judges
Oct 14, 2026Sebastian NaglAutomated Grading LLM-as-a-Judge
Technical University of Munich (TUM), Germany
Machine--human agreement in marking undergraduate assignments is specific to model, prompt and institution
Oct 13, 2026Giulio Corsi, Maryam Abo-Tabik, Yael Benn +4Automated Grading LLM-as-a-Judge
Centre for the Future of Intelligence, University of Cambridge, Cambridge, UK · Department of Computer Science, University of Lancashire, Preston, UK · School of Psychology, Manchester Metropolitan University, Manchester, UK +3
TeachGrade: Training-Free Short Answer Scoring with Teacher Evidence
Oct 10, 2026Jinhao Yan, Zhihong Shen, Shengjie Wang +4Automated Grading LLM-as-a-Judge
Fuzhou University · Peking University
Who Verifies the Verifier? Co-Evolving Inspectable Graders with Self-Improving Agents
Oct 8, 2026Xing Zhang, Guanghui Wang, Yanwei Cui +4Automated Evaluation LLM Agent Self-Improvement
AWS Forward Deployed Engineering · HSBC Holdings Plc., HSBC Technology Center, China
Alice: A Large-Scale German Benchmark for Rubric-Based Multi-Dimensional Automatic Short Answer Scoring
Oct 7, 2026Zhifan Sun, Sebastian Gombert, Jannik Lossjew +6Automated Grading Educational Assessment
DIPF | Leibniz Institute for Research and Information in Education · IPN | Leibniz Institute for Science and Mathematics Education · Umeå University +1
Rubric Spans are Label Representations: Joint LLM Encoding for Short Answer Scoring
Oct 7, 2026Zhifan Sun, Sebastian Gombert, Fabian Zehner +3Automated Grading Rubric-Based Evaluation
DIPF | Leibniz Institute for Research and Information in Education · Centre for International Student Assessment (ZIB) · Computer Science Department & Studiumdigitale, Goethe University Frankfurt
LENS-GRF: Permutation-Invariant Lesion Evidence Network with Gated Residual Fusion for Acne Severity Grading and Multi-Rater Clinical Oracle Analysis
Sep 25, 2026Muhammad Muhtasim Shahriar, M. F. MridhaSet Transformer Automated Grading
Department of Computer Science, International Islamic University Chittagong (IIUC), Chittagong, Bangladesh · Department of Computer Science, American International University-Bangladesh (AIUB), Dhaka, Bangladesh
Where LLM Graders Succeed and Break: Evidence from Two Computer-Science Exams
Sep 24, 2026Ali Habibullah, Yazan Alshoibi, Mohammad Alshiekh +2Prompt Sensitivity LLM-as-a-Judge
KAUST Academy Computer, Electrical & Mathematical Sciences & Engineering (CEMSE) King Abdullah University of Science and Technology (KAUST) Thuwal, Saudi Arabia · Visual Artificial Intelligence Laboratory Oxford Brookes University · KAUST Academy & CEMSE, KAUST +1
ImpossibleRubrics: Stress-Testing Generated Rubrics as Reward Signals
Sep 15, 2026Bowen Qin, Yi Xie, Yesheng Liu +1Reward Hacking Language Model Generation Evaluation
National University of Singapore · Peking University · Institute of Automation, Chinese Academy of Sciences
Vision-Language Models for Criterion-Level Grading of Handwritten Examinations in Outcome-Based Education
Sep 13, 2026Asif Hasan Tonmoy, Saad Ahmed, Md Khalid Syfullah +1VLM Evaluation Rubric-Based Evaluation
Reliability, validity, and diagnostic evidence for multi-model LLM short-answer scoring
Sep 6, 2026Chunyi Zhao, Chao LiAutomated Grading Educational Assessment
AIEd Governance
Who Should Grade My Work? Student Perspectives on Transparent AI-Assisted Writing Assessment in Higher Education
Sep 4, 2026Rayed AlGhamdiGenerative AI in Education Human Oversight of AI
Department of Information Technology Faculty of Computing and Information Technology King Abdulaziz University, Jeddah, Saudi Arabia
A Human-in-the-Loop Framework for AI-Assisted Scoring in Large-Scale Writing Assessment
Sep 4, 2026María Eugenia Curi, Germán Capdehourat, Isabel Amigo +4Educational Assessment Automated Grading
Ceibal, Avda. Italia 6201, Montevideo, 11500, Uruguay. · ANEP, Av. Libertador 1409, Montevideo, 11100, Uruguay.
Check The Scoreboard: An Analysis of Scoring Schemes on Multiple-Choice Evaluation
Aug 30, 2026Nishant Balepur, Paiheng Xu, Wei Ai +3LLM Evaluation Multiple-Choice Question Answering
University of Maryland · New York University · Nanyang Technological University
Large Scale AI Grading of Handwritten Physics Assessments: Score Agreement and Olympiad Team Selection Outcomes
Aug 20, 2026Praveen Pathak, Siddharth Tiwary, Charudatt Kadolkar +4Automated Grading Educational Assessment
Homi Bhabha Centre for Science Education–TIFR, Mumbai, India · Lawrence Livermore National Laboratory, Livermore, CA, USA · University of California, Berkeley, CA, USA +2
Measuring the Partial-Credit Gap: A Strict Benchmark on Vietnam's 2025 Convex Marking Scheme
Aug 18, 2026Nguyen Quoc Hung, Nguyen Dang Minh, Le Nhu Quynh +2Educational Assessment Benchmark Validity
Posts and Telecommunications Institute of Technology, Hanoi, Vietnam
Grading Needs a Rubric, Not Intelligence
Aug 18, 2026Jhen-Ke LinAutomated Grading Rubric-Based Evaluation
National Yang Ming Chiao Tung University
WebGrader: Training LLMs for Web Development with Self-Evolving Programmatic Grader
Aug 6, 2026Boshui Chen, Huiping Liu, Shaolei ZhangRL for Code Generation Automated Software Testing
Beijing Institute of Technology, China · Beijing Jiaotong University, China · Renmin University of China, China
AI-Based Thesis Assessment: An Empirical Study of Human Evaluation Priorities and Their Impact on Automated Assessment
Aug 1, 2026Garv Vikram Gursahaney, Baskhad Idrisov, Thorsten Fröhlich +1Rubric-Based Evaluation Automated Grading
IU International University of Applied Sciences, Germany.
The Theoretical Foundation of Socratic Tests: Dynamic, Multimodal, Conversational Examinations
Jul 31, 2026Ilya MikhelsonEducational Assessment Multi-Turn Dialogue Evaluation
Department of Electrical and Computer Engineering, Northwestern University, Evanston, IL, USA
Weakly Supervised Instance-Level Gleason Pattern Estimation Using Primary and Secondary Labels
Jul 26, 2026Nao Sugeta, Kaito Shiku, Shinnosuke Matsuo +1Weakly Supervised Learning Oncology
Kyushu University, Japan
Bounding Boxes to Improve Small Language Model Performance on Vision-Based Grading Tasks
Jul 21, 2026Lachlan McGinnessVLM Evaluation Educational Assessment
Australian National University, Acton, ACT, Australia
From Pixel to Prognosis: Convolutional and GLCM Feature Fusion for Automated Four-Class Cataract Severity Classification
Jul 20, 2026K. Mithra, Prem Kumar SanthanamMedical Image Classification Automated Grading
Independent Researcher, Information and Communication Engineering, Scottsdale, AZ, USA · Seidenberg School of Computer Science and Information Systems, Pace University, New York City, NY, USA
ICME 2026 Grand Challenge on Cross-Scenario Defect Detection and Fine-Grained Severity Grading for High-Precision Manufacturing
Jul 6, 2026Wei Sun, Weixia Zhang, Linhan Cao +30Industrial Anomaly Detection Industrial Inspection
East China Normal University, Shanghai Jiao Tong University, Tsinghua Shenzhen International Graduate School, and Universit´e de Nantes.
Automated grading of Linux/bash examinations using large language models: a four-level cognitive taxonomy approach
Jul 2, 2026Manuel Alonso-Carracedo, Ruben Fernandez-Boullon, Pedro Celard +2LLM-as-a-Judge Computer Science Education
aUniversidade de Vigo, Department of Computer Science, ESEI-Higher School of Computer Engineering, Edificio Politécnico, Campus Universitario As Lagoas s/n, Ourense, 32004, Spain · bIFCAE-Institute for Research in Physics, Computing and Aerospace Science, Universidade de Vigo, Ourense, 32004, Spain
HG-Bench: A Benchmark for Multi-Page Handwritten Answer-Region Grounding in Automated Homework Assessment
Jun 24, 2026Chuangxin Zhao, Boyan Shi, Yanling Wang +7VLM Evaluation Multimodal Grounding
Grading the Grader: Lessons from Evaluating an Agentic Data Analysis System
Jun 23, 2026Tian Zheng, Kai-Tai HsuLLM-as-a-Judge Human-in-the-Loop Evaluation
Department of Statistics Columbia University New York, NY
LLM Performance on a Real, Double-Marked GCSE Benchmark
Jun 23, 2026Malachy Fox, Kavi Samra, Paul JungLLM Evaluation Educational Assessment
Medly AI
Confidence-Aware Automated Assessment of Student-Drawn Scientific Models
Jun 18, 2026Luyang Fang, Yingchuan Zhang, Jongchan Park +3Selective Prediction Vision Transformer
AI4STEM Education Center, Athens, GA, USA · Department of Statistics, University of Georgia, Athens, GA, USA

Oct 27, 2026J/K move · Enter open · S save

Can LLMs Reliably Grade Olympiad Proofs? A Study of Mathematical Verification with LLMs

Azim Ospanov, Zijin Feng, Ding Ding, Chengwu Liu+4

Huawei Foundation Model Department · The Chinese University of Hong Kong · The Hong Kong University of Science and Technology

Automated Grading Language Model Reasoning Evaluation Automated Theorem Proving

Open paper AskPDF arXiv

Large Language Models (LLMs) have made remarkable progress on solving competition-level mathematics, yet their ability to verify natural-language mathematical proofs remains relatively underexplored. This gap is especially pronounced for Olympiad-level proofs, which require meticulous assessment of each claim in a long reasoning chain. Current proof verification thus primarily relies on expert inspection, which is costly to scale. In this paper, we present a systematic study of LLM-based verification in this setting. Specifically, we introduce a human-curated dataset of USAMO 2025 candidate solutions with expert grading, and conduct a broad evaluation of existing LLM verifiers across frontier models. We also propose TROJAN, an iterative self-critique pipeline for generating high-fidelity adversarial proofs at scale. Our comprehensive evaluation covers 20 models spanning seven metrics, two prompt styles, three inference methods, and three prompt templates, showing that state-of-the-art models still leave considerable room for improvement. To narrow this gap, we propose MABGrader, a Multi-Armed Bandit framework that reframes grading as arm selection over the discrete score space. MABGrader outperforms the SOTA ProofGrader by 26.4% in Quadratic Weighted Kappa and reduces grading error by 14%. Code and dataset are available at https://github.com/aziksh-ospanov/TROJAN.