cs.CVAug 23, 2026

Multimodal examination answer data with expert-designed Outcome-Based Education rubrics for criterion-level assessment

Authors: Jahangir Alam SM, Md Khalid Syfullah, Saad Ahmed, Munira Akter Mou, A K Z Rasel Rahman, A. K. M. Masudur Rahman, Mohammed Sowket Ali

Organizations: Department of Computer Science and Engineering (CSE), Bangladesh Army University of Science and Technology (BAUST), Saidpur, Bangladesh

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

This data article describes a multimodal collection of scanned examination answers paired with expert-designed Outcome-Based Education (OBE) grading metadata. The collection contains 485 answer submissions from 415 consenting students at four academic institutions. Eight faculty contributors supplied examination materials covering nine subjects and 12 distinct question templates. Each answer-level item links a scanned PDF to a randomized identifier, subject label, question, model answer, criterion definitions, performance-level descriptions, criterion marks, and a total mark. The 12 rubrics contain 47 criteria in total. The scans retain realistic academic content, including handwriting, printed text, equations, tables, code, figures, sketches, and diagrams. CamScanner, Adobe Scan, and conventional scanners contributed variation in illumination, contrast, orientation, compression, and resolution. Diverse handwriting, crossed-out work, revised calculations, and inserted corrections add further visual variability for robustness and generalization studies. Preparation involved heterogeneous-source consolidation, label and text standardization, score validation, identifier randomization, filename randomization, and JSON-to-PDF integrity checks. An answer-level audit confirmed 485 unique identifiers, 485 unique PDF filenames, agreement between each total mark and its criterion-mark sum, and scores within the applicable rubric maximum. The data can support rubric-aware automated evaluation, multimodal document understanding, criterion-level feedback, score prediction, and privacy-aware OBE assessment research. Access is restricted to research use and is available from the corresponding author upon reasonable request.

Figures & tables

Figure 1: Representative scanned answer pages from the dataset. The overview shows variation in handwriting, page count, equations, tables, diagrams, and institutional document formats.
Figure 2: Answer-level JSON representation linking a randomized scan with its question, reference material, rubric criteria, obtained marks, and performance-level rules.
SubjectAnswersQuestionsShare (%)Mean normalized mark (%)Visual answer content
Machine Learning88218.1477.90Primarily text
Digital Image Processing75215.4677.20Primarily text
Database Systems80216.4962.91Text, tables, equations
Computer Networks70114.4364.57Text, optional diagrams
Data Mining4218.6656.40Primarily text
Algorithms83117.1182.98Text, equations, diagrams
Table 2: Subject-level composition. Mean normalized mark is descriptive metadata rather than a model evaluation.
QSubject and focusQuestion summaryAnswer typeAnswersCriteriaMean markRange
1Machine Learning, R2R^{2}Importance, mathematical interpretation, limitations, and model comparisonPlain text2542.82/50 to 5
2Machine Learning, paradigmsSupervised, unsupervised, and reinforcement learningPlain text6348.64/100 to 10
3Digital Image Processing, quantizationPurpose of quantization and the Max-Lloyd algorithmPlain text4048.53/102 to 10
4Digital Image Processing, histogramsHistogram equalization and histogram matchingPlain text3546.80/104 to 10
5Database Systems, normalizationRedundancy, update anomalies, normal forms, and examplesText, tables, equations5046.25/102 to 9.5
6Database Systems, keysSchema, instance, database keys, and relation-specific identificationText, tables, equations3046.37/103 to 10
Table 3: Question-level grading structure, answer type, and mark coverage.
Figure 3: Outcome-Based Education rubric materials used during data acquisition. The three source PDFs are included directly and displayed as subfigures.
Figure 4: Dataset preparation workflow from raw examination materials to the standardized answer-level collection. The supplied process diagram is included directly.
More in this topic

Educational Assessment

Educational assessment measures what learners know and can do, using tests, rubrics, and learning evidence to guide decisions.

See all 151 papers

Explore similar work

CardsList
Aug 18, 2026

Grading Needs a Rubric, Not Intelligence

Small language models can grade open-ended examination answers as reliably as substantially more expensive models when they grade against an explicit rubric. We test this claim as the design principle behind any-to-bench: a frontier model reads source documents once, at ingestion, to extract each question and its rubric; lower-cost models then perform all repeated grading work. We evaluate six cost-efficient model configurations from two model families at three reasoning-effort levels. Each configuration answers 24 open-ended examination questions, and each also grades every answer sheet three times, yielding 3,456 per-question grades. Scores depend overwhelmingly on the answer being graded: answer identity explains 95.6% of score variance, whereas judge identity explains only 0.2%. Raising a writer's reasoning effort moves earned scores by as much as 0.143 of full marks, while raising a judge's reasoning effort moves assigned scores by at most 0.006. Six frontier-tier judges, added as a check, reproduce these scores and are no more reliable as a panel. Two ablations then decompose the rubric on the same questions and answers. Removing its criteria and levels while keeping the official answer changes nothing measurable. Removing the official answer as well collapses reliability (ICC 0.888 to 0.628), inflates scores, and makes judge reasoning effort matter again. The rubric is what decouples grading from judge intelligence, and within the rubric the official answer does nearly all the work. We find no evidence of length preference or same-family preference under rubric-anchored grading.

Jhen-Ke Lin

National Yang Ming Chiao Tung University

Automated GradingRubric-Based EvaluationLLM-as-a-JudgeLLM ReliabilityBias in LLM-as-a-Judge
AskPDF
May 12, 2026

Human-Grounded Multimodal Benchmark with 900K-Scale Aggregated Student Response Distributions from Japan's National Assessment of Academic Ability

Authentic school examinations provide a high-validity test bed for evaluating multimodal large language models (MLLMs), yet benchmarks grounded in Japanese K-12 assessments remain scarce. We present a multimodal dataset constructed from Japan's National Assessment of Academic Ability, comprising officially released middle-school items in Science, Mathematics, and Japanese Language. Unlike existing benchmarks based on synthetic or curated data, our dataset preserves real exam layouts, diagrams, and Japanese educational text, together with nationwide aggregated student response distributions (N ≈\approx 900{,}000). These features enable direct comparison between human and model performance under a unified evaluation framework. We benchmark recent multimodal LLMs using exact-match accuracy and character-level F1 for open-ended responses, observing substantial variation across subjects and strong sensitivity to visual reasoning demands. Human evaluation and LLM-as-judge analyses further assess the reliability of automatic scoring. Our dataset establishes a reproducible, human-grounded benchmark for multimodal educational reasoning and supports future research on evaluation, feedback generation, and explainable AI in authentic assessment contexts. Our dataset is available at: https://github.com/KyosukeTakami/gakucho-benchmark

Kyosuke Takami, Yuka Tateisi, Satoshi Sekine +1

Osaka Kyoiku University · University of Tokyo · NII LLMC

Educational AssessmentMultimodal ReasoningMultimodal Model EvaluationMultimodal UnderstandingMLLM EvaluationMultimodal Dataset
AskPDF
Jun 9, 2026

Towards Fully Automated Exam Grading: Fairness-Aware Recognition of Handwritten Answers with Foundation Models

Correcting handwritten exams by hand is time-consuming and error-prone, particularly for large cohorts, while fully digital exams tend to force a didactic narrowing towards closed question formats. A practical middle ground keeps paper-based, problem-oriented tasks but records the assessment-relevant answers as single capital letters in a table that a machine can read. The open question is whether this reading can be made accurate and, above all, fair enough for unsupervised grading. Earlier automated approaches reached only about 88%--91% recognition -- too low -- and failed on the cases that matter most: answers placed outside the cell, crossed out, or written in cursive. We show that general-purpose vision-language foundation models (VLMs), which interpret the page rather than match pixel templates, close this gap. On a benchmark of 61 anonymised exams (3141 answer positions) the best model reaches 98.4% accuracy, well above the previous baseline. Crucially, we centre the evaluation on fairness: we distinguish false negatives (a correct answer marked wrong, which disadvantages the student) from false positives, and a lightweight prompt that supplies the reference solution as context lowers the false-negative rate to 0.58%. Under an exemplary grading scheme only three of the 61 exams would be graded worse, all caught by a student self-review step. Fully automated, fairness-aware exam grading at scale is therefore defensible; we release the anonymised benchmark to support reproducibility.

Hartwig Grabowski

Institute for Machine Learning and Analytics (IMLA), Offenburg University, Offenburg, Germany.

VLM EvaluationAlgorithmic FairnessOnline Handwriting RecognitionOptical Character RecognitionAutomated Grading
AskPDF