Multimodal examination answer data with expert-designed Outcome-Based Education rubrics for criterion-level assessment

Jahangir Alam SM; Md Khalid Syfullah; Saad Ahmed; Munira Akter Mou; A K Z Rasel Rahman; A. K. M. Masudur Rahman; Mohammed Sowket Ali

cs.CVAug 23, 2026

Multimodal examination answer data with expert-designed Outcome-Based Education rubrics for criterion-level assessment

Authors: Jahangir Alam SM, Md Khalid Syfullah, Saad Ahmed, Munira Akter Mou, A K Z Rasel Rahman, A. K. M. Masudur Rahman, Mohammed Sowket Ali

Organizations: Department of Computer Science and Engineering (CSE), Bangladesh Army University of Science and Technology (BAUST), Saidpur, Bangladesh

Abstract

This data article describes a multimodal collection of scanned examination answers paired with expert-designed Outcome-Based Education (OBE) grading metadata. The collection contains 485 answer submissions from 415 consenting students at four academic institutions. Eight faculty contributors supplied examination materials covering nine subjects and 12 distinct question templates. Each answer-level item links a scanned PDF to a randomized identifier, subject label, question, model answer, criterion definitions, performance-level descriptions, criterion marks, and a total mark. The 12 rubrics contain 47 criteria in total. The scans retain realistic academic content, including handwriting, printed text, equations, tables, code, figures, sketches, and diagrams. CamScanner, Adobe Scan, and conventional scanners contributed variation in illumination, contrast, orientation, compression, and resolution. Diverse handwriting, crossed-out work, revised calculations, and inserted corrections add further visual variability for robustness and generalization studies. Preparation involved heterogeneous-source consolidation, label and text standardization, score validation, identifier randomization, filename randomization, and JSON-to-PDF integrity checks. An answer-level audit confirmed 485 unique identifiers, 485 unique PDF filenames, agreement between each total mark and its criterion-mark sum, and scores within the applicable rubric maximum. The data can support rubric-aware automated evaluation, multimodal document understanding, criterion-level feedback, score prediction, and privacy-aware OBE assessment research. Access is restricted to research use and is available from the corresponding author upon reasonable request.

Figures & tables

Figure 1: Representative scanned answer pages from the dataset. The overview shows variation in handwriting, page count, equations, tables, diagrams, and institutional document formats.

Figure 2: Answer-level JSON representation linking a randomized scan with its question, reference material, rubric criteria, obtained marks, and performance-level rules.

Subject	Answers	Questions	Share (%)	Mean normalized mark (%)	Visual answer content
Machine Learning	88	2	18.14	77.90	Primarily text
Digital Image Processing	75	2	15.46	77.20	Primarily text
Database Systems	80	2	16.49	62.91	Text, tables, equations
Computer Networks	70	1	14.43	64.57	Text, optional diagrams
Data Mining	42	1	8.66	56.40	Primarily text
Algorithms	83	1	17.11	82.98	Text, equations, diagrams

Table 2: Subject-level composition. Mean normalized mark is descriptive metadata rather than a model evaluation.

Q	Subject and focus	Question summary	Answer type	Answers	Criteria	Mean mark	Range
1	Machine Learning, $R^{2}$	Importance, mathematical interpretation, limitations, and model comparison	Plain text	25	4	2.82/5	0 to 5
2	Machine Learning, paradigms	Supervised, unsupervised, and reinforcement learning	Plain text	63	4	8.64/10	0 to 10
3	Digital Image Processing, quantization	Purpose of quantization and the Max-Lloyd algorithm	Plain text	40	4	8.53/10	2 to 10
4	Digital Image Processing, histograms	Histogram equalization and histogram matching	Plain text	35	4	6.80/10	4 to 10
5	Database Systems, normalization	Redundancy, update anomalies, normal forms, and examples	Text, tables, equations	50	4	6.25/10	2 to 9.5
6	Database Systems, keys	Schema, instance, database keys, and relation-specific identification	Text, tables, equations	30	4	6.37/10	3 to 10

Table 3: Question-level grading structure, answer type, and mark coverage.

Panel of Figure 3 — Figure 3: Outcome-Based Education rubric materials used during data acquisition. The three source PDFs are included directly and displayed as subfigures.

Figure 4: Dataset preparation workflow from raw examination materials to the standardized answer-level collection. The supplied process diagram is included directly.

Educational Assessment

Educational assessment measures what learners know and can do, using tests, rubrics, and learning evidence to guide decisions.

See all 151 papers

Grading Needs a Rubric, Not Intelligence
Aug 18, 2026Jhen-Ke LinAutomated Grading Rubric-Based Evaluation
National Yang Ming Chiao Tung University
Human-Grounded Multimodal Benchmark with 900K-Scale Aggregated Student Response Distributions from Japan's National Assessment of Academic Ability
May 12, 2026Kyosuke Takami, Yuka Tateisi, Satoshi Sekine +1Educational Assessment Multimodal Reasoning
Osaka Kyoiku University · University of Tokyo · NII LLMC
Towards Fully Automated Exam Grading: Fairness-Aware Recognition of Handwritten Answers with Foundation Models
Jun 9, 2026Hartwig GrabowskiVLM Evaluation Algorithmic Fairness
Institute for Machine Learning and Analytics (IMLA), Offenburg University, Offenburg, Germany.

Aug 18, 2026J/K move · Enter open · S save

Grading Needs a Rubric, Not Intelligence

Jhen-Ke Lin

National Yang Ming Chiao Tung University

Automated Grading Rubric-Based Evaluation LLM-as-a-Judge LLM Reliability Bias in LLM-as-a-Judge

Open paper AskPDF arXiv

Small language models can grade open-ended examination answers as reliably as substantially more expensive models when they grade against an explicit rubric. We test this claim as the design principle behind any-to-bench: a frontier model reads source documents once, at ingestion, to extract each question and its rubric; lower-cost models then perform all repeated grading work. We evaluate six cost-efficient model configurations from two model families at three reasoning-effort levels. Each configuration answers 24 open-ended examination questions, and each also grades every answer sheet three times, yielding 3,456 per-question grades. Scores depend overwhelmingly on the answer being graded: answer identity explains 95.6% of score variance, whereas judge identity explains only 0.2%. Raising a writer's reasoning effort moves earned scores by as much as 0.143 of full marks, while raising a judge's reasoning effort moves assigned scores by at most 0.006. Six frontier-tier judges, added as a check, reproduce these scores and are no more reliable as a panel. Two ablations then decompose the rubric on the same questions and answers. Removing its criteria and levels while keeping the official answer changes nothing measurable. Removing the official answer as well collapses reliability (ICC 0.888 to 0.628), inflates scores, and makes judge reasoning effort matter again. The rubric is what decouples grading from judge intelligence, and within the rubric the official answer does nearly all the work. We find no evidence of length preference or same-family preference under rubric-anchored grading.

Abstract

Educational Assessment

Explore similar work

Grading Needs a Rubric, Not Intelligence

Human-Grounded Multimodal Benchmark with 900K-Scale Aggregated Student Response Distributions from Japan's National Assessment of Academic Ability

Towards Fully Automated Exam Grading: Fairness-Aware Recognition of Handwritten Answers with Foundation Models