VeriMath: Human-in-the-Loop Generation and Verification of Curriculum-Aligned Mathematics Assessment Items
Authors: Quy P. Nguyen, Duong Q. Nguyen, Dinh P. T. Nguyen, Truong-Giang T. Le, Truong A. Bui, Vi H. Vo, Quang-Thinh Bui, My-Phuong Ngo, Trong N. Do, Thao P. Le, Phuc M. Nguyen, H. Nguyen-Xuan
Organizations: Center for AI Research, VinUniversity, Ho Chi Minh City, Viet Nam · An Lao High School, Gia Lai, Viet Nam · Chu Van An High School for Gifted Students, Gia Lai, Viet Nam · FPT Play, Ho Chi Minh City, Viet Nam · Applied Mathematics and Data Science (AMaDaS) Research Group, Tien Giang University, Dong Thap, Viet Nam · Tien Giang High School for the Gifted, Dong Thap, Viet Nam · Department of Mathematics and Statistics, Quy Nhon University, Quy Nhon City, Viet Nam · College of Electro-mechanics, Construction and Agro-forestry of Central Vietnam, Gia Lai, Viet Nam · College of Engineering and Computer Science, VinUniversity, Hanoi, Viet Nam
Automatic generation of mathematics assessment items is a growing area in educational artificial intelligence, yet classroom deployment remains challenging. Generated questions must satisfy multiple constraints: mathematical correctness, curriculum alignment, appropriate cognitive difficulty, and educator reviewability. While prior work has advanced text generation, reliable assessment authoring requires granular control over localized examination formats and expert educational judgment. To address this challenge, we present VeriMath, a generalized, human-in-the-loop framework for generating rigorous, curriculum-constrained mathematics assessment drafts. Using annotated reference items, the framework anchors generation to the requested examination section and cognitive reasoning level, then applies sequential automated verification and guided repair before submitting drafts for expert review. We evaluate VeriMath through a national curriculum case study using 4,555 mathematics items from the Vietnamese Grade 12 curriculum and 50 distinct generation requirements covering multiple-choice, true/false, and short-answer formats. Across two model backbones, the framework produces verifier-accepted drafts for 94% (47/50) of requirements per run. Blind expert evaluations confirm high mathematical accuracy and curriculum alignment, with strict teacher-readiness assessments accepting up to 56.8% of generated items. Analysis of rejected drafts reveals persistent domain-agnostic challenges involving formatting, statement validity, and solution-step completeness. These findings demonstrate that VeriMath can bridge generative AI and dependable educational applications by producing highly reviewable assessment drafts within tightly constrained national curriculum frameworks.
Figures & tables
Research line
Representative work
Main technical idea
Strengths and remaining gap
Structured generation
Polozov et al. (2015)
Generate problems from structured specifications, mathematical models or student relevant contexts.
Strong control over formal structure, but limited variation and weak support for local examination formats.
Neural word problem generation
Zhou and Huang (2019) , Wang et al. (2021) , Niyarepola et al. (2022)
Condition generation on equations, topics, contexts or language resources.
Improves fluency and diversity, but does not fully address curriculum metadata, item schema validity or expert readiness.
Pedagogically guided educational authoring
Kibar and Ilgaz (2026) ; Cassano et al. (2026)
Connect AI-assisted design with pedagogical review and explicit item-writing criteria.
Provides an instructional-design and assessment-authoring basis; VeriMath examines these concerns in curriculum-constrained mathematics.
Planning and retrieval
Qin et al. (2023) ; Qin et al. (2024) , Cheng et al. (2025)
Use structure planning, memory retrieval or question planning to guide generation.
Improves controllability, yet generated outputs still require stronger review for mathematical validity and assessment use.
Teacher evaluated generation
Christ et al. (2024) ; Christ et al. (2026) , Utami et al. (2024)
Use teacher labels, standards or authentic contexts as educational quality signals.
Connects generation to pedagogy, but many systems focus on word problems rather than national examination item schemas.
Verification and evaluation
Cobbe et al. (2021) , Lightman et al. (2024) , Liu et al. (2023) , Zheng et al. (2023)
Use verifier models, process feedback or model based rubrics to screen generated reasoning.
Scales diagnostic review, but automatic judgment remains an imperfect substitute for expert educational assessment.
Table 1: Positioning of VeriMath Relative to Major Lines of Mathematics Item Generation
Case-study code s
Item format
Required output
Level specification
1
Multiple choice
One stem, four answer options, one correct label and a detailed solution.
One level from Table 3 .
2
True or false cluster
One shared stem, four statements, four binary labels and a detailed solution for all statements.
Four entry profile L=(ℓ1,ℓ2,ℓ3,ℓ4) .
3
Short answer
One stem, one numeric final answer and a detailed solution, with rounding instructions when needed.
One level from Table 3 .
Table 2: Assessment Schemas Used in the Vietnamese Case-Study Instantiation
Level
Definition
Recognition
Direct formula or concept recall with at most one reasoning step.
Comprehension
Requires interpretation and combination of known concepts.
Application
Requires multi step reasoning or a non routine strategy.
Table 3: Teacher-Annotated Cognitive/Difficulty Levels in the Vietnamese Case Study
Curriculum subject
Slots
Share
Calculus/statistics
67
83.8%
Geometry
13
16.2%
Table 4: Subject Distribution of the Grade 12 Requirement Set
Figure 1: Topic-block distribution of the Grade 12 requirement set. Bars report the share of the 80 learning-outcome slots, with slot counts shown in parentheses
Data split or pool
Total
s=1
s=2
s=3
Reference corpus Dref
4,555
2,988
449
1,118
Experiment requirements
50
30
10
10
Experiment learning outcome slots
80
30
40
10
Table 5: Dataset Statistics
Figure 2: Overview of the VeriMath framework. Difficulty Anchoring selects schema- and level-matched reference items, while CVM repeatedly verifies and repairs a candidate before acceptance
1
Input: requirement R , corpus Dref , anchor count k , pass threshold N , repair limit T .
2
Select anchors BR←SelectAnchors(R,Dref,k) .
3
Generate initial candidate (P,S,M)←Mgen(R,BR) .
4
Self-improve (P,S,M) using the schema associated with s .
5
Set passes←0 and repairs←0 .
6
while passes<N and repairs<T do
7
(d,E)←Mver(P,S,M,R) .
Table 8
Step
Decision or output
Relevant content
Init
Candidate generated
Stem transforms y=f(x)=x3 by horizontal compression, vertical stretch, reflection over the x -axis and a shift right by 1. It asks for the x -coordinate where the tangent is horizontal. Answer: 1.
Self-improve
Candidate rewritten
The candidate changes to a horizontal stretch by 3, vertical compression by 1/2 , reflection over the y -axis and upward shift by 2. It asks for slope −3 but reports answer 2.
Verify 1
IS_CORRECT: FALSE
The verifier derives h(x)=−x3/54+2 and h′(x)=−x2/18 . Solving h′(x)=−3 gives x=±36 , so the official answer 2 is invalid.
Correct 1
Candidate rewritten
The corrected candidate uses horizontal stretch by 2, vertical compression by 1/2 , reflection over the y -axis and upward shift by 1. It derives h(x)=−x3/16+1 but reports answer 0.5.
Verify 2
IS_CORRECT: FALSE
The verifier accepts the transformation and derivative but rejects the answer: h′(x)=−3 gives x=±4 , not 0.5.
Correct 2
Candidate rewritten
The candidate reports answer −4 , but the stem still has no extra condition, so both x=4 and x=−4 satisfy the slope condition.
Table 6: Verifier-Guided Revision Trace for a Generated Item
Figure 3: Pipeline cost by processing step for accepted outputs. Panel (a) reports token cost normalized per accepted item; Panel (b) reports aggregate recorded duration. Paired points within a row compare the two models
Backbone
Accepted
Acceptance
Novelty ↑
Verifier calls/item
Repairs/item
Qwen3-8B
47/50
94.0%
0.813
3.85
0.70
gpt-oss-20b
47/50
94.0%
0.787
3.53
0.34
Table 7: Full-Pipeline Diagnostics for the Reported Backbones
Backbone
Mean time/item
Median time/item
Input tok./item
Output tok./item
Total tok./item
Recorded tok.
Qwen3-8B
136.1 s
91.4 s
10,547
11,111
21,659
1,125,879
gpt-oss-20b
468.0 s
318.7 s
11,425
13,588
25,012
1,175,581
Table 8: Runtime and Token Cost of the Full Pipeline
Backbone
Schema
Accepted/ recorded
Mean time
Median time
Input tok.
Output tok.
Total tok.
Qwen3-8B
s=1
29/30
125.8 s
83.4 s
9,372
10,625
19,997
Qwen3-8B
s=2
8/9
162.3 s
105.2 s
15,175
11,085
26,260
Qwen3-8B
s=3
10/10
145.3 s
130.1 s
10,255
12,542
22,798
gpt-oss-20b
s=1
30/30
367.3 s
309.3 s
9,744
12,541
22,285
gpt-oss-20b
s=2
7/10
941.2 s
659.0 s
19,265
19,304
38,569
gpt-oss-20b
s=3
10/10
438.9 s
290.2 s
10,979
12,727
23,705
Table 9: Runtime and Token Cost by Examination Schema
Backbone
Step
Calls
Mean sec./call
Total sec.
Mean tok./call
Tok./item
Qwen3-8B
Initial generation
47
27.0
1,267.1
4,678
4,678
Qwen3-8B
Self-improvement
47
22.0
1,033.8
4,560
4,560
Qwen3-8B
Verification
171
12.6
2,157.9
2,276
8,280
Qwen3-8B
Verifier-guided correction
27
46.3
1,250.8
5,887
3,382
Qwen3-8B
L a T e X formatting repair
7
38.9
272.2
5,093
759
gpt-oss-20b
Initial generation
47
107.2
5,036.8
6,108
6,108
Table 10: Token and Duration Breakdown by Pipeline Step
Figure 4: Expert-review outcomes for Qwen/Qwen3-8B and gpt-oss-20b. Binary criteria are shown as pass rates; ordinal criteria are shown as mean ratings on a 1–5 scale. Paired points within a row compare the two models
Model
Reviewed items
Expert pass ↑
Correct. ↑
Align. ↑
Level ↑
Clarity ↑
Qwen3-8B
44
56.8%
95.5%
95.5%
3.83
4.10
gpt-oss-20b
39
56.4%
93.6%
94.9%
3.64
3.69
Table 11: Blind Expert Review Under the Four-Criterion Teacher Rubric
Error flag
Count
Share
Invalid problem statement
14
43.8%
Formatting or formula error
5
15.6%
Missing or incorrect solution step
3
9.4%
Wrong learning outcome
2
6.2%
Too hard
2
6.2%
Ambiguous or missing condition
2
6.2%
Table 12: Expert-Marked Error Flags in Reviewed Generated Items
Figure 5: Interfaces of the Vietnamese case-study digital mathematics learning platform. The platform includes modules for problem solving, item generation and exam synthesis, while the present study evaluates the curriculum-constrained item-authoring component
Automatic Item Generation (AIG) is pivotal for personalized education, yet guaranteeing the pedagogical value of generated items remains a bottleneck. Existing Item Quality Assessment (IQA) methods typically rely on unscalable manual reviews or shallow stem-based metrics, failing to capture the reasoning process required for mathematical problem-solving. To bridge this gap, this paper proposes Process-based Item Quality Assessment (ProIQA), a process-aware framework for fine-grained quality assessment of math items. We first formulate IQA across three heterogeneous dimensions, including knowledge concepts, difficulty, and disciplinary competencies, under a unified process-aware perspective. Based on this formulation, we construct a process-enhanced IQA resource by augmenting original item data with structured reasoning trees derived from raw solutions. Technically, ProIQA leverages Large Language Modelsto construct hierarchical reasoning trees and employs Graph Neural Networks (GNN) to encode their topological dependencies and procedural semantics. The resulting solving representation is fused with stem semantics through a dual-view (``Stem + Solving'') architecture, enabling comprehensive assessment across learning objectives. Extensive experiments on K12 mathematical datasets show that ProIQA effectively captures process-oriented features, offering a scalable data-driven solution for evaluating AIG outputs in intelligent education systems.
Junkai Tong, Mingjia Li, Haoran Chen +4
East China Normal University · Shanghai, China · East China Normal University, Shanghai, China +1
Generative artificial intelligence (GenAI) tools such as OpenAI's ChatGPT are transforming the educational landscape, prompting reconsideration of traditional assessment practices. In parallel, universities are exploring alternatives to in-person, closed-book examinations, raising concerns about academic integrity and pedagogical alignment in uninvigilated settings. This study systematically investigates the performance of GenAI on typical mathematics questions from across a first-year mathematics curriculum. Adopting an empirical approach and utilising current examination questions as a proxy for course content, we generate, transcribe, and blind-mark GenAI submissions to eight undergraduate mathematics assessments, spanning the entirety of the first-year curriculum. By combining independent GenAI responses to individual questions, we enable a meaningful evaluation of GenAI performance, both at the level of modules and across the first-year curriculum. We find that GenAI attainment is at the level of a first-class degree, though current performance can vary between modules. Further, we find that GenAI performance is remarkably consistent when viewed across the entire curriculum, significantly more so than that of students in invigilated examinations. Our findings evidence the pressing need for redesigning assessments in mathematics in the era of generative artificial intelligence.
Benjamin J. Walker, Nikoleta Kalaydzhieva, Beatriz Navarro Lameda +1
Department of Mathematics, University College London, Gordon Street, WC1H 0AY, London, UK
Automatically generating textbook-grounded assessment items can reduce science teachers' workload, but existing retrieval-augmented generation (RAG) systems rely on flat retrieval, support only single-question generation, lack safeguards against weak evidence, and are ill-suited to low-resource, board-exam-structured curricula. We address these limitations with TeachMateGPT, a multi-agent system contributing four advances to curriculum-grounded science-assessment authoring. (i) COPE, a hierarchical knowledge base replacing token-window chunking with a multi-resolution index that segments documents along syllabus structure and links them at three granularities via a traversable graph-based lineage, matching evidence to each topic's instructional level. (ii) A staged, fail-closed agent pipeline replacing one-shot retrieve-then-generate: routing gates search, retrieval fuses dense and lexical evidence under a coverage gate that withholds generation on insufficient evidence, and specialist agents draft objective and constructed-response items. (iii) SAVER, a source-attributed verification protocol scoring faithfulness, relevance, and hallucination risk against retrieved evidence, applying stricter grounding checks across each creative question's four sub-parts, paired with teacher-in-the-loop evaluation rather than automatic filtering. (iv) NCTB-SciGen8, a curriculum-grounded dataset of 198 items (143 multiple-choice, 55 creative questions) spanning all 14 chapters of the NCTB Class 8 science textbook, produced by the pipeline and rated by three practicing teachers. TeachMateGPT raises faithfulness (0.68 → 0.96) and answer relevancy (0.60 → 0.89) over a vanilla RAG baseline.
Fatema Tuj Johora Faria, Mukaffi Bin Moin, M. F. Mridha +1
Ahsanullah University of Science and Technology, Bangladesh · American International University - Bangladesh · Jashore University of Science and Technology, Bangladesh