Chinese Grammatical Error Correction (CGEC) is typically formulated as a text-to-text problem. While suitable for digital texts, this formulation falls short in real-world K-12 education, where AI-assisted corrections are expected directly on essay images, which are primarily handwritten and exhibit many irregularities. To bridge this gap, we introduce Grounded Chinese Grammatical Error Correction (G-CGEC), an image-to-grounded-edit task requiring systems to predict both corrections and their positions on essay images. We propose G-CGEC-Eval, a benchmark comprising real-world images of K-12 Chinese essays with fine-grained annotations. For evaluation, we adapt IoU-based metrics from object detection and propose a more tolerant alternative based on center-distance matching. Using this benchmark, we evaluate open-source and proprietary vision-language models under a direct single-pass setting, as well as settings with auxiliary components (OCR and character grounding) coordinated via pipelining or agentic harnessing. Results show that even top-tier proprietary models perform surprisingly poorly in the single-pass setting, while pipelining and agentic harnessing yield meaningful improvements. Notably, an error-type-based multi-agent workflow achieves the best performance, establishing it as the strongest baseline for future G-CGEC research. Through ablations and fine-grained analysis, we find the task remains far from solved, with linguistic error discovery and character-level grounding as the primary bottlenecks. This underscores the need for models that better integrate visual perception with linguistic reasoning and visual grounding.
Figures & tables
Figure 1: When the input is in the image format, our image-to-grounded-edit formulation of the error correction task offers intuitive advantages: giving a system complete information as opposed to lossy OCR transcription; providing practical usefulness (e.g., for K-12 education).
Figure 2: Error category distribution in the G-CGEC-Eval benchmark. Horizontal bars show fine-grained categories grouped by 7 coarse categories (bold labels, separated by gray lines). All bars within a group share the same color. Note that syntactic, discourse-level errors, and punctuation errors together account for over 80% of all errors.
Statistic
Value
Total essays
150
Total images
251
Total corrected text segments
2 731
Total edits (primary)
2 913
Avg. edits per segment
1.1
Edits with insertion operations
849 (29.1%)
Table 1: Statistics of G-CGEC-Eval. Operations are counted from the primary solution of each segment.
Scheme
Model
P (%)
R (%)
F1 (%)
Single-pass
Qwen3.5-27B
1.86
0.55
0.85
MiMo-V2.5
0.80
0.10
0.18
GPT-5.5
0.53
0.14
0.22
Pipeline
Qwen3.5-27B
18.69
10.55
13.48
MiMo-V2.5
20.10
10.75
14.01
GPT-5.5
16.32
9.97
12.37
Table 2: Main results across schemes and models. For Agentic schemes, (S), (E), (O) represents the S ingle-agent paradigm, the multi-agent paradigm with the E rror-type-based sub-agents, and the multi-agent paradigm with the O peration-type-based sub-agents, respectively.
Scheme
Model
F1 (%)
VLM (Single-pass)
Qwen3.5-27B
0.85
MiMo-V2.5
0.18
GPT-5.5
0.22
OCR + VLM
Qwen3.5-27B
1.13
MiMo-V2.5
0.00
GPT-5.5
0.38
Table 3: Ablation experiments to study the individual impacts of the OCR module and the visual grounding (VG) module
Scheme
Model
F1 (%) & Δ
OCR + VLM
Qwen3.5-27B
0.83 ( − 0.30)
MiMo-V2.5
0.06 ( + 0.06)
GPT-5.5
0.44 ( + 0.06)
Pipeline
Qwen3.5-27B
15.24 ( + 1.76)
MiMo-V2.5
16.37 ( + 2.36)
GPT-5.5
16.86 ( + 4.49)
Table 4: Results of the oracle experiment to study the effect of OCR quality. Each cell reports the F1 after replacing the OCR module results with the human-annotated transcriptions, with the absolute change shown in parentheses.
Figure 3: Fine-grained failure decomposition. Each configuration has three bars: TP, FP, and FN. Bar heights are raw counts. FP and FN bars are internally stacked by subtype, and in-bar percentages (rounded to integers) denote subtype proportions. In the legend, visual , textual , and both indicate which kind of match failed.
Figure 4: Center-distance-based analysis as a complement to Table 2 . Colors indicate inference paradigms (Single-pass, Pipeline, Agentic), while marker shapes distinguish individual systems. Line segments up to CD@ 38 are shown as solid lines and the remaining segments are shown as dashed lines.
Appendix figures & tables8 assets
Supplementary material from the paper’s appendix.
Appendix
Scheme
Model
P (%)
R (%)
F1 (%)
Single-pass
Gemini 3.1 Pro
11.10
3.19
4.95
Qwen3.5-397B-A17B
6.04
1.70
2.65
Qwen3.5-27B
1.86
0.55
0.85
GPT-5.5
0.53
0.14
0.22
MiMo-V2.5
0.80
0.10
0.18
Pipeline
Gemini 3.1 Pro
22.50
15.32
18.23
Appendix
Table 5: Deployable system results across a broader set of VLM backends. Agentic (S), (E), and (O) denote single-agent, error-type-based sub-agent, and operation-type-based sub-agent workflows, respectively.
Scheme
Model
P (%)
R (%)
F1 (%)
Single-pass
Gemini 3.1 Pro
16.00
4.43
6.94
Qwen3.5-27B
1.64
0.55
0.83
GPT-5.5
1.04
0.28
0.44
MiMo-V2.5
0.26
0.03
0.06
Pipeline
Gemini 3.1 Pro
28.26
18.80
22.58
GPT-5.5
18.86
15.24
16.86
Appendix
Table 6: Oracle results using human-annotated transcriptions across additional VLM backends. Agentic (S) and (E) denote single-agent and error-type-based sub-agent workflows, respectively.
IoU threshold
Per-system optimal- r range
Per-system optimal- r median
Per-system optimal- r mean
Median-curve peak r
Median P at peak
Median R at peak
Median F1 at peak
0.3
0 – 22
18
16.54
18
0.8504
0.9075
0.8808
0.01
19 – 42
36
34.38
38
0.9145
0.9039
0.9065
Appendix
Table 7: Empirical pair-level mapping between IoU thresholds and CD thresholds. For each IoU threshold τ , we report the distribution of per-system optimal r (the r that maximizes pair-level F1 ) and the aggregate median P/R/ F1 at the peak.
Figure 5: Pair-level precision, recall, and F1 of CD matching against IoU matching across the 37 system outputs. Shaded regions denote the 25–75 percentile range across systems.
Figure 6: Instance-level illustration of the IoU–CD correspondence. Top row: progressively relaxing the IoU threshold. Bottom row: increasing the CD threshold r . Blue solid boxes are ground truth, orange dashed boxes are predictions. Gray boxes are not yet visually eligible; thicker outlines indicate pairs newly admitted at each threshold. Intermediate thresholds are case-specific breakpoints chosen for visualization.
Visual-only pass@ k (%)
Full-match pass@ k (%)
Model
@1
@2
@4
@8
@16
@1
@2
@4
@8
@16
Qwen3-VL-235B-A22B-Thinking
3.97
6.72
10.32
14.56
19.45
0.38
0.73
1.35
2.34
3.70
Qwen3-VL-30B-A3B-Instruct
9.92
15.63
23.33
33.28
43.94
1.48
2.83
5.19
8.87
13.46
Qwen3-VL-32B-Instruct
5.53
9.42
14.60
20.86
28.91
1.31
2.33
3.80
5.48
7.43
Qwen3-VL-32B-Thinking
1.69
3.20
5.78
9.62
14.28
0.04
0.08
0.17
0.34
0.68
Qwen3-VL-8B-Instruct
2.32
3.35
4.95
7.33
10.34
0.00
0.00
0.00
0.00
0.00
Appendix
Table 8: Pass@ k results for the selected default prompt (Variant 1) at IoU@ 0.3 . Each backend has 32 sampled generations on the same 10-essay subset. Scores are percentages; Visual-only requires only spatial matching, while Full-match additionally requires the textual edit to match exactly.
Table 9: Prompt variant sensitivity results. U denotes the model-normalized utility score ( μ−σ ) aggregated over pass@ k for k∈{1,2,4,8,16} . Visual-only computes pass@ k using only the visual matching criterion, while Full-match requires both visual and exact textual matches. All variants are evaluated on the same set of VLM backends.
IoU Threshold
Avg. IoU
Recall (%)
Precision (%)
0.2
60.60
90.04
91.17
0.5
67.65
69.00
69.87
Appendix
Table 10: Intrinsic evaluation results of the visual grounding module on the held-out test set. Performance is reported under varying IoU thresholds for spatial matching.
Large Language Model (LLM) based Chinese Grammatical Error Correction (CGEC) systems face two critical challenges: general-purpose models lack specialized linguistic priors for subtle grammatical distinctions, and Supervised Fine-Tuning (SFT) with Maximum Likelihood Estimation fails to optimize for precision-focused metrics, leading to systematic over-correction. We propose CSRP, a three-stage framework that progressively builds correction capability through Continual Pre-training (CPT) on 5.9M balanced samples to internalize domain knowledge, Chain-of-Thought SFT with explicit error reasoning for diagnostic transparency, and Group Relative Policy Optimization with a novel Efficiency-Aware Reward that explicitly penalizes unnecessary edits. On the NACGEC benchmark, CSRP achieves state-of-the-art performance with 50.99 F0.5 and 57.17 precision, substantially outperforming previous best results while effectively mitigating the over-correction bias inherent in MLE-trained models. Our method also advances CSCD spelling correction to 59.61 F1, surpassing GPT-4 by 5.20 points. Comprehensive ablation studies demonstrate that the RL alignment stage contributes a 8% relative gain over the SFT baseline, and that this gain is orthogonal to the contribution of large-scale CPT, validating that explicit optimization for edit efficiency is essential for high-quality grammatical error correction. Our code is available at https://github.com/TW-NLP/ChineseErrorCorrector.
Wei Tian, Yuhao Zhou, Man Lan
School of Computer Science and Technology, East China Normal University · Shanghai Institute of Artificial Intelligence for Education, East China Normal University
Automated assistants for Grammatical Error Correction are now embedded in educational platforms serving millions of learners, yet three critical gaps remain in this domain: (1) latest-generation Large Language Models (LLMs) lack comprehensive evaluation on grammar correction tasks; (2) whether combining these LLMs improves correction quality is unexplored; and (3) the extent to which reference-based metrics underestimate GEC system performance has not been adequately quantified. In this study, first, we evaluate latest-generation LLMs on edit precision, fluency preservation, and meaning retention, showing fine-tuned GPT-4o achieves state-of-the-art performance across all three dimensions. Second, through grammatical error type analysis we demonstrate that individual LLMs exhibit highly similar error correction patterns (ρ=0.947). Third, we show that reference-based metrics underestimate GEC performance with 73.76% of GPT-4o corrections different from gold standards being equally valid or even superior. These GEC evaluation findings equip educators with guidance for selecting GEC assistants that enhance rather than constrain student linguistic development. We make our data, code, and models publicly available.
Adnan Labib, Qiao Wang, Yixuan Huang +1
King’s College London, London, United Kingdom · Hosei University, Tokyo, Japan · Waseda University, Tokyo, Japan +1
A Grammatical Error Correction (GEC) system produces a sequence of edits to correct an erroneous sentence. The quality of these edits is typically evaluated against human annotations. However, a sentence may admit multiple valid corrections, and existing evaluation settings do not fully accommodate diverse application scenarios. Recent meta-evaluation approaches rely on human judgments across multiple references, but they are difficult to scale to large datasets. In this paper, we propose a new task, Scoring Edit Impact in GEC, which aims to automatically estimate the importance of edits produced by a GEC system. To address this task, we introduce a scoring framework based on an embedded association graph. The graph captures latent dependencies among edits and syntactically related edits, grouping them into coherent groups. We then perform perplexity-based scoring to estimate each edit's contribution to sentence fluency. Experiments across 4 GEC datasets, 4 languages, and 4 GEC systems demonstrate that our method consistently outperforms a range of baselines. Further analysis shows that the embedded association graph effectively captures cross-linguistic structural dependencies among edits.
Qiyuan Xiao, Xiaoman Wang, Yunshi Lan
School of Data Science and Engineering, East China Normal University