cs.CLMar 28, 2026
SaveListen, Correct, and Feed Back: Spoken Pedagogical Feedback Generation
Organizations: Mohamed bin Zayed University of Artificial Intelligence, Abu Dhabi, UAE
Abstract
Grammatical error correction (GEC) and explanation (GEE) have made rapid progress, but real teaching scenarios also require learner-friendly pedagogical feedback that is actionable, level-appropriate, and encouraging. We introduce SPFG (Spoken Pedagogical Feedback Generation), a dataset built based on the Speak & Improve Challenge 2025 corpus, pairing fluency-oriented transcriptions with GEC targets and human-verified teacher-style feedback, including preferred/rejected feedback pairs for preference learning. We study a transcript-based Spoken Grammatical Error Correction (SGEC) setting and evaluate three instruction-tuned LLMs (Qwen2.5, Llama-3.1, and GLM-4), comparing supervised fine-tuning (SFT) with preference-based alignment (using DPO and KTO) for jointly generating corrections and feedback. Results show that SFT provides the most consistent improvements, while DPO/KTO yield smaller or mixed gains, and that correction quality and feedback quality are weakly coupled. Our implementation is available at https://github.com/Skywalker-Harrison/spfg.
Figures & tables
Figure 1: Illustration of three learner-support functions in GEC. Correction rewrites the original utterance into a grammatically well-formed sentence; explanation identifies the error and states the underlying rule; feedback provides learner-friendly guidance with actionable suggestions and encouraging tone.
Figure 2: Distribution of the top 15 grammatical error types in the dataset. The color intensity corresponds to the frequency of each error type, with darker shades indicating higher counts.
| Split | # Audio | Time (hrs) | # Words | Avg. Words |
|---|---|---|---|---|
| Train | 4,285 | 30.24 | 151,859 | 35.44 |
| Dev | 500 | 3.53 | 17,739 | 35.48 |
| Eval | 2,793 | 20.40 | 102,108 | 36.56 |
Table 1: Summary statistics of the dataset across different splits.
Figure 3: Overview of SPFG. Left : data construction from Speak & Improve (S&I), producing GEC targets and human-verified pedagogical feedback (see Section 3 ). Middle : inference format and a two-stage training pipeline that combines preference-based alignment and supervised fine-tuning (SFT) to jointly generate corrections and feedback (see Section 4 ). Right : evaluation with automatic metrics (e.g., WER and ERRANT) and pedagogical quality assessment using an LLM-as-a-judge (see Sections 5 ).
| CEFR | Percentage | Avg. Len |
|---|---|---|
| A2 | 1.0% / 1.8% / 1.2% | 135.5 / 152.2 / 166.8 |
| A2–B1 | 3.0% / 3.5% / 2.6% | 154.7 / 166.9 / 142.4 |
| B1 | 10.9% / 10.5% / 8.4% | 159.2 / 180.1 / 165.2 |
| B1–B2 | 20.7% / 22.7% / 23.8% | 166.0 / 182.8 / 168.4 |
| B2 | 24.7% / 26.2% / 23.4% | 170.1 / 186.3 / 175.7 |
| B2–C1 | 25.1% / 23.2% / 28.8% | 166.9 / 186.6 / 162.8 |
Table 2: Distribution of CEFR levels and average feedback length (in words) across train (4,285), eval (2,793), and dev (500) splits. CEFR levels range from A2 to C1–C2.
| Model | WER | ERRANT | Feedback Quality (LLM-judge) | ||||||
|---|---|---|---|---|---|---|---|---|---|
| Correct. | Level | Sugg. | Posit. | Avg. | |||||
| Proprietary Models | |||||||||
| DeepSeek-Chat | 21.74 | 43.8 | 58.6 | 46.1 | 4.97 | 4.99 | 4.86 | 4.99 | 4.95 |
| Gemini-2.5-Flash | 25.34 | 40.0 | 54.7 | 42.3 | 5.00 | 4.98 | 4.85 | 5.00 | 4.96 |
| Qwen-Plus | 38.76 | 25.7 | 44.9 | 28.1 | 4.97 | 4.94 | 4.90 | 4.79 | 4.90 |
| Open-source LLMs | |||||||||
Table 3: Grammar correction and feedback generation results across different models. WER controls ASR quality; ERRANT evaluates grammatical correction accuracy; feedback quality is assessed by an LLM judge along four pedagogically motivated dimensions, with Avg. denoting their mean. The best results for Open-source LLMs are marked in bold and the second best results are underlined .
Figure 4: Pearson correlation matrix between Word Error Rate (WER) and LLM-evaluated feedback quality dimensions (Correctness, Level Appropriateness, Suggestion Quality, and Positiveness).
Figure 5: ERRANT’s score heatmap comparing SFT, DPO+SFT and KTO+SFT of all open-sourced models on top 15 error types.
Appendix figures & tables4 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 6: Readability and difficulty statistics of training feedback across fluency-score bins, including Reading Ease, grade-level estimates, and difficult-word counts for y-axis respectively, while x-axis is CEFR level. ranging from A2 (2) to C2 (6)
| Field | Text |
|---|---|
| Source (fluent) | i think it depends on the activity we are going to do because if i am trying to play video games or playing football i prefer to do it with my brothers |
| Gold Label | i think it depends on the activity we are going to do because if i am trying to play video games or play football i prefer to do it with my brothers |
| Vanilla Prediction | i think it depends on the activity we are going to do because if i am trying to play video games or playing football i prefer to do it with my brothers |
| SFT Prediction | i think it depends on the activity we are going to do because if i am trying to play video games or play football i prefer to do it with my brothers |
| Vanilla Feedback | Great effort, and nice job on speaking clearly! Let’s take a look at the sentence together. One thing I’d like to suggest is that you can use more precise vocabulary to express your idea. Instead of saying "it depends on the activity we are going to do", you could say "it depends on the activity we’re planning to do" or even "it depends on the activity we’re going to engage in". This will make your sentence sound more natural and fluent. Also, when you say "if i am trying to play video games or playing football", you’re using a bit of a double negative. Try to simplify it to "if I’m trying to play video games or play football". This will make your sentence easier to understand. Lastly, instead of saying "i prefer to do it with my brothers", you could say "i prefer to do it with my brothers" is actually fine but "i prefer to do it with my brother and sister" or "i prefer to do it with my family" would be better options. Overall, your sentence is clear and easy to understand, but with a few tweaks, it can sound even more natural and polished. Keep up the good work! |
| SFT Feedback | Great job expressing your thoughts clearly! There is just a small adjustment needed to make your sentence more grammatically correct. In your sentence, you used "playing" after "play video games." Since you are using "or" to list activities, it’s important to keep the verb form consistent. So, instead of "playing football," it should be "play football" to match the form of "play video games." This way, both parts of the sentence have the same structure, making it clearer and more grammatically sound. You’re doing really well at this level, and with attention to these small details, your expression will become even more precise. Keep practicing, and don’t hesitate to experiment with different sentence structures to enhance your fluency. You’re on the right track! |
Table 4: Case study comparing vanilla and SFT corrections and feedback. Red indicates incorrect explanations, orange indicates excessive suggestions beyond the rubric, and green indicates correct and level-appropriate feedback.
| Error Code | Explanation | Example (erroneous corrected) |
|---|---|---|
| R:OTHER | Other replacement (often multi-word or idiomatic). | nature food is good for health natural food is good for health |
| R:NOUN:NUM | Incorrect noun number (singular vs. plural). | I like to know different culture I like to know different cultures |
| R:PREP | Incorrect preposition choice. | so at summer we usually bike to the beach so in summer we usually bike to the beach |
| U:DET | Unnecessary determiner (article). | in general the technology is good in general technology is good |
| R:VERB:FORM | Incorrect verb form (e.g., infinitive vs. gerund). | we love swim we love swimming |
| R:VERB | Incorrect lexical verb choice. | the best way is to enter into a page the best way is to open a page |
Table 5: The 15 most frequent ERRANT error types in the evaluation corpus. Prefixes denote edit operations: R (replacement), M (missing), and U (unnecessary). Examples are drawn from the spoken learner corpus and illustrate typical error patterns.
Figure 7: We present the distribution of the top 15 grammatical error types across eight CEFR proficiency bands (A2 through C1–C2) in the training set. Each bar represents the percentage of a given error type relative to the total number of errors produced at that proficiency level. Intermediate bands (A2–B1, B1–B2, B2–C1, C1–C2) correspond to half-point boundary scores on the assessment scale. Error types are labeled according to the ERRANT annotation scheme, where prefixes denote the edit operation (R: replacement, M: missing insertion, U: unnecessary deletion) and suffixes indicate the linguistic category. The color scale transitions from red (lower proficiency) to green (higher proficiency), highlighting proficiency-related trends.