BRIDGE the Gap: Mitigating Bias Amplification in Automated Scoring of English Language Learners via Inter-group Data Augmentation
Authors: Yun Wang, Xuansheng Wu, Jingyuan Huang, Lei Liu, Xiaoming Zhai, Ninghao Liu
Organizations: School of Computing, University of Georgia, Athens, GA, USA · Educational Testing Service, Princeton, NJ, USA · AI4STEM Education Center, University of Georgia, Athens, GA, USA · The Hong Kong Polytechnic University, Hong Kong, China
In the field of educational assessment, automated scoring systems increasingly rely on deep learning and large language models (LLMs). However, these systems face significant risks of bias amplification, where model prediction gaps between student groups become larger than those observed in training data. This issue is especially severe for underrepresented groups such as English Language Learners (ELLs), as models may inherit and further magnify existing disparities in the data. We identify that this issue is closely tied to representation bias: the scarcity of minority (high-scoring ELL) samples makes models trained with empirical risk minimization favor majority (non-ELL) linguistic patterns. Consequently, models tend to under-predict ELL students who even demonstrate comparable domain knowledge but use different linguistic patterns, thereby undermining the fairness of automated scoring outcomes. To mitigate this, we propose BRIDGE, a Bias-Reducing Inter-group Data GEneration framework designed for low-resource assessment settings. Instead of relying on the limited minority samples, BRIDGE synthesizes high-scoring ELL samples by "pasting" construct-relevant (i.e., rubric-aligned knowledge and evidence) content from abundant high-scoring non-ELL samples into authentic ELL linguistic patterns. We further introduce a discriminator model to ensure the quality of synthetic samples. Experiments on California Science Test (CAST) datasets demonstrate that BRIDGE effectively reduces prediction bias for high-scoring ELL students while maintaining overall scoring performance. Notably, our method achieves fairness gains comparable to using additional real human data, offering a cost-effective solution for ensuring equitable scoring in large-scale assessments.
Figures & tables
Figure 1: Overview of the bias amplification loop and the BRIDGE framework. (a) Bias Propagation: Representation bias in training data creates a feedback loop that reinforces educational disparities. (b) ERM Vulnerability: Under ERM, the decision boundary for high scores is skewed by the majority non-ELL group, causing systematic underprediction of the sparse ELL high-score subgroup. (c) BRIDGE Framework: Our approach mitigates this by performing inter-group stylistic reformulation, extracting construct-relevant content from high-scoring non-ELL samples and injecting ELL-specific linguistic patterns, followed by a discriminator for quality control.
Fairness
Performance
Dataset
Aug. Method
MSG Gap ↓
BiasAmp ↓
Acc.(%) ↑
MAE ↓
QWK(%) ↑
F1(%) ↑
Dataset 1
Baseline (no aug.)
0.0279 ± 0.0134
0.0097 ± 0.0081
90.17 ± 0.48
0.0988 ± 0.0053
89.77 ± 0.64
88.68 ± 0.50
+ Real-data (oracle)
0.0100 ± 0.0129
0.0025 ± 0.0093
90.19 ± 0.48
0.0993 ± 0.0046
89.68 ± 0.34
88.58 ± 0.59
+ Oversampling
0.0279 ± 0.0092
0.0107 ± 0.0054
90.41 ± 0.42
0.0962 ± 0.0043
90.05 ± 0.40
89.01 ± 0.40
+ Paraphrasing
0.0268 ± 0.0137
0.0082 ± 0.0094
90.31 ± 0.32
0.0974 ± 0.0035
89.86 ± 0.39
88.77 ± 0.33
+ Ours
0.0079 ± 0.0139
-0.0014 ± 0.0085
90.22 ± 0.27
0.0994 ± 0.0031
89.53 ± 0.38
88.63 ± 0.27
Table 1: Bias mitigation performance of different data augmentation methods on five CAST datasets, each with 20 additional SEL,high samples per method. Mean ± standard deviation over 10 random seeds. Best and second-best MSG Gap and BiasAmp are highlighted in bold and underlined , respectively.
Automated Essay Scoring systems disproportionately penalize high-proficiency English as a Second Language (ESL) learners. We propose Contrastive Learning with Matched Essay Pairs (CL-MEP), a bi-directional alignment strategy. CL-MEP reduces this scoring bias by 39.9% while improving overall accuracy, successfully disentangling valid syntactic complexity from surface-level grammatical errors.
Kevin Fan, Eric Yun
Georgia Institute of Technology · Georgia State University
Large Language Models (LLMs) are often used as evaluators of text quality, known as LLM-as-a-Judge, which can outperform conventional automatic evaluation metrics that rely on reference texts. However, LLM evaluators tend to generate particular scores regardless of the context of the evaluated text, which is known as scoring bias. This study proposes a novel method to mitigate this scoring bias. An LLM is instructed to randomly generate number tokens, and the latent numerical bias of the LLM is identified by measuring the deviation of the observed distribution of numbers from the uniform distribution. A definition of a downstream task, for which an LLM evaluator is used, is added to the prompts for random number generation to measure task-specific latent number bias. In the evaluation by an LLM, the token generation probabilities for a given input are rectified considering the LLM's latent number bias. Results of the experiment on four different tasks, evaluation of LLM alignment, evaluation of summarization, Semantic Textual Similarity, and Semantic Textual Relatedness, demonstrate that our proposed method outperforms the baselines, including an LLM without debiasing and previous calibration methods. In addition, it is confirmed that scoring bias varies across LLMs, tasks, and score ranges, indicating the importance of measuring latent number bias as the case may be.
This study examines the cross-prompt generalization and first-language (L1) scoring effects of a LoRA-adapted open-weight large language model (Gemma-3-27B-it) applied to automated essay scoring. Using the identical model and inference configuration reported in "AiAWE: An Open-Source LLM Automated Writing Evaluation System Using LoRA-Adapted Instruction-Tuned Models" (Gayed, 2026), which was fine-tuned on 480 argumentative essays from two prompts, we evaluate scoring accuracy on the full TOEFL11 corpus: 12,100 essays written by test-takers from 11 first-language backgrounds across eight prompts, none of which were seen during training. The model's raw scores (0.5-5.0) are mapped to the same three proficiency bands (low, medium, high) used by ETS, enabling direct comparison. The model achieved an overall band agreement of 77.79% and a quadratic weighted kappa of 0.702, with adjacent-band agreement of 99.98%. Accuracy was stable across all eight unseen prompts, with no advantage for prompts thematically related to the training data, indicating robust cross-prompt generalization. However, the model exhibited a systematic, L1-linked scoring offset. Within every proficiency band, essays from European-language backgrounds received consistently higher scores than essays from East-Asian-language backgrounds, a pattern not attributable to the composition of the fine-tuning data. This is the first large-scale L1 fairness analysis of a fine-tuned open-weight LLM for automated essay scoring.