Exploring Data Augmentation and Resampling Strategies for Transformer-Based Models to Address Class Imbalance in AI Scoring of Scientific Explanations in NGSS Classroom
Authors: Prudence Djagba, Kevin Haudek, Clare G. C. Franovic, Leonora Kaldaras
Organizations: CREATE for STEM, Michigan State University, East Lansing, USA · Human Biology, Michigan State University, East Lansing, USA · University of Houston, Houston, USA
Automated scoring of students' scientific explanations offers the potential for immediate, accurate feedback, yet class imbalance in rubric categories particularly those capturing advanced reasoning remains a challenge. This study investigates augmentation strategies to improve transformer-based text classification of student responses to a physical science assessment based on an NGSS-aligned learning progression. The dataset consists of 1,466 high school responses scored on 11 binary-coded analytic categories. This rubric identifies six important components including scientific ideas needed for a complete explanation along with five common incomplete or inaccurate ideas. Using SciBERT as a baseline, we applied fine-tuning and test these augmentation strategies: (1) GPT-4--generated synthetic responses, (2) EASE, a word-level extraction and filtering approach, and (3) ALP (Augmentation using Lexicalized Probabilistic context-free grammar) phrase-level extraction. While fine-tuning SciBERT improved recall over baseline, augmentation substantially enhanced performance, with GPT data boosting both precision and recall, and ALP achieving perfect precision, recall, and F1 scores across most severe imbalanced categories (5,6,7 and 9). Across all rubric categories EASE augmentation substantially increased alignment with human scoring for both scientific ideas (Categories 1--6) and inaccurate ideas (Categories 7--11). We compared different augmentation strategies to a traditional oversampling method (SMOTE) in an effort to avoid overfitting and retain novice-level data critical for learning progression alignment. Findings demonstrate that targeted augmentation can address severe imbalance while preserving conceptual coverage, offering a scalable solution for automated learning progression-aligned scoring in science education.
Figures & tables
Figure 1: Cart item.
Level
Brief Description
3
Student models and explanations represent causal relationships that integrate ideas of energy and Coulombic interactions at the atomic-molecular level to explain phenomena.
2
Student models and explanations represent causal relationship that use but do not integrate (or inaccurately integrate) the ideas of energy and/or Coulombic interactions at the macro or atomic-molecular level to explain phenomena with some inaccuracies
1
Student models and explanations represent partially causal relationship that use ideas of Coulombic interactions or energy with inaccurate/incomplete ideas to explain phenomena
0
Student models and explanations don’t represent causal relationships and use ideas of Coulombic interactions and/or energy with significantly inaccurate/incomplete ideas
Table 1: 3D Learning Progression for electrical interactions. Adapted from [27, 29].
Rubric Category
Rubric Description
Class imbalance ratio
1
Predicts the direction of cart motion, indicating that the carts move away from each other (e.g., repel, separate, move backward, or specifying the direction of each cart).
5.817
2
Uses the fundamental property of electric charge to justify motion, explicitly stating that like (negative) charges repel, causing the carts to move apart.
1.090
3
Predicts when or whether the carts will stop or slow down, regardless of correctness (e.g., "eventually slow down," "stop when…").
1.287
4
Uses Coulombic reasoning (charge, force, or field) to explain when the carts stop, emphasizing the role of distance in weakening interactions or forces.
1.725
5
Explains when the carts stop using energy ideas only, such as energy conversion (potential , kinetic, surroundings) or systems moving toward a minimum energy state, without integrating Coulomb’s law.
19.551
6
Integrates energy and Coulomb’s law to explain motion and stopping, linking high potential energy and strong repulsive forces when close, and decreasing force/energy as distance increases.
40.103
Table 2: Analytic rubric for the explanation component of the CART items and their corresponding levels of class imbalance.
Figure 2: Taxonomy and grouping for different data augmentation methods [11].
Category
C5
C6
C7
C9
Class ratio
10.06
8.57
9.98
10.06
Table 3: Summary of the level of balance ratio after GPT augmentation.
Category
C5
C6
C7
C8
C9
C10
C11
Class ratio
1.13
1.14
1.13
1.05
1.13
1.08
1.08
Table 4: Summary of the level of balance ration after the EASE Augmentation.
Category
C5
C6
C7
C8
C9
C10
C11
Class ratio
9.33
28.33
7.80
4.60
8.95
1.84
6.32
Table 5: Summary of the level of balance ration after the ALP Augmentation.
Category
Metrics
Baseline model: SB
FT SB
SB+ GPT Augmentation
SB+ ALP Augmentation
SB+ EASE Augmentation
SB + SMOTE Over sampling
5
Accuracy
97.61
97.26
97.70
100
99.89
96.92
5
Precision
83.33
60.00
95.80
100
99.64
56.25
5
Recall
45.45
81.81
79.30
100
99.88
81.81
5
F1-score
58.82
69.33
86.80
100
99.76
66.66
6
Accuracy
98.29
99.31
98.50
99.00
100
98.63
6
Precision
0.00
100
89.70
100
100
50.00
Table 6: Accuracy, precision, recall, F1 for different strategies on Categories 5 and 6 (Scientific ideas). Note: SB = SciBERT, FT = Fine-tuning.
Category
Metrics
Baseline model: SB
FT SB
SB+ GPT Augmentation
SB+ EASE Augmentation
SB+ ALP Augmentation
SB + SMOTE Over sampling
7
Accuracy
98.29
99.31
97.40
99.63
100
99.65
7
Precision
92.30
100
100
98.68
100
93.33
7
Recall
75.00
87.50
75.76
99.01
100
100
7
F1-score
82.75
93.33
86.21
98.85
100
96.55
9
Accuracy
92.49
93.17
95.50
99.60
99.50
95.56
9
Precision
0
0
77.78
97.55
100
40.00
Table 7: Accuracy, precision, recall, F1 for different strategies on Categories 7 and 9 (Non Scientific ideas). Note: SB = SciBERT, FT = Fine-tuning.
Category
Metrics
Baseline model: SciBERT
FT SB
SB+ ALP
SB+ EASE Augmentation
SB + SMOTE
1-6
Accuracy
92.03
96.13
99.08
99.47
95.96
1-6
Precision
72.22
88.91
98.72
99.36
80.03
1-6
Recall
67.20
83.30
92.89
99.45
81.96
1-6
F1-score
68.75
83.02
94.96
99.41
79.84
7-11
Accuracy
93.31
95.22
99.40
99.63
95.56
7-11
Precision
47.69
58.96
98.34
98.76
64.64
Table 8: Accuracy, precision, recall, F1 for different strategies for all Categories (divided by Scientific ideas and non Scientific ideas). Note: SB = SciBERT, FT = Fine-tuning.
N
Generated responses with ALP
1
They are unstable so they will gained and they will increases when they are far apart because so that means they will repel from each other .
2
I suppose that since they are the high ( they are both somewhere charged ) they with move the same way and wo n’t stop until they get to no forces between them .
3
When the aka are known the object would move away from each other since they have the same charges . Being both negative are both a negative charge and like charges repel . The would both stop when the charge ca n’t reach each other .
4
When the whole will given less from each other because they both have negatively charged plates on them and same charges repel , they will move away from each other because they both have negatively charged plates on them and same charges repel . They move that direction , because both the metal sheets are negatively charged , We know that like charges repel , so depending on how strong the negative charge is , is how far the carts will move away from each other .
5
They will comes and attracted the remove way from a strong force between them . Considering that they are opposites they will not connect whatsoever . They would most likely stop when they will be a strong force between them two probably making the carts stop when they are really far away from each other .
6
The simulation attracting the repel.they because the cars have the same charge negative charge . They will push away until there is no more energy then they are negatively charged
Table 9: Examples of ALP generated responses.
N
Generated responses with EASE
1
They move that direction, because both the metal sheets are negatively charged, We know that like charges repel, so depending on how strong the negative charge is, is how far the carts will move away from each other.
2
when the wedges move the carts would move away from each other because like charges do not attract.
3
I think that the cars will move away from each other due to both of the sheets having the same charge of the other, and will only come to a stop upon hitting a hall, loosing momentum, or coming back around to face the opposite vehicle sheet to sheet.
4
They will stop when they leave the magnetic field around them the magnetic field is very small when they are close together but the farther away they get from each other the weaker the field will get.
5
The cars will move away from each other cause they are both negative and so opposites repel so the forces and energy will push them away from each other.
6
The two carts will repel from one another by different forces and end up the same distance apart because the energy and force between the same charged plates will make the carts have the same repelling force.
Table 10: Examples of EASE generated responses.
N
Generated responses with GPT-5
1
The cars move because of repulsion from similar charges. The energy is high when they’re close and goes down as they move farther apart until there is no field pushing them anymore. They stop when the energy is at its lowest. They continue to move until the field is minimized.
2
since the cars are negatively charged, the potential energy between them will turn into kinetic energy as they move apart. but once the energy is gone, they’ll just stop cause they don’t have anymore force pushing them.
3
the cars start moving cause the energy between them is high, and it turns into kinetic energy as they move away. as they get farther, the system’s energy goes down but its not clear if they’ll ever actually stop.
4
they move apart because they’re trying to reach a low energy state. the potential energy is high when they’re close and decreases as they get farther apart. they stop when the field is at its lowest. they will keep moving until the energy is gone, reaching a stable position.
5
the repulsive force between the carts is what makes them move when the wedges are removed. when they’re close, they have a lot of energy, but as the carts move apart, both the energy and force decrease until they can’t move anymore.
6
when the wedges come out, the carts will start moving because their potential energy is really high. the kinetic energy increases as they move apart, but they won’t stop until the energy between them is used up.
Table 11: Examples of GPT generated responses.
Category
Metrics
Baseline model: SB
FT SB
SB+ GPT Augmentation
FT SB + GPT Augmentation
5
Accuracy
97.61
97.26
97.70
97.75
5
Precision
83.33
60.00
95.80
86.67
5
Recall
45.45
81.81
79.30
89.66
5
F1-score
58.82
69.33
86.80
88.14
6
Accuracy
98.29
99.31
98.50
99.06
6
Precision
0.00
100
89.70
93.10
Table 12: Accuracy, precision, recall, and F1-score for Categories 5, 6, 7, and 9 across the different strategies, using SciBERT fine-tuned with the GPT-based augmentation approach. Note: SB = SciBERT, FT = Fine-tuning.
Synthetic data augmentation is widely used to mitigate class imbalance, but its theoretical effects on score-based classification remain poorly understood. This paper develops a framework for characterizing when synthetic minority augmentation can improve threshold-integrated and threshold-optimized metrics, including AUROC, AUPRC, best-threshold balanced accuracy, and best-threshold \F1 score. We separate the effect of augmentation into two components: a change in effective class weighting and a discrepancy between the synthetic and true minority distributions. Under well-specified score models, the raw estimator already targets the likelihood-ratio ordering, which is population-optimal for the metrics considered. Consequently, augmentation cannot provide a fundamental population-level improvement beyond possible finite-sample variance reduction, and may introduce additional bias through synthetic distributional error. We further establish minimax lower bounds showing that the raw estimator already achieves the optimal metric-regret rate in the well-specified regime. Under misspecification, however, augmentation can play a qualitatively different role: by changing the effective class balance, it can alter the restricted-class projection and correct ranking errors induced by the raw imbalanced objective. We provide explicit improvement bounds quantifying the roles of approximation error, finite-sample estimation error, and synthetic distributional error. Simulation studies corroborate the theory, demonstrating limited gains under well-specification and nontrivial but nonmonotone improvements under misspecification.
Zhengchi Ma, Pengfei Lyu, Anru R. Zhang
Department of Electrical & Computer Engineering, Duke University · Department of Biostatistics & Bioinformatics, Duke University · Department of Biostatistics & Bioinformatics and Department of Computer Science, Duke University
Analyzing the reasoning patterns of students in science classrooms is critical for understanding knowledge construction mechanism and improving instructional practice to maximize cognitive engagement, yet manual coding of classroom discourse at scale remains prohibitively labor-intensive. We present an automated discourse analysis system (ADAS) that jointly classifies teacher and student utterances along two complementary dimensions: Utterance Type and Reasoning Component derived from our prior CDAT framework. To address severe label imbalance among minority classes, we (1) stratify-resplit the annotated corpus, (2) apply LLM-based synthetic data augmentation targeting minority classes, and (3) train a dual-probe head RoBERTa-base classifier. A zero-shot GPT-5.4 baseline achieves macro-F1 of 0.467 on UT and 0.476 on RC, establishing meaningful upper bounds for prompt-only approaches motivating fine-tuning. Beyond classification, we conduct discourse pattern analyses including UTxRC co-occurrence profiling, Cognitive Complexity Index (CCI) computation per session, lag-sequential analysis, and IRF chain analysis, revealing that teacher Feedback-with-Question (Fq) moves are the most consistent antecedents of student inferential reasoning (SR-I). Our results demonstrate that LLM-based augmentation meaningfully improves UT minority-class recognition, and that the structural simplicity of the RC task makes it tractable even for lexical baselines.
Jiho Noh, Mukhesh Raghava Katragadda, Raymond Carl +1
Department of Computer Science, Kennesaw State University, Kennesaw, GA 30144 USA · Bagwell College of Education, Kennesaw State University, Kennesaw, GA 30144 USA
Generative models are widely used to compensate for class imbalance in AI training pipelines, yet their failure modes under low-data conditions are poorly understood. This paper reports a controlled benchmark comparing three augmentation strategies applied to a fine-grained animal classification task: traditional transforms, FastGAN, and Stable Diffusion 1.5 fine-tuned with Low-Rank Adaptation (LoRA). Using the Oxford-IIIT Pet Dataset with eight artificially underrepresented breeds, we find that FastGAN augmentation does not merely underperform at very low training set sizes but actively increases classifier bias, with a statistically significant large effect across three random seeds (bias gap increase: +20.7%, Cohen's d = +5.03, p = 0.013). The effect size here is large enough to give confidence in the direction of the finding despite the small number of seeds. Feature embedding analysis using t-distributed Stochastic Neighbor Embedding reveals that FastGAN images for severe-minority breeds form tight isolated clusters outside the real image distribution, a pattern consistent with mode collapse. Stable Diffusion with Low-Rank Adaptation produced the best results overall, achieving the highest macro F1 (0.9125 plus or minus 0.0047) and a 13.1% reduction in the bias gap relative to the unaugmented baseline. The data suggest a sample-size boundary somewhere between 20 and 50 training images per class below which GAN augmentation becomes harmful in this setting, though further work across additional domains is needed to establish where that boundary sits more precisely. All experiments run on a consumer-grade GPU with 6 to 8 GB of memory, with no cloud compute required.
Shesh Narayan Gupta, Nik Bear Brown
College of Engineering, Northeastern University, Boston, MA 02115, USA