Enhancing Sentiment Classification on Small Datasets through Data Augmentation and Transfer Learning
Organizations: National Conservatory of Arts and Crafts (CNAM), Paris, France · An-Najah National University, Nablus, Palestine
Abstract
Small-scale sentiment classification often suffers from data scarcity, which limits the generalization ability of the models. This study evaluates and compares the effectiveness of three data augmentation strategies: Easy Data Augmentation (EDA), back-translation, and contextual token substitution (nlpaug-style), with both traditional machine learning classifiers (Logistic Regression, Random Forest) and transformer-based models (BERT). We perform a comprehensive empirical comparison with low-resource sentiment datasets by summarizing the results of recent studies and performing targeted head-to-head experiments. Our findings indicate that all augmentation methods improve performance. Contextual augmentation yields the most consistent gains for BERT models, while EDA and back-translation provide greater benefits for traditional classifiers. These insights help guide the selection of data augmentation techniques tailored to model type and dataset size, filling a critical gap in research on data augmentation for sentiment classification on small datasets.
Figures & tables
| Study | Augmentation | Datasets | Models | Key findings | Year |
|---|---|---|---|---|---|
| Wei & Zou [ 10 ] | EDA | SST-2, IMDb | CNN, LSTM | Simple token edits improve small-data accuracy at low compute cost. | 2019 |
| Shleifer [ 7 ] | Back-translation | IMDb (50–500) | ULMFit | Back-translation adds paraphrases; +8% accuracy in extreme low-resource IMDb. | 2019 |
| Abonizio et al. [ 1 ] | EDA, BT | Twitter, Yelp | SVM, BERT | EDA+BT raise F1 on reviews, combining lexical and semantic gains. | 2022 |
| Duong et al. [ 9 ] | Lexical + BT | Vietnamese SA | BiLSTM, CNN | Lexical+BT improves Vietnamese sentiment; confirms cross-lingual effectiveness. | 2022 |
| Pietri & Stanley [ 6 ] | EDA, Synonym swap | Social media | BERT | Random synonym swaps unstable for transformers; effects vary by corpus. | 2023 |
| Pingle et al. [ 2 ] | Contextual, Paraphr., BT | Marathi SA | BERT, LSTM | Contextual/paraphrase augmentation outperforms EDA on low-resource Marathi. | 2023 |
| Augmentation | Mean similarity | Std. deviation |
|---|---|---|
| EDA | 0.944 | 0.035 |
| NLPaug | 0.950 | 0.045 |
| Back-translation | 0.9999 | 0.0036 |
| Variant | Example 4003 (spoof film review) |
|---|---|
| Original | A series of shorts spoofing dumb TV shows, Groove Tube hits and misses a lot. Overall, I do really like this movie. |
| EDA | A series of shorts spoofing dumb TV shows, Groove Tube hits misses a lot. Overall, I do really, really like this movie. |
| NLPaug | Series a shorts of spoofing dumb TV shows, Groove hits Tube and misses a lot Overall. I really do like this movie. |
| Back-translation | A series of shorts spoofing dumb TV shows, Groove Tube hits and misses a lot. Overall, I do really like this movie. |
| Variant | Example 2968 (humorous sci-fi review) |
| Original | If you have trouble dreaming, you may give this movie a low rating. But you just have to realize this movie was not made to please everyone, just people with a sense of humor. |
| Component | Setting |
|---|---|
| BERT Model Configuration | |
| Model | bert-base-uncased (110M parameters) |
| Optimizer | AdamW |
| Learning Rate | |
| Batch Size (Train) | 8 |
| Batch Size (Eval) | 32 |
| Augmentation | Model | Accuracy [CI] | F1 [CI] | AUC [CI] | Train Time (s) |
|---|---|---|---|---|---|
| Original | Logistic Regression | 0.8540 [0.845, 0.863] | 0.8564 [0.848, 0.865] | 0.9317 [0.924, 0.940] | 8.15 |
| Original + EDA | Logistic Regression | 0.9058 [0.899, 0.913] | 0.9064 [0.899, 0.914] | 0.9665 [0.962, 0.971] | 13.54 |
| EDA | Logistic Regression | 0.8472 [0.834, 0.860] | 0.8495 [0.837, 0.862] | 0.9235 [0.914, 0.933] | 7.32 |
| Original + NLPaug | Logistic Regression | 0.9135 [0.908, 0.919] | 0.9142 [0.908, 0.920] | 0.9701 [0.965, 0.975] | 14.00 |
| NLPaug | Logistic Regression | 0.8540 [0.845, 0.863] | 0.8564 [0.848, 0.865] | 0.9317 [0.924, 0.940] | 7.92 |
| Original + Back-translation | Logistic Regression | 0.9134 [0.908, 0.919] | 0.9140 [0.909, 0.920] | 0.9703 [0.965, 0.975] | 14.38 |
| Model | Augmentation | Baseline Acc | Augmented Acc | Cohen’s / |
|---|---|---|---|---|
| Logistic Regression | Original + EDA | 0.8540 | 0.9058 | |
| Logistic Regression | Original + NLPaug | 0.8540 | 0.9135 | |
| Logistic Regression | Original + Back-translation | 0.8540 | 0.9134 | |
| Random Forest | Original + EDA | 0.8300 | 0.9796 | |
| Random Forest | Original + NLPaug | 0.8300 | 0.9834 | |
| Random Forest | Original + Back-translation | 0.8300 | 0.9828 |
| Setting | Seed | Acc | F1 | AUC |
|---|---|---|---|---|
| Original | 12 | 0.898 | 0.898 | 0.958 |
| Original | 44 | 0.886 | 0.883 | 0.953 |
| Original | 127 | 0.886 | 0.889 | 0.929 |
| Orig+NLPaug | 12 | 0.975 | 0.975 | 0.992 |
| Orig+NLPaug | 44 | 0.963 | 0.964 | 0.995 |
| Orig+NLPaug | 127 | 0.974 | 0.974 | 0.984 |
| Model | TN | FP | FN | TP |
|---|---|---|---|---|
| Logistic Regression | 919 | 87 | 68 | 926 |
| Random Forest | 988 | 18 | 36 | 958 |
| BERT (seed 12) | 493 | 10 | 15 | 482 |
| BERT (seed 44) | 470 | 33 | 4 | 493 |
| BERT (seed 127) | 489 | 14 | 12 | 485 |
| Model | Dataset | Accuracy |
|---|---|---|
| Logistic Regression | Original | 0.854 |
| Logistic Regression | Original + NLPaug | 0.914 |
| Random Forest | Original | 0.830 |
| Random Forest | Original + NLPaug | 0.983 |
| BERT | Original | 0.890 |
| BERT | Original + NLPaug | 0.971 |
| Method | Aug. Time | LR | RF | BERT |
|---|---|---|---|---|
| Original (baseline) | – | 8.15s | 62.17s | 548s |
| EDA | 2s | 13.54s | 121.38s | – |
| NLPaug | 45s | 14.00s | 111.99s | 1,078s |
| Back-Translation | 250s | 14.38s | 115.96s | – |
Data Augmentation
Data augmentation expands or transforms training examples to improve learning, data efficiency, and robustness to distribution shifts.
See all 273 papers