Geometric Filtering of LLM-Generated Samples for Few-Shot Text Classification
Organizations: Faculty of Engineering and Sciences Universidad Adolfo Ibáñez Millennium Nucleus for Social Data Science (SODAS) Santiago, Chile · Faculty of Engineering and Sciences Universidad Adolfo Ibáñez Millennium Nucleus for Social Data Science (SODAS) Millennium Nucleus in Data Science for Plant Resilience (PhytoLearning) Santiago, Chile
Abstract
Large language models (LLMs) can generate synthetic training data for text classification, but the quality of generated samples is heterogeneous: some fall in correct class regions of the embedding space while others land in peripheral or cross-class zones. We propose a geometric filtering framework that evaluates each LLM-generated sample by its Euclidean distance to real class examples in a sentence embedding space, selecting only geometrically consistent candidates. A soft weighting mechanism transforms filter scores into sample weights for classifier training. Evaluated across 13 datasets, 5 classifiers, 10 augmentation methods, and over 6,700 configurations, our method achieves +2.61 percentage points (pp) over SMOTE (, Cohen's , 88.9% win rate). The approach generalizes to named entity recognition (+9.26pp, 100% win rate) without filter modification, and is robust across 5 LLMs from 4 providers. A key finding is that the simplest distance-based filter consistently outperforms complex multi-criteria alternatives.
Figures & tables
| Method | (pp) | 95% CI | Win% | |
|---|---|---|---|---|
| Soft-weighted | +2.61 *** | [+1.98, +3.25] | 0.95 | 88.9 |
| Binary filter | +2.39*** | [+1.75, +3.07] | 0.82 | 79.4 |
| No augmentation | +0.64*** | [+0.42, +0.89] | 0.64 | 73.0 |
| Back-translation | 0.07 | [ 0.42, +0.26] | 0.05 | 57.1 |
| EDA | 0.15 | [ 0.57, +0.27] | 0.08 | 52.4 |
| BERT contextual | 0.26 | [ 0.58, +0.02] | 0.21 | 46.0 |
| Soft-weighted | Binary filter | |||||
| Win | Win | |||||
| 10 | +4.89*** | 1.48 | 90.5 | +4.44*** | 1.15 | 87.3 |
| 25 | +2.17*** | 1.57 | 95.2 | +1.96*** | 1.18 | 90.5 |
| 50 | +0.76*** | 0.80 | 74.6 | +0.77*** | 0.71 | 63.5 |
| *** (paired -test). in pp. | ||||||
| Classifier | (pp) | Cohen’s | Win% | |
| Linear SVC | +2.98*** | 1.02 | 92.1 | 63 |
| Ridge | +2.74*** | 1.04 | 92.1 | 63 |
| MLP | +2.16*** | 0.74 | 88.6 | 105 |
| Logistic Reg. | +2.09*** | 0.82 | 76.2 | 63 |
| Random Forest | +1.42** | 0.35 | 61.9 | 105 |
| ** , *** (Bonferroni-corrected). | ||||
| Corpus | Baseline | No filter | Cascade | LOF | |
|---|---|---|---|---|---|
| MultiNERD | 10 | .247 | .393 | .407 | .426 |
| 25 | .315 | .419 | .438 | .431 | |
| 50 | .415 | .486 | .455 | .495 | |
| WikiANN | 10 | .162 | .268 | .303 | .302 |
| 25 | .253 | .272 | .299 | .276 | |
| 50 | .244 | .295 | .287 | .293 |