Not All Pretraining are Created Equal: Threshold Tuning and Class Weighting for Imbalanced Polarization Tasks in Low-Resource Settings
Organizations: African Institute of Mathematical Sciences, South Africa
Abstract
This paper describes my submission to the Polarization Shared Task at SemEval-2025, which addresses polarization detection and classification in social media text. I develop Transformer-based systems for English and Swahili across three subtasks: binary polarization detection, multi-label target type classification, and multi-label manifestation identification. The approach leverages multilingual and African language-specialized models (mDeBERTa-v3-base, SwahBERT, AfriBERTa-large), class-weighted loss functions, iterative stratified data splitting, and per-label threshold tuning to handle severe class imbalance. The best configuration, mDeBERTa-v3-base, achieves 0.8032 macro-F1 on validation for binary detection, with competitive performance on multi-label tasks (up to 0.556 macro-F1). Error analysis reveals persistent challenges with implicit polarization, code-switching, and distinguishing heated political discourse from genuine polarization.
Figures & tables
| Model | English | Swahili |
|---|---|---|
| TwHIN-BERT | 0.7279 | 0.7048 |
| DistilBERT-multilingual (Baseline) | 0.7279 * | 0.8022 * |
| mDeBERTa-v3-base | 0.8032 | 0.7850 |
| SwahBERT | 0.6375 | 0.8050 |
| AfriBERTa-large | 0.7333 | 0.8075 |
| AfroXLMR-large | 0.5317 | 0.5788 |
| Model | Before | After |
|---|---|---|
| English | ||
| DistilBERT-multilingual | 0.245 | 0.464 |
| AfriBERTa-large | 0.330 | 0.437 |
| Swahili | ||
| AfriBERTa-large | 0.132 | 0.556 |
| SwahBERT | 0.210 | 0.325 |
Multi-Label Classification
Multi-label classification assigns several potentially related tags to each example, from medical findings and topics to objects and user interests.
See all 137 papersYEZE at SemEval-2026 Task 9: Detecting Multilingual, Multicultural and Multievent Online Polarization via Heterogeneous Ensembling
Polarization Detection: A Hybrid Approach with AfroXLMR-Social and DeBERTa for Low- and High-Resource Settings
nlpaug. We report competitive results across all three subtasks, demonstrating that model selection tailored to specific subtask requirements yields the best balance of performance.