This paper evaluates data augmentation and feature enhancement techniques for hate speech detection, comparing traditional classifiers, e.g., Delta Term Frequency-Inverse Document Frequency (Delta TF-IDF), with transformer-based models (DistilBERT, RoBERTa, DeBERTa, Gemma-7B, gpt-oss-20b) across diverse datasets. It examines the impact of Synthetic Minority Over-sampling Technique (SMOTE), weighted loss determined by inverse class proportions, Part-of-Speech (POS) tagging, and text data augmentation on model performance. The open-source gpt-oss-20b consistently achieves the highest results. On the other hand, Delta TF-IDF responds strongly to data augmentation, reaching 98.2% accuracy on the Stormfront dataset. The study confirms that implicit hate speech is more difficult to detect than explicit hateful content and that enhancement effectiveness depends on dataset, model, and technique interaction. Our research informs the development of hate speech detection by highlighting how dataset properties, model architectures, and enhancement strategies interact, supporting more accurate and context-aware automated detection.
Figures & tables
Fig. 1: Training & evaluation process for each dataset.
Dataset
Metric
Delta TF-IDF
DistilBERT
DeBERTaV3
RoBERTa
gpt-oss-20b
Gemma-7B
Hate Corpus
Accuracy
0.655 ( 0.566 )
0.694 ( 0.475 )
0.712 ( 0.507 )
0.738 ( 0.488 )
0.757 ( 0.695 )
0.728 ( 0.596 )
Macro F1
0.412 ( 0.382 )
0.444 ( 0.369 )
0.487 ( 0.418 )
0.480 ( 0.396 )
0.483 ( 0.566 )
0.490 ( 0.540 )
AUC
0.312 ( 0.345 )
0.400 ( 0.336 )
0.524 ( 0.456 )
0.561 ( 0.401 )
0.518 ( 0.564 )
0.592 ( 0.597 )
Gab Reddit
Accuracy
0.889 ( 0.890 )
0.908 ( 0.907 )
0.909 ( 0.907 )
0.910 ( 0.909 )
0.919 ( 0.916 )
0.856 ( 0.859 )
Macro F1
0.874 ( 0.876 )
0.900 ( 0.899 )
0.903 ( 0.901 )
0.902 (0.902)
0.912 ( 0.909 )
0.842 ( 0.849 )
AUC
0.918 ( 0.914 )
0.925 ( 0.943 )
0.952 ( 0.954 )
0.949 ( 0.945 )
0.911 (0.911)
0.923 ( 0.924 )
TABLE I: SMOTE & Weighted Loss Configuration Results
The proliferation of hate speech on social media platforms has necessitated the development of effective detection and moderation tools. This study evaluates the efficacy of various machine learning models in identifying hate speech and offensive language and investigates the potential of text transformation techniques to neutralize such content. We compare traditional models like CNNs and LSTMs with advanced neural network models such as BERT and its derivatives, alongside exploring hybrid models that combine different architectural features. Our results indicate that while advanced models like BERT show superior accuracy due to their deep contextual understanding, hybrid models exhibit improved capabilities in certain scenarios. Furthermore, we introduce innovative text transformation approaches that convert negative expressions into neutral ones, thereby potentially mitigating the impact of harmful content. The implications of these findings are discussed, highlighting the strengths and limitations of current technologies and proposing future directions for more robust hate speech detection systems.
Saurabh Mishra, Shivani Thakur, Radhika Mamidi
International Institute of Information Technology - Hyderabad Professor CR Rao Rd, Gachibowli Hyderabad 500032, Telangana, India
This paper investigates the use of an LLM to generate auxiliary background context for social media posts, and explores four methods to incorporate this context into the input of an SBERT-based Hate Speech Detection (HSD) classifier. These are: text concatenation, embedding concatenation, a hierarchical transformer-based fusion, and LLM-driven text enhancement. We evaluate the impact of our context generation and incorporation strategies in a textual setting on the Latent Hatred dataset of implicitly hateful tweets and a multimodal setting on the MAMI dataset of misogynous internet memes. Results are evaluated against a zero-context baseline, two previous approaches based on entity linking, and a zero-shot LLM classifier. Findings indicate that incorporating generated context improves HSD performance by up to 3 and 6 F1 points on textual and multimodal settings respectively, from a zero-context baseline to the highest-performing system, based on embedding concatenation.
Joshua Wolfe Brook, Ilia Markov
Computational Linguistics & Text Mining Lab, Vrije Universiteit Amsterdam
Standard hate speech detection datasets often emphasize explicit harmful language, but directed harm can also be expressed indirectly, without clearly hateful wording. Sarcasm is one such form of indirect expression, as it can convey harmful intent through mock praise, exaggeration, humor, or apparent politeness. In this paper, we introduce SarcasTraps, a targeted external test set for analyzing the effect of sarcasm on Turkish hate speech detection. SarcasTraps contains 479 entries annotated along two dimensions: hate speech and sarcasm. We train classifiers using BERTurk and XLM-RoBERTa on two existing Turkish hate speech datasets and evaluate them on SarcasTraps, using it strictly as an external test set. We also evaluate GPT-4.1 and Llama-3.3-70B in a zero-shot setting on SarcasTraps. Results show that sarcasm substantially affects hate speech detection. Fine-tuned encoder models lose recall on sarcastic hate, while zero-shot LLMs maintain higher recall but suffer substantial precision drops on sarcastic examples. These findings show that both fine-tuned hate speech detectors and zero-shot LLMs struggle when sarcasm is involved, and SarcasTraps provides quantitative evidence of this challenge through a targeted Turkish test set. Due to the presence of sensitive and hateful language, SarcasTraps will be shared upon request for academic research purposes. Warning: This paper has examples from the dataset which contains offensive and hateful content towards groups of people.