HateSense: Hierarchical Sentiment Analysis Framework for Hate Speech Detection: Implementing Binary and Multiclass Classification Strategy

Faria Naznin; Md Touhidur Rahman; Shahran Rahman Alve

cs.CLNov 3, 2024

HateSense: Hierarchical Sentiment Analysis Framework for Hate Speech Detection: Implementing Binary and Multiclass Classification Strategy

Authors: Faria Naznin, Md Touhidur Rahman, Shahran Rahman Alve

Organizations: Department of Electrical & Computer & Engineering, North South University, Dhaka, Bangladesh

Abstract

Hate speech detectors must separate hateful posts from the far more common offensive but non-hateful ones, and two popular ideas for doing so are hierarchical (coarse-to-fine) classification and sentiment features. We revisit our earlier transformer-based framework, which combines both ideas, and test each component in a controlled study on two public benchmarks (Davidson et al., 24,783 tweets; HateXplain, 19,229 posts) with five random splits, validation-based tuning, and paired bootstrap tests. With linear models over word and character n-grams, the dominant-class-first hierarchy used in our framework lowered macro-F1 by 1.1-1.5 points relative to a flat classifier (p <= 0.007), and adding VADER sentiment features changed Davidson macro-F1 by at most 0.6 points, never significantly in the class-weighted settings. Class-weighted training was the only intervention with a large effect (+2.0 to +3.5 macro-F1 and +5.9 to +9.1 hate-class F1), and sentiment cannot separate the two abusive classes because hateful and offensive tweets are almost equally negative (mean VADER compound -0.35 vs. -0.29). Cross-dataset transfer cut macro-F1 to 43-53, and a reclaimed slur doubled the false hate rate of a logistic-regression detector, showing that in-domain scores overstate real-world reliability.

Figures & tables

Panel of Figure 1 — Figure 1: The two hierarchical decompositions. (a) is the merge-then-split design of our earlier framework, in which stage 2 never sees an offensive example. (b) makes the hate/offensive decision only on posts already judged abusive.

				Davidson (5 splits)							HateXplain
	Wt.	Decomposition	Sent.	Macro-F1	Hate P	Hate R	Hate F1	Off. F1	Neither F1	o $\to$ h	Macro-F1	Hate F1	Off. F1	Normal F1
LogReg	–	Flat	–	71.7 $\pm$ 2.0	54.7	26.6	35.7 $\pm$ 6.8	94.2	85.3	1.5	65.9	75.7	48.6	73.5
	–	Flat	✓	71.7 $\pm$ 2.2	53.0	26.4	35.2 $\pm$ 7.8	94.2	85.8	1.6	66.0	75.7	48.6	73.6
	–	Dom.-first (v1)	–	69.9 $\pm$ 2.1	49.0	26.3	33.9 $\pm$ 6.4	93.5	82.2	1.9	63.1	74.5	42.2	72.5
	–	Dom.-first (v1)	✓	70.2 $\pm$ 1.1	47.9	27.7	34.9 $\pm$ 5.2	93.4	82.4	2.0	63.2	75.3	41.9	72.6
	–	Abus.-first	–	71.9 $\pm$ 2.1	57.3	25.3	35.0 $\pm$ 6.9	94.3	86.3	1.3	64.9	72.1	52.9	69.6

Table 1: All 24 configurations on both datasets (%). Davidson: mean

\pm

std over five seeded splits; o

\to

h is the share of offensive tweets misclassified as hate. HateXplain: official test split. “Wt.” marks class-weighted training; “Sent.” marks VADER features. The lexicon row flags any tweet containing a hate-lexicon n-gram.

Stage 1: binary (label 1 vs. merged label 0)
Model	Class	P	R	F1	Acc.
RoBERTa-base	0	0.67	0.45	0.54	87%
	1	0.89	0.95	0.92	73%
XLM-RoBERTa	0	0.69	0.48	0.57	88%
	1	0.91	0.97	0.94	75%
Stage 2: multiclass (labels 2, 3, 4)

Table 2: Results reported in the earlier version of this work ( Naznin et al., 2024 ) , reproduced unchanged. F1 is derived here from the reported precision (P) and recall (R). “Acc.” is the per-class accuracy as reported; its definition and the class names of labels 0–4 were not specified in that version. Stage-2 precision and recall were reported in percent and are shown here as fractions.

Figure 2: Effect of each design choice on the pooled Davidson test predictions (paired bootstrap, 2,000 resamples, 95% intervals). Hierarchy and sentiment rows compare against the flat, sentiment-free model; all models except the weighting row use class-weighted training.

Figure 3: Row-normalised confusion matrices (%) for class-weighted LogReg without sentiment features. Davidson: pooled over the five test splits; HateXplain: official test split. Rows are true classes.

Figure 4: Distribution of VADER compound scores by class on Davidson. Hateful and offensive tweets have almost the same sentiment profile; only the neither class differs.

Class	Mean compound	Mean neg. score	Share negative
Hate	$-0.35$	0.27	66.8%
Offensive	$-0.29$	0.25	65.1%
Neither	$+0.09$	0.07	24.9%

Table 3: VADER sentiment by Davidson class. Negative: compound

<-0.05

	Decomp.	Sent.	Macro-F1	Hate P	Hate R	Hate F1	Off. F1	Neither F1
D $\to$ H	Flat	–	45.1	45.9	56.7	50.7	35.4	49.3
D $\to$ H	Flat	✓	44.6	46.4	52.0	49.0	38.5	46.3
D $\to$ H	Dom.-first (v1)	–	44.4	42.9	57.6	49.2	36.0	47.9
D $\to$ H	Dom.-first (v1)	✓	42.3	40.4	57.9	47.6	33.3	46.1
D $\to$ H	Abus.-first	–	43.8	47.3	50.0	48.6	36.8	45.9
D $\to$ H	Abus.-first	✓	43.4	45.4	52.9	48.9	34.9	46.4

Table 4: Cross-dataset transfer (%), LogReg with class weighting. D

\to

H: train on Davidson, test on HateXplain; H

\to

D: the reverse (seed-0 Davidson test split). In-domain flat macro-F1 is 74.5 (Davidson) and 66.2 (HateXplain).

Model	With term	Without term	Ratio
	( $n=916$ )	( $n=10{,}764$ )
LogReg, flat	9.0	3.9	2.3 $\times$
LogReg, dominant-first (v1)	8.2	4.5	1.8 $\times$
Linear SVM, flat	4.9	3.1	1.6 $\times$
Linear SVM, flat + sentiment	4.3	3.0	1.4 $\times$

Table 5: Lexical bias probe on Davidson: share (%) of non-hateful test tweets classified as hate, with and without the reclaimed slur, pooled over five test splits (class-weighted models).

Hate Speech Detection

Hate Speech Detection identifies hateful or abusive language, its targets and intent, to support safer and more accountable content moderation.

See all 100 papers

An Explainable DistilBERT-BiLSTM-Attention Framework for Binary and Multi-Class Hate Speech Detection
Sep 23, 2026Rameesha Zia, Muhammad Shahid Iqbal MalikExplainable Text Classification Text Classification
School of Computing Sciences, Pak-Austria Fachhochschule, Institute of Applied Sciences and Technology, Haripur 22620, Pakistan
Hate Speech Detection using Large Language Models with Data Augmentation and Feature Enhancement
Mar 5, 2026Brian Jing Hong Nge, Stefan Su, Thanh Thi Nguyen +3Hate Speech Detection Data Augmentation
AiLECS Lab, Monash University, Melbourne, Australia · School of Social Sciences, Monash University, Melbourne, Australia
SarcasTraps: Effect of Sarcasm on Turkish Hate Speech Detection
Oct 12, 2026Niyazi Ahmet Metin, Dilara KeküllüoğluHate Speech Detection
Sabanci University Istanbul, Türkiye

Sep 23, 2026J/K move · Enter open · S save

An Explainable DistilBERT-BiLSTM-Attention Framework for Binary and Multi-Class Hate Speech Detection

Rameesha Zia, Muhammad Shahid Iqbal Malik

School of Computing Sciences, Pak-Austria Fachhochschule, Institute of Applied Sciences and Technology, Haripur 22620, Pakistan

Explainable Text Classification Text Classification Hate Speech Detection

Open paper AskPDF arXiv

Hate speech on social media poses serious risks to social harmony, mental well-being, and public safety, making its timely and accurate detection essential for content moderation systems. Most existing studies focus on binary classification, evaluated their frameworks on a single dataset, and provide limited insight into how decisions are made, which limits their real-world applicability. In addition, limited work is done on the explainability of their predictive inference. To address these challenges, this study proposes a multilevel and explainable hate speech detection framework. The proposed model integrates DistilBERT (Distilled Bidirectional Encoder Representations from Transformers) embeddings with a Bi-LSTM (Bidirectional Long Short-Term Memory) model, and an attention mechanism to capture both contextual meaning and sequential dependencies in text. To enhance trust and transparency, LIME (Local Interpretable Model-agnostic Explanations) is employed to explain model predictions by highlighting influential textual features. The framework is evaluated on two benchmark datasets using both binary and multi-class classification to examine robustness and generalization. In addition, an ablation study is presented to highlight the significance of various components of proposed framework. For binary classification, the proposed model achieves F1-scores of 96.78% on the Davidson dataset and 99.53% on the SMHS dataset. In the multi-class setting, it attains F1-scores of 97.00% and 94.99% on the Davidson and SMHS datasets, respectively, outperforming existing baseline approaches. The results demonstrate that multilevel evaluation improves the reliability that the proposed framework effectively balances performance and efficiency. This makes the framework suitable for practical hate speech moderation systems that require accurate, generalizable, and explainable decisions.

Abstract

Hate Speech Detection

Explore similar work

An Explainable DistilBERT-BiLSTM-Attention Framework for Binary and Multi-Class Hate Speech Detection

Hate Speech Detection using Large Language Models with Data Augmentation and Feature Enhancement

SarcasTraps: Effect of Sarcasm on Turkish Hate Speech Detection