Hate speech has become increasingly prevalent on social media, raising concerns about its detection, interpretation, and moderation. Persian, a relatively low-resource language, lacks sufficient annotated data and computational resources for fine-grained hate speech analysis. In this study, we introduce a fine-grained, multi-hop benchmark for explainable Persian hate speech analysis, covering content triage, target identification, hate tactic and moral foundation analysis, textual evidence extraction, harmful narrative inference, and severity assessment. By evaluating state-of-the-art large language models (LLMs), we find that performance generally declines as tasks require more fine-grained semantic and inferential reasoning. Proprietary models generally outperform open-weight models, while reasoning-oriented models show no consistent advantage. Incorporating intermediate reasoning steps substantially improves severity prediction compared with one-shot assessment. Our analysis further reveals that Incitement to Violence or Harm is most strongly associated with higher severity, while political-affiliation targets are particularly associated with subversion-based moral framing. Overall, this work provides insights into LLM-based hate speech analysis and supports more accountable moderation of Persian-language social media. The benchmark is publicly available at https://github.com/kalhorghazal/From-Triage-to-Impact.
Figures & tables
Figure 1: Pipeline illustrating multi-hop reasoning steps, human verification, and LLM evaluation.
Hop
Definition
Output Schema
Initial Content Triage
Determines whether the content is harmful and identifies its primary category.
Labels: Offensive Language; Hate Speech; Dangerous Speech; Not Harmful.
Target Group Identification
Identifies the explicit or implicit target of the speech and the basis of identity targeted by the content.
Targeted Group (multi-label); Basis of Identity (multi-label). Labels provided in Table 2 .
Hate Tactic Identification
Analyzes the rhetorical strategy used to express or legitimize hostility toward the target.
Multi-label: Slurs/Derogatory Language; Dehumanization; Harmful Stereotyping; Incitement to Violence or Harm; Calls for Discrimination or Exclusion; Conspiracy Theories.
Moral Foundation Analysis
Identifies moral foundations invoked or violated to justify hostility, grounded in Moral Foundations Theory.
Extracts explicit textual evidence supporting the identified targets, tactics, and moral foundations.
Target Span; Tactic Span; Moral Span.
Harmful Narrative Inference
Infers the implicit ideological or toxic narrative promoted by the content.
Concise free-text narrative description.
Table 1: Definitions of the reasoning hops and their corresponding output schemas.
Category
Standard Values
Targeted Group
Iranian Officials & Politicians; Iranian Security & Military Forces; Domestic Political Opponents; Clerics & Religious Leaders; Human Rights, Media & Academic Activists; Migrants & Ethnic/Religious Minorities; Foreign Adversaries; Iranian Women; Iranian Men; Artists, Actors & Celebrities; Students, Children & Adolescents; Companies, Organizations, & Their Employees
Basis of Identity
Ethnicity/Nationality; Religion/Belief; Social Class; Gender; Political Affiliation; Migration Status; Occupation/Profession; Organizational/Institutional Affiliation; Group Membership; Social/Family Role; Personal Identity; Mental/Behavioral Traits; Educational/Academic Background; Humanity/Personhood
Table 2: Categories for targeted groups and identity bases used in the study.
Metric
Grok 4.6
GPT-5.2
o3
Sonnet 4.6
DeepSeek R1
DeepSeek V3
Qwen3 Think
Qwen3 Inst
Hop 1: Initial Content Triage
Macro Precision
82.7
82.6
79.0
79.5
77.0
76.7
67.8
69.6
Macro Recall
83.0
80.4
77.7
79.2
75.0
68.6
66.7
52.0
Macro F1
82.8
79.7
77.8
79.2
75.0
68.5
66.4
48.0
Hop 2: Targeted Group
Micro F1
71.1
66.0
68.1
66.7
60.1
58.3
42.7
43.0
Table 3: LLM performance across Hops 1–6 of the multi-hop Persian hate-speech reasoning benchmark. All metrics are reported on a 0–100 scale, with task-appropriate evaluation metrics used for each hop. Higher values indicate better performance.
Metric
Grok 4.6
GPT-5.2
o3
Sonnet 4.6
DeepSeek R1
DeepSeek V3
Qwen3 Think
Qwen3 Inst
Severity Level
Macro Precision
51.5
51.6
45.5
52.0
45.1
49.3
42.9
37.0
Macro Recall
53.1
51.7
47.7
40.5
34.4
43.0
31.9
33.5
Macro F1
50.7
50.8
44.6
43.5
32.1
43.9
27.9
29.9
Weighted Cohen’s κ
.829
.819
.760
.797
.558
.736
.525
.525
Severity Level (Ablation)
Table 4: LLM performance on hate speech severity (multi-hop and one-shot) and potential impact. All metrics are reported on a 0–100 scale, except for weighted Cohen’s κ .
Figure 2: Conditional probability distribution of identity bases given the presence of each moral foundation.
Figure 3: Conditional probability distribution of severity levels given the presence of each hate speech tactic.
Figure 4: Recall of hate speech detection across targeted groups for each LLM.
Appendix figures & tables11 assets
Supplementary material from the paper’s appendix.
Appendix
Data Source
Platform
No. in Benchmark
Kebriaei et al.
X
146
Sabri et al.
X
25
Elahimanesh et al.
X
46
Davardoust et al.
Instagram
81
Nazarizadeh et al.
Instagram
50
Appendix
Table 5: Statistics of datasets used to construct our benchmark, including source, platform, and instance counts.
Hop / Annotation Dimension
IAA Results
Hop 1: Initial Content Triage
Cohen’s κ : .743
Observed Agreement: 81.0
Hop 2: Targeted Group
Micro F1: 82.0
Macro F1: 69.3
Hop 2: Basis of Identity
Micro F1: 66.3
Macro F1: 51.2
Appendix
Table 6: Inter-annotator agreement results across annotation hops.
Hop / Annotation Dimension
Human–LLM Alignment Results
Hop 1: Initial Content Triage
Macro Precision: 89.0
Macro Recall: 90.3
Macro F1: 88.8
Hop 2: Targeted Group
Micro F1: 94.0
Macro F1: 94.3
Hop 2: Basis of Identity
Micro F1: 88.2
Appendix
Table 7: Human–LLM alignment results across annotation hops.
Figure 5: Prompt template for detecting hate speech keywords from a given text, along with its English translation.
Figure 6: Prompt template for detecting offensive language keywords from a given text, along with its English translation.
Figure 7: Prompt template for detecting dangerous speech keywords from a given text, along with its English translation.
Figure 8: Prompt template for single-step severity classification of Persian hate speech in the ablation study, along with its English translation.
Figure 9: Prompt template for multi-hop analytical annotation of Persian hate speech.
Figure 10: English translation of the prompt template for multi-hop analytical annotation of Persian hate speech.
Hate Speech Tactic
χ2
df
padj
Cramer’s V
Slurs/Derogatory Language
118.79
3
<.001
.363
Dehumanization
63.48
3
<.001
.266
Harmful Stereotyping
71.53
3
<.001
.282
Calls for Discrimination or Exclusion
94.83
3
<.001
.325
Incitement to Violence or Harm
645.77
3
<.001
.847
Conspiracy Theories
6.96
3
.073
.088
Appendix
Table 8: Associations between hate speech tactics and severity levels. p -values are adjusted using the Benjamini–Hochberg procedure; Cramer’s V reports effect sizes.
Moral Foundation
Identity Basis
pBH
Cramér’s V
Betrayal
Political Affiliation
<.001
.271
Harm
Personal Identity
<.001
.281
Harm
Organizational/Institutional Affiliation
<.001
.273
Degradation
Religion/Belief
<.001
.209
Subversion
Organizational/Institutional Affiliation
<.001
.199
Degradation
Organizational/Institutional Affiliation
<.001
.194
Appendix
Table 9: Significant associations between moral foundations and identity bases. Associations are tested using χ2 tests of independence based on instance-level presence/absence indicators. p -values are corrected using the Benjamini–Hochberg procedure. Cramér’s V is reported as the effect size.
Department of Computer Sciences Information Technology University Lahore (54000), Pakistan · Artificial Intelligence Technology Centre National Centre for Physics Islamabad (44000), Pakistan