Prolonged exposure to hate speech on social media can cause lasting psychological distress among users. Counterspeech has emerged as an effective, non-censorial mitigation strategy, but its deployment requires rigorous quality assessment. Manual assessment is unscalable due to the large volume of content on the Web and imposes severe emotional strain on human annotators. Large Language Models (LLMs) offer a scalable alternative, yet concerns remain regarding their reliability. In this paper, we investigate the extent to which LLMs can effectively perform this task. We test 12 open-weight LLMs of different sizes and families under zero-shot, few-shot, and chain-of-thought prompting, to approximate pointwise and pairwise human expert assessment of counterspeech quality across four dimensions: relevance, coherence, aggressiveness, and suitableness. We find that (i) LLMs show low agreement and moderate rank correlations with human experts: our best configuration, Qwen 2.5 72B on coherence, achieves up to a linear-weighted Cohen's κl=0.19 and Spearman's ρ=0.42 in pointwise evaluation and an unweighted Cohen's κ=0.25 in pairwise evaluation. (ii) Converting pointwise scores into pairwise labels substantially improves human--LLM agreement for selected configurations. For same-instance coherence, Qwen 2.5 72B under zero-shot prompting achieves κ=0.35 with converted labels, compared with κ=0.25 when judging pairs directly, and Mixtral 8×7B under few-shot prompting improves from κ=0.07 to κ=0.34. (iii) LLMs agree more with one another than with human experts, particularly in pairwise evaluation, where the strongest model pair, OLMo 2 32B and Llama 3.3 70B on coherence, reaches substantial agreement with κ=0.76, whereas their agreement with the experts in the same configuration does not exceed κ=0.21.
Figures & tables
Figure 1. Overview of the evaluation settings. In setting (a), a single counterspeech item for one hateful instance is evaluated on a Likert scale. In settings (b) and (c), two items are compared: in (b) on the same hateful instance, while in (c) on two different instances matched by target group and counterspeech strategy.
Dimension
n
1
2
3
4
5
Relevance
1,389
296
811
217
65
0
Coherence
1,397
612
712
57
16
0
Aggressiveness
1,092
91
225
568
194
14
Suitableness
1,386
693
542
151
–
–
Table 1. Expert score distribution in the pointwise evaluation set (1,400 instances). Suitableness uses a 1–3 scale.
Same-instance
Cross-instance
Dimension
n
A
B
Tie
n
A
B
Tie
Relevance
441
104
106
231
6,978
1,884
2,007
3,087
Coherence
450
115
111
224
7,047
1,626
1,703
3,718
Aggressiveness
263
68
66
129
5,963
1,890
1,826
2,247
Suitableness
447
102
104
241
6,890
2,050
1,850
2,990
Table 2. Number of pairwise comparisons per dimension and distribution of reference labels.
Family
Model
Size
Group
Identifier
Llama
Llama 3.1 8B
8B
Small
Llama-3.1-8B-Instruct
Llama 3.3 70B
70B
Large
Llama-3.3-70B-Instruct
OLMo 2
OLMo 2 7B
7B
Small
OLMo-2-1124-7B-Instruct
OLMo 2 32B
32B
Medium
OLMo-2-0325-32B-Instruct
Qwen 2.5
Qwen 2.5 7B
7B
Small
Qwen2.5-7B-Instruct
Qwen 2.5 72B
72B
Large
Qwen2.5-72B-Instruct
Table 3. Evaluated open-weight LLMs. For the Mixtral mixture-of-experts models, size groups are based on active parameters.
Relevance
Coherence
Aggressiveness
Suitableness
Model
Prompt
κl
ρ
κl
ρ
κl
ρ
κl
ρ
OLMo 7B
Zero-shot
.026
.065
.077
.264
.001
.014
.000
–
Few-shot
.060
.048
.058
.201
− .025
− .067
− .098
− .142
CoT
.062
.125
.143
.320
.000
–
.000
–
Qwen 7B
Zero-shot
.070
.122
.161
.378
.064
.307
− .021
− .037
Few-shot
.037
.067
.149
.320
.063
.296
− .056
− .101
Table 4. Pointwise human–LLM agreement. Avg.: group mean; ρ is averaged over models for which it is defined. –: same score for every instance. Bold: highest κl per dimension. Majority: most frequent expert score per dimension. Random (95% UB): upper end of the range covering 95% of the κl values from 2,000 runs of random scoring; higher values are unlikely to arise by chance.
Relevance
Coherence
Aggressiveness
Suitableness
Model
Prompt
κ
κpw
κ
κpw
κ
κpw
κ
κpw
OLMo 7B
Zero-shot
.080
.040
.159
.216
.010
− .003
− .014
.000
Few-shot
.008
.150
− .003
.217
− .002
.069
− .001
− .049
CoT
.091
.086
.099
.264
.056
.000
− .001
.000
Qwen 7B
Zero-shot
.073
.017
.192
.305
.010
.176
− .019
.036
Few-shot
.056
.097
.178
.251
.029
.169
− .030
.036
Table 5. Same-instance comparisons: pairwise human–LLM agreement. Bold: highest κ and κpw per dimension. Avg.: group mean. Majority and Random (95% UB): as in Table 4 .
Relevance
Coherence
Aggressiveness
Suitableness
Model
Prompt
κ
κpw
κ
κpw
κ
κpw
κ
κpw
OLMo 7B
Zero-shot
.029
.012
− .025
.037
.011
− .003
− .011
.000
Few-shot
− .001
− .012
.000
− .046
.002
− .008
.000
.002
CoT
− .004
.045
− .006
.029
.028
.000
− .005
.000
Qwen 7B
Zero-shot
.038
.070
.042
.037
.039
.129
.011
.025
Few-shot
.033
.059
.036
.034
− .016
.119
− .023
.054
Table 6. Cross-instance comparisons: pairwise human–LLM agreement. Bold: highest κ and κpw per dimension. Avg.: group mean. Majority and Random (95% UB): as in Table 4 .
Relevance
Coherence
Aggressiveness
Suitableness
Pointwise
n
κl
ρ
n
κl
ρ
n
κl
ρ
n
κl
ρ
All
1389
.024
.198
1397
.136
.368
1092
.042
.273
1386
.031
.053
Muslims
460
.013
.148
463
.132
.363
393
.012
.099
459
.057
.112
Women
224
.023
.207
226
.083
.250
168
.069
.418
225
− .025
− .049
Black people
173
.030
.126
176
.149
.491
89
.069
.264
175
− .024
− .042
LGBTQ+
151
.056
.354
151
.165
.433
113
.065
.344
149
.049
.073
Table 8. Human–LLM agreement of OLMo 2 32B per target group under zero-shot prompting. n : instances or pairs annotated for the dimension. Bold: highest κl or κ per dimension and setting.
Table 10. Model size within families: configurations in which the larger model has higher/lower κ , with Benjamini–Hochberg adjusted p -values in parentheses. PW: pointwise.
Large language model (LLM)-as-a-judge research uses language models to score, compare, and verify AI outputs as scalable alternatives to human evaluation.
Hate speech annotation is costly, subjective, and prone to annotator disagreement, making large-scale dataset construction challenging. We systematically analyze how well large language models (LLMs) align with human judgments across ten theoretically grounded subjective attributes, such as dehumanization, violence, and sentiment, evaluating both small and large variants of Llama 3.1 and Qwen 2.5. Our analysis reveals a consistent split across all models: behaviorally explicit dimensions (insult, humiliate, attack-defend) correlate strongly with human annotations, while evaluative dimensions (respect, sentiment, hate speech) are systematically inverted. Demographic persona conditioning reduces model confidence without improving alignment. Building on these insights, we propose combining attribute-level LLM predictions via a confidence-weighted Ridge regression to reconstruct continuous hate speech scores from the Measuring Hate Speech corpus, achieving R2 of up to 0.71 and outperforming direct prompting baselines, demonstrating that structured attribute decomposition recovers a richer and more human-aligned signal than end-to-end label prediction alone.
Mohammad Amine Jradi, Faeze Ghorbanpour, Alexander Fraser
School of Computation, Information and Technology, TU Munich Munich Center for Machine Learning (MCML)
Hate speech and misinformation frequently co-occur online, amplifying prejudice and polarization. Given their scale, using Large Language Models (LLMs) to assist expert counterspeech (CS) writing has gained interest, yet prior work has addressed these phenomena separately. We bridge this gap by studying CS generation in contexts where both hate and misinformation co-occur. We test three knowledge-driven generation strategies: first we prompt an LLM with fact-checkers' guidelines and fact-checking articles; secondly, with NGOs' guidelines and reports; thirdly, we create a mixed strategy that combines guidelines and documents from both. 23 experts revise the generated CS, which are assessed via human and automatic metrics. While LLMs produce adequate CS in 40% of cases, expert edits substantially improve naturalness, exhaustiveness, and adherence to guidelines. Based on the post-edited CS, the mixed strategy proves to be the most effective in crowdsourcing evaluation, pairing strong factual correction with stereotype mitigation and empathetic engagement. We release a dataset of hateful and misinformed claims with expert-verified CS and supporting knowledge.
Genoveffa Martone, Helena Bonaldi, Marco Guerini
Fondazione Bruno Kessler, Italy · Università Cattolica del Sacro Cuore, Italy
A Large Language Model (LLM) as judge evaluates the quality of victim Machine Learning (ML) models, specifically LLMs, by analyzing their outputs. An LLM as judge is the combination of one model and one specifically engineered judge prompt that contains the criteria for the analysis. The resulting automation of the analysis scales up the complex evaluation of the victim models' free-form text outputs by faster and more consistent judgments compared to human reviewers. Thus, quality and security assessments of LLMs can cover a wide range of the victim models' use cases. Being a comparably new technique, LLMs as judges lack a thorough investigation for their reliability and agreement to human judgment. Our work evaluates the applicability of LLMs as automated quality assessors of victim LLMs. We test the efficacy of 37 differently sized conversational LLMs in combination with 5 different judge prompts, the concept of a second-level judge, and 5 models fine-tuned for the task as assessors. As assessment objective, we curate datasets for eight different categories of judgment tasks and the corresponding ground-truth labels based on human assessments. Our empirical results show a high correlation of LLMs as judges with human assessments, when combined with a suitable prompt, in particular for GPT-4o, several open-source models with ⩾ 32B parameters, and a few smaller models like Qwen2.5 14B.
Tom Biskupski, Stephan Kleber
Hochschule der Medien Stuttgart, Germany · Ravensburg-Weingarten University of Applied Sciences Weingarten, Germany