Nürnberg NLP @ GermEval Shared Task 2026: Harmful Content Detection in German Social Media through Error-Independent LLM Voters

Philipp Steigerwald; Eric Rudolph; Jens Albrecht

cs.CLAug 23, 2026

Nürnberg NLP @ GermEval Shared Task 2026: Harmful Content Detection in German Social Media through Error-Independent LLM Voters

Authors: Philipp Steigerwald, Eric Rudolph, Jens Albrecht

Organizations: Technische Hochschule Nürnberg Georg Simon Ohm

Abstract

Harmful content in German social media does real-world damage, from calls to action to criminal defamation. The GermEval 2026 shared task scores its detection in four subtasks. The technical challenge is a severe class imbalance. The harmful classes are rare and share surface language with the dominant majority class, yet under macro-F1 they decide the score. The decisive lever is then not a stronger single model but error independence. This insight becomes a per-subtask nine-voter ensemble spanning three orthogonal axes: LLM, training method and class scope. Selected mainly on internal cross-validation, the system reaches macro-F1 of 89.56 (C2A), 71.63 (DBO), 54.84 (VIO) and 83.02 (DEF) on the hidden test set, placing first on all four subtasks.

Figures & tables

Panel of Figure 1 — Figure 1: Per-subtask ensemble: three scope branches, three voters each, plain majority decides.

Subtask	Label	Train	Prior	+Aug	Prior ${}_{\text{+aug}}$
C2A	false	14,405	90.9%	—	90.9%
C2A	true	1,434	9.1%	—	9.1%
DBO	nothing	13,818	87.2%	—	84.3%
	criticism	1,839	11.6%	—	11.2%
	agitation	143	0.9%	+400	3.3%
	subversive	53	0.3%	+144	1.2%

Table 1: Per-subtask label counts. Prior is the label’s share before, Prior

{}_{\text{+aug}}

after augmentation.

Panel of Figure 2 — Figure 2: Both modes select the subtask by its prompt (dashed); ClsHead-G adds one head per subtask.

Sub.	V	Subtask	Voter composition	$\mathit{F1}_{\mathrm{cv}}^{\mathrm{ens}}$	$\mathit{F1}_{\mathrm{test}}$
1	6	C2A	llamB-G-cls + min-G-cls	84.93	89.17
		DBO	llamB-G-cls + min-G-cls	63.56	66.89
		VIO	llamB-G-cls + min-G-cls	51.56	53.43
		DEF	llamB-G-cls + min-G-cls	79.25	78.61
2	9	C2A	llamB-G-cls + min-G-cls + min-S-cls	86.00	89.08
		DBO	llamB-G-cls + min-G-cls + min-S-cls	68.41	67.15

Table 2: Every submission per subtask, internal

\mathit{F1}_{\mathrm{cv}}^{\mathrm{ens}}

against hidden-test

\mathit{F1}_{\mathrm{test}}

(%). V = voters per subtask, bold = per-subtask best per column, (na) = no augmentation.

Model	Method	GPU	s/tweet
llamB	ClsHead	RTX 2080 Ti	0.42
min	ClsHead	RTX 2080 Ti	0.79
phi4	ClsHead	RTX 2080 Ti	1.63
phi4	SFT	RTX 2080 Ti	2.42
llamB	SFT	RTX 2080 Ti	2.57
mis128	SFT	H200	0.91

Table 3: Measured per-voter inference latency, median seconds per tweet.

Figure 3: Core-voter agreement per winner, darker shades = more agreement, nine-voter Krippendorff’s

\alpha

on top.

Figure 4: Third-branch flips among the

n

contested tweets per subtask (core

\leq

4/6). Black is the total flip rate.

Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

	SFT	ClsHead
LoRA rank	32	16
LoRA $\alpha$	64	32
LoRA dropout	0.05	0.05
Learning rate	$1\mathrm{e}{-4}$	$2\mathrm{e}{-5}$
LR schedule	cosine	cosine
Warmup ratio	0.1	—

Appendix

Table 4: Core training hyperparameters of the 7–14B models (4-bit NF4 QLoRA, no truncation). 128B deviations in the text.

Model	Method	f0	f1	f2	f3	f4	top3
min	Prompt (S)	62.48	61.70	62.15	62.03	62.44	62.36
min	SFT (S)	83.93	84.23	84.97	84.84	85.75	85.19
min	ClsHead (S)	86.13	85.34	86.03	84.36	85.30	85.83
phi4	Prompt (S)	75.59	73.61	73.67	75.24	74.75	75.19
phi4	SFT (S)	85.75	86.24	86.35	85.79	85.39	86.12
phi4	ClsHead (G)	85.85	86.49	85.84	84.48	86.24	86.19

Appendix

Table 5: C2A: per-fold CV5 macro-

\mathit{F1}

(%), full-label scope in parentheses, best top3 bold.

Model	Method	f0	f1	f2	f3	f4	top3
min	Prompt (S)	18.42	18.59	18.41	17.49	17.82	18.47
min	SFT (S)	67.47	66.36	62.25	68.26	66.92	67.55
min	ClsHead (S)	66.11	68.80	65.57	66.49	64.95	67.13
phi4	Prompt (S)	33.91	33.63	36.39	33.17	31.85	34.64
phi4	SFT (S)	64.15	70.87	65.20	63.46	55.30	66.74
phi4	ClsHead (S)	69.20	68.50	66.55	61.77	67.54	68.41

Appendix

Table 6: DBO: per-fold CV5 macro-

\mathit{F1}

(%), full-label scope in parentheses, best top3 bold.

Model	Method	f0	f1	f2	f3	f4	top3
min	Prompt (S)	33.56	35.73	27.08	31.50	34.29	34.53
min	SFT (S)	45.75	54.93	45.26	50.31	53.01	52.75
min	ClsHead (G)	48.98	55.96	54.46	47.66	56.87	55.76
phi4	Prompt (S)	19.72	20.32	17.90	20.92	21.95	21.06
phi4	SFT (S)	48.73	55.16	43.78	41.79	61.07	54.98
phi4	ClsHead (S)	51.96	53.75	49.14	48.27	55.60	53.77

Appendix

Table 7: VIO: per-fold CV5 macro-

\mathit{F1}

(%), full-label scope in parentheses, best top3 bold.

Model	Method	f0	f1	f2	f3	f4	top3
min	Prompt (S)	50.83	52.95	51.98	51.26	51.98	52.30
min	SFT (S)	76.09	80.61	79.12	77.57	82.34	80.69
min	ClsHead (S)	79.18	79.12	80.90	82.50	82.44	81.95
phi4	Prompt (S)	67.01	67.14	69.23	66.01	66.59	67.80
phi4	SFT (S)	79.41	78.88	80.56	81.03	82.52	81.37
phi4	ClsHead (S)	78.37	78.67	80.02	83.09	82.90	82.01

Appendix

Table 8: DEF: per-fold CV5 macro-

\mathit{F1}

(%), full-label scope in parentheses, best top3 bold.

Figure 5: Per-subtask t-SNE of the untrained 128B last-token hidden states, with the rare positive classes overlapping off the shelf.

Language Model Ensembles

Language model ensembles combine predictions or reasoning from multiple models to improve accuracy, reliability, coverage, or efficiency.

See all 134 papers

EVIL-Detect for NLPCC 2026 Shared Task 6: LLM-Generated Text Detection
Aug 11, 2026Hongrui Bao, Hangyu Rong, Zhuoshang Wang +2AI-Generated Text Detection Language Model Generation Evaluation
Institute of Information Engineering, Chinese Academy of Sciences, Beijing, China · School of Cyber Security, University of Chinese Academy of Sciences, Beijing, China
ArGuard Shared Task: Harmful Content Detection in Arabic Memes and LLM Prompts
Sep 24, 2026Firoj Alam, Md. Rafiul Biswas, Mohamed Bayan Kmainasi +5LLM Safety Benchmarks Arabic NLP
Qatar Computing Research Institute, Qatar · Hamad Bin Khalifa University, Qatar · APAVI.AI, France +1
REGLAT at ArGuard Shared Tasks: An Ensemble-Based Model Integrating NER for Harmful Content Detection in Arabic
Oct 27, 2026Ahmed M. Fetouh, Mariam Labib, Nsrin Ashraf +2Text Classification Arabic NLP
Faculty of Computers and Artificial Intelligence, Benha University, Egypt · Computer Engineering, Elsewedy University of Technology, Cairo, Egypt · College of Engineering, Wadi Ad Dwaser, Prince Sattam Bin Abdulaziz University, KSA

Aug 11, 2026J/K move · Enter open · S save

EVIL-Detect for NLPCC 2026 Shared Task 6: LLM-Generated Text Detection

Hongrui Bao, Hangyu Rong, Zhuoshang Wang, Yubing Ren+1

Institute of Information Engineering, Chinese Academy of Sciences, Beijing, China · School of Cyber Security, University of Chinese Academy of Sciences, Beijing, China

AI-Generated Text Detection Language Model Generation Evaluation Ensemble Learning Robustness of Machine-Generated Text Detection

Open paper AskPDF arXiv

The rapid development of large language models (LLMs) has increased the need for reliable detection of LLM-generated text, especially in realistic Chinese scenarios involving human-written text (HWT), LLM-generated text (LGT), and LLM-refined text (HLT). This paper presents EVIL-Detect, a multi-signal ensemble framework with conflict-aware fusion for NLPCC 2026 Shared Task 6. The system integrates edit-extent regression, zero-shot likelihood-contrast signals, lexical statistics, and conservative text rules. With calibrated decision boundaries and conflict-aware integration, our system improves robustness under strong out-of-distribution shifts, achieving a macro-F1 score of 0.8888 and ranking first in the official evaluation. Our code is available at https://github.com/bbbbhrrrr/evildetect.

Abstract

Language Model Ensembles

Explore similar work

EVIL-Detect for NLPCC 2026 Shared Task 6: LLM-Generated Text Detection

ArGuard Shared Task: Harmful Content Detection in Arabic Memes and LLM Prompts

REGLAT at ArGuard Shared Tasks: An Ensemble-Based Model Integrating NER for Harmful Content Detection in Arabic