"bot lane noob" Towards Deployment of NLP-based Toxicity Detectors in Video Games

Jonas Ave; Irdin Pekaric; Matthias Frohner; Giovanni Apruzzese

cs.CRApr 11, 2026

"bot lane noob" Towards Deployment of NLP-based Toxicity Detectors in Video Games

Authors: Jonas Ave, Irdin Pekaric, Matthias Frohner, Giovanni Apruzzese

Organizations: University of Liechtenstein, Vaduz, Liechtenstein · Reykjavik University, Reykjavik, Iceland

Abstract

Toxicity and harassment are widespread in the video-gaming context. Especially in competitive online multiplayer scenarios, gamers oftentimes send harmful messages to other players (teammates or opponents) whose consequences span from mild annoyance to withdrawal and depression. Abundant prior work tackled these problems, e.g., pointing out the negative effects of toxic interactions. However, few works proposed countermeasures specifically developed and tested on textual messages sent during a match -- i.e., when the "harassment" actually occurs. We posit that such a scarcity stems from the lack of high-quality datasets that can be used to devise "automated" detectors based on natural-language processing (NLP) and machine learning (ML), and which can -- ideally -- mitigate the harm of toxic comments during a gaming session. This work provides a foundation for addressing the problem of toxicity and harassment in video games. First, through a systematic literature review (n=1,039), we provide evidence that only few works proposed ML/NLP-based detectors of toxicity/harassment during live matches. Then, we partner-up with 8 expert League of Legend (LoL) players and create a fine-grained labelled dataset, L2DTnH, containing 1.4k toxic and 13.8k non-toxic messages exchanged during LoL matches. We use L2DTnH to develop a detector that we then empirically show outperforms general-purpose and state-of-the-art toxicity detectors reliant on NLP. To further demonstrate the practicality of our resources, we test our detector on game-related data beyond that included in L2DTnH; and we develop a Web-browser extension that flags toxic content in Webpages -- without querying third-party servers owned by AI companies. We publicly release all of our resources. Our contributions pave the way for more applied research devoted to fighting the spread of toxicity and harassment in video games.

Figures & tables

Figure 1: Overview of the creation process of L2DTnH .

ID	Gaming Exp.	Rank	Hrs/wk	Other Exp.	Msgs
A1	17 yrs	Masters	17	MOBA, FPS, MMOS	5k
A2	20 yrs	Diamond	21	MOBA, FPS, MMOS	12k
A3	9 yrs	Diamond	30	MOBA, SBX, FPS	5k
A4	8 yrs	Gold	28	MOBA, SBX, FPS	5k
A5	6 yrs	Diamond	19	MOBA, SBX, FPS	5k
A6	7 yrs	Bronze	10	MOBA, SBX, FPS	15k

Table 1: Annotator background and labeling contribution.

ID	Message	A1	A2	A3	A4	A5	A6	A7	A8	Label
37	mother fucking noob	1	1	1	1	1	1	1	1	1
83	rammus is such a slut	1	0	0	0	0	1	1	1	1
87	FUCK	1	0	0	0	0	0	0	0	0
114	get fked	1	1	0	0	1	1	1	1	1
123	rammus useless	0	1	0	1	0	1	0	1	1
172	leona you are so bad. you lost this lane gg	1	1	0	0	0	1	0	1	1

Table 2: Examples of annotated chat messages with decisions.

Model	Acc.	Prec.	Rec.	F1
protectai_unbiased_toxic_roberta_onnx	0.9021	0.4659	0.4143	0.4386
nicholas_kluge_toxicity_model	0.8708	0.3866	0.6821	0.4935
junglelee_bert_toxic_comment_classification	0.8722	0.3977	0.7500	0.5198
martin_ha_toxic_comment_model	0.9068	0.4908	0.2857	0.3612
garak_llm_roberta_toxicity_classifier	0.9074	0.4978	0.3964	0.4414
ChatGPT 4o (June 2025) P1	0.9008	0.3495	0.9231	0.5070

Table 3: Comparative performance of state-of-the-art toxicity detection models on the test portion (20%) of our L2DTnH dataset. For cost reasons ChatGPT was tested on a subset of our test set.

Evaluation Level	Acc.	Prec.	Rec.	F1
Message level	0.9605	0.8571	0.6857	0.7619
Grouped-message level	0.9712	0.8974	0.8065	0.8491
Match level	0.9157	0.9701	0.8442	0.9028

Table 4: Performance of IGC-BERT across different contextual granularities of message aggregation (note: the IGC-BERT model here follows a different fine-tuning process than that in Table 3 )

Figure 2: Exemplary application of our Browser Extension (website: [ 8 ] )

Dataset	Access	Gran.	Size	Source	Game/Platform	Used in
Corpus-Twitch	✓	message	2.6k	chat msgs.	Twitch	[ 56 ]
WotReplays	✗	message	15k	replays	WOT	[ 58 ]
Custom Dataset	✗	phrase	765k	chat msgs.	RO, DOTA	[ 26 ]
Custom Dataset	✓	message	100k	chat msgs.	Twitch	[ 30 ]
Dotalicious	✗	word	7M	chat msgs.	DOTA	[ 55 ]
Cyberbullying dataset	✓	comment	34k	forum	LoL, WoW	[ 82 ]

Table 5: Datasets Used in Game-Related Toxicity Research

Figure 3: The graphical interface of our browser extension. The analysis can be stopped or resumed.

Toxicity Detection

Toxicity detection identifies harmful language for moderation and predicts toxic effects of chemicals, drugs, or biological exposures.

See all 46 papers

Sep 28, 2026

In-game Toxic Detection: Bi-directional Representations with Attention Residuals

In-game toxic language has emerged as a critical concern in the gaming industry and community. While several frameworks and models for online game toxicity analysis have been proposed, detecting toxicity in player chat utterances remains a formidable challenge: stemming not only from the extremely short length of such utterances but also from the heavy reliance on game slang, abbreviations, and domain-specific jargon, which generic language models are poorly suited to recognize. This paper presents a shared task for in-game toxic language detection built upon real-world in-game chat data, and proposes the best-preforming model for the toxic language slot filling: Bi-directional Representations with Attention Residuals (BRAR). Experimental results demonstrate that BRAR effectively captures the global context and outperforms the existing baselines on slot filling.

Yuanzhe Jia

University of Sydney, Australia

Representation Learning Toxicity Detection

Ask PDF

May 8, 2026

PSK@EEUCA 2026: Fine-Tuning Large Language Models with Synthetic Data Augmentation for Multi-Class Toxicity Detection in Gaming Chat

This paper describes our system for the EEUCA 2026 Shared Task on Understanding Toxic Behavior in Gaming Communities. The task involves classifying World of Tanks chat messages into six toxicity categories: Non-toxic, Insults/Flaming, Other Offensive, Hate/Harassment, Threats, and Extremism. We explore multiple approaches including encoder-based models, instruction-tuned LLMs with LoRA fine-tuning, hierarchical classification, one-vs-rest strategies, and various ensemble methods. Our best system combines Llama 3.1 8B with carefully calibrated 5% synthetic data augmentation, achieving an F1-macro score of 0.6234 on the test set, placing 4th out of 35 participating teams. We provide extensive analysis of the dataset's annotation patterns and their impact on model generalization, revealing a critical ''validation trap'' phenomenon where high validation performance correlates with poor test transfer.

Srikar Kashyap Pulipaka

Independent Researcher

LLM Fine-Tuning Toxicity Detection

Ask PDF

May 18, 2026

Toxicity in Twitch Chats: An LLM-Based Analysis Across Gaming Communities

Toxicity in online gaming communities remains a persistent challenge, manifesting across genres, platforms, and player interactions. While much research is focused on in-game toxicity, less is known about how toxic behavior varies between gaming communities on streaming platforms. To address this shortcoming, we analyze approximately 20 million chat messages from 4,452 streams, spanning seven game genres on Twitch. We categorize messages according to Twitch's toxicity taxonomy with a pre-trained Large Language Model using zero-shot classification. The taxonomy comprises four categories and eight subclasses, including harassment, discrimination, sexual content, and profanity. Our approach achieves an F1 score of 94.5% on the TextDetox dataset and demonstrates human-model agreement comparable to inter-human agreement. Our analysis reveals that 2.4% of all messages are classified as toxic, with notable differences across genres: streams of MOBA games exhibit the highest relative rate of toxicity (3.2%), and sports games show the lowest rate (2%). Furthermore, results indicate that individual games differ significantly in their toxicity distributions, even within genres, suggesting the existence of game-specific community norms and mechanics that shape toxic behavior beyond genre-level effects. These findings offer empirical insights into genre- and game-specific toxicity patterns on Twitch and can inform more targeted moderation strategies for gaming communities.

Ronja Fuchs, Florian Rupp, Timo Bertram +2

Institute for Information Processing Leibniz University Hannover Hannover, Germany · Department of Computer Science Technische Hochschule Mannheim Mannheim, Germany · Institute for Machine Learning Johannes Kepler University Linz, Austria +1

Toxicity Detection

Ask PDF

Abstract

Toxicity Detection

Explore similar work

In-game Toxic Detection: Bi-directional Representations with Attention Residuals

PSK@EEUCA 2026: Fine-Tuning Large Language Models with Synthetic Data Augmentation for Multi-Class Toxicity Detection in Gaming Chat

Toxicity in Twitch Chats: An LLM-Based Analysis Across Gaming Communities