PSK@EEUCA 2026: Fine-Tuning Large Language Models with Synthetic Data Augmentation for Multi-Class Toxicity Detection in Gaming Chat

Srikar Kashyap Pulipaka

cs.CLMay 8, 2026

PSK@EEUCA 2026: Fine-Tuning Large Language Models with Synthetic Data Augmentation for Multi-Class Toxicity Detection in Gaming Chat

Authors: Srikar Kashyap Pulipaka

Organizations: Independent Researcher

Abstract

This paper describes our system for the EEUCA 2026 Shared Task on Understanding Toxic Behavior in Gaming Communities. The task involves classifying World of Tanks chat messages into six toxicity categories: Non-toxic, Insults/Flaming, Other Offensive, Hate/Harassment, Threats, and Extremism. We explore multiple approaches including encoder-based models, instruction-tuned LLMs with LoRA fine-tuning, hierarchical classification, one-vs-rest strategies, and various ensemble methods. Our best system combines Llama 3.1 8B with carefully calibrated 5% synthetic data augmentation, achieving an F1-macro score of 0.6234 on the test set, placing 4th out of 35 participating teams. We provide extensive analysis of the dataset's annotation patterns and their impact on model generalization, revealing a critical ''validation trap'' phenomenon where high validation performance correlates with poor test transfer.

Explore similar work

Measuring and Mitigating Toxicity in Large Language Models: A Comprehensive Replication Study
May 13, 2026Mokshit Surana, Archit Rathod, Akshaj SatishkumarLarge Language Model Safety Toxicity
ToxiREX: A Dataset on Toxic REasoning in ConteXt
Jun 26, 2026Stefan F. Schouten, Ilia Markov, Piek VossenToxicity Harmful Content
Toxicity in Twitch Chats: An LLM-Based Analysis Across Gaming Communities
May 18, 2026Ronja Fuchs, Florian Rupp, Timo Bertram +2Toxicity

May 13, 2026 · cs.CLJ/K move · Enter open · S save

Measuring and Mitigating Toxicity in Large Language Models: A Comprehensive Replication Study

Mokshit Surana, Archit Rathod, Akshaj Satishkumar

University of Illinois Chicago

Large Language Models (LLMs) trained on web-scale corpora inherently absorb toxic patterns from their training data. This leads to toxic degeneration where even innocuous prompts can trigger harmful outputs. This phenomenon poses significant risks for real-world deployments. Thus, necessitating effective mitigation strategies that should maintain model utility while ensuring safety. In this comprehensive replication study, we evaluate the efficacy of DExperts (Decoding-time Experts), which is an inference-time mitigation technique that steers generation without requiring model retraining. We structured our research into three systematic phases: (1) establishing baseline toxicity measurements using RealToxicityPrompts on standard GPT-2 models; then (2) implementing and evaluating DExperts to mitigate explicit toxicity; and finally (3) stress-testing the method against implicit hate speech using the adversarial ToxiGen dataset. Our empirical results confirm that while DExperts achieves near-perfect safety rates (100%) on explicit toxicity benchmarks, it exhibits brittleness against adversarial, implicit hate speech, with safety rates dropping to 98.5%. Furthermore, we quantify a critical trade-off. The method introduces a 10x latency penalty (from 0.2s to 2.0s per generation), posing challenges for real-time deployment scenarios. This study contributes to the growing body of work on AI safety by highlighting the robustness gap between explicit and implicit toxicity mitigation. We emphasize the need for more sophisticated approaches that generalize across diverse hate-speech patterns without incurring prohibitive computational costs.

Large Language Model Safety Toxicity Hate Speech Reproducibility

Open paper Ask about it PDF

Abstract

Explore similar work

Measuring and Mitigating Toxicity in Large Language Models: A Comprehensive Replication Study

ToxiREX: A Dataset on Toxic REasoning in ConteXt

Toxicity in Twitch Chats: An LLM-Based Analysis Across Gaming Communities