cs.AINov 3, 2026

Beyond Fine-Tuning: Evaluating Prompt Engineering and Prompt Tuning for Roman Urdu Hate Speech Detection

Authors: Toneema Zubair, Muhammad Junaid Asif, Faisal Kamiran, Hafiz Hassan Saeed, Rana Fayyaz Ahmad

Organizations: Department of Computer Sciences, Information Technology University (ITU), Lahore 54000, Pakistan · Natural Language Processing (NLP) Lab, Artificial Intelligence Technology Centre (AITeC), Islamabad 44000, Pakistan · Faculty of IT and Computer Sciences, University of Central Punjab (UCP), Lahore (54000), Pakistan

Abstract

Low Resource Languages (LRLs) pose significant challenges for hate speech detection because of a scarcity of readily available annotated data, language variation, non-standard spelling, and context-dependent ambiguity. Beyond the problem of informal conventions of writing in Roman Urdu, there is also the issue of code-mixing. After working on the previous project about zero-shot learning and parameter-efficient fine-tuning (PEFT), this research systematically compares prompt engineering and prompt tuning for Roman Urdu hate speech detection. To study the effect of in-context examples, we will compare the zero-shot and few-shot prompting approaches. To understand the advantage of structured prompting and learnable prompt representations, we will conduct studies of manual vs. mixed prompt-based adaptation strategies. Experimentations are carried out on the PURUT dataset (~72,771 social media comments) with GPT-3.5, LLaMA-3, Mistral-7B, and multilingual BERT in several settings. Experimental results show that few-shot prompting always outperforms zero-shot prompting, with the Mistral-7B model scoring 0.67 F1 compared to 0.56 of the zero-shot models. Based on this observation, prompt-based adaptation is proposed to introduce additional improvements, with the proposed mixed prompt configuration able to yield a F1-score up to 0.87 which further surpasses the performance of regular manual prompts. Despite the larger size of the model, its performance trade-off between inference-based prompting, prompt optimization, and parameter-efficient model adaptation is compared to the previously reported PEFT-based F1-score, which is ~0.93.

Explore similar work

CardsList
  1. Efficient Adaptation of LLMs for Hate Speech Detection in Low-Resource Languages: A Comparative Study on Roman Urdu

    Aug 6, 2026Toneema Zubair, Muhammad Junaid Asif, Faisal Kamiran +2LLM EvaluationHate Speech Detection

  2. Hate Speech Classification In Roman Urdu: A Comparative Study On Parameter Efficient Fine-Tuning And Prompt Engineering

    Aug 10, 2026Toneema ZubairHate Speech DetectionPrompt Engineering

  3. From Specialization to Generalization: Instruction-tuned LLMs for Robust Harmful Content Mitigation

    Aug 26, 2026Lukas Edman, Daryna Dementieva, Alexander FraserContent ModerationHate Speech Detection