REGLAT at ArGuard Shared Tasks: An Ensemble-Based Model Integrating NER for Harmful Content Detection in Arabic
Authors: Ahmed M. Fetouh, Mariam Labib, Nsrin Ashraf, Omer Dawood, Hamada Nayel
Organizations: Faculty of Computers and Artificial Intelligence, Benha University, Egypt · Computer Engineering, Elsewedy University of Technology, Cairo, Egypt · College of Engineering, Wadi Ad Dwaser, Prince Sattam Bin Abdulaziz University, KSA
Detecting harmful content in Arabic is challenging because of dialectal variations and the semantic overlap between different harm categories. This paper presents a two-stage approach for classifying Arabic user prompts into seven categories of harmful content. First, GLiNER is used to extract harm-related entities from each prompt. The extracted entities are then added to the original text and passed to a fine-tuned MARBERT classifier for final classification. The approach combines explicit entity information with the contextual understanding of a pretrained Arabic language model. The system was evaluated on the development set of the ArGuard 2026 shared task. The results achieved a macro-F1 score of 0.886 and an accuracy of 0.897, demonstrating promising performance in fine-grained harmful content detection. The system reported an accuracy of 0.793 and Macro-F score of 0.735 on a blind test set for subtask 1. For subtask 2, it achieved an accuracy of 0.770 and Macro-F score of 0.754. The analysis also showed that distinguishing between related categories remains challenging. These findings suggest that integrating NER with Arabic language models is a promising approach to harmful content detection.
Figures & tables
Figure 1: Granular category distribution in the training set.
Figure 2: The pipeline of the proposed system.
Hyperparameter
Value
Learning rate
2×10−5
Warmup
10%
Epochs
3
Train / eval batch size
32 / 16
Weight decay
0.01
Model selection
Macro-F1
Table 1: Training hyperparameters.
Metric
Score
Macro-F1
0.886
Accuracy
0.897
Macro-Precision
0.885
Macro-Recall
0.888
Table 2: Overall development-set results.
Category
Precision
Recall
F1
Support
Bully
0.8341
0.8341
0.8341
217
FraudandDeception
0.8842
0.9087
0.8963
252
adult_content
0.9863
0.9878
0.9871
657
harassment
0.8457
0.8509
0.8483
322
harm-to-others
0.8231
0.8060
0.8145
433
hate_speech
0.9191
0.9259
0.9225
135
Table 3: Per-class development-set classification report of the GLiNER + MARBERT system.
Figure 3: Development-set confusion matrix of the GLiNER + MARBERT system.
ArGuard is a shared task on harmful content detection in Arabic memes and LLM prompts. It includes two tracks: Track A focuses on multimodal hate detection in Arabic memes, while Track B addresses harmful prompt detection for Arabic LLM safety evaluation. In total, 58 teams registered, 35 participated in the final evaluation, and 27 submitted system-description papers. Participating teams explored models such as AraBERT, Jais, and Qwen3-VL. The best systems achieved macro-F1 scores of 0.823 on A1, 0.419 on A2, 0.984 on B1, and 0.790 on B2. Fine-grained meme classification in A2 was the most challenging setting, partly due to sparse labels and train-test distribution shifts.
Firoj Alam, Md. Rafiul Biswas, Mohamed Bayan Kmainasi +5
Qatar Computing Research Institute, Qatar · Hamad Bin Khalifa University, Qatar · APAVI.AI, France +1
We present AK_QMUL's submissions to the ArGuard-2026 shared tasks on Arabic harmful-prompt detection: a binary safe/unsafe task (B1) and a seven-way harm-domain task (B2). Both submissions use one backbone -- JAIS-6.7B fine-tuned with 4-bit QLoRA and a linear classification head over its last-token representation. For B1 we train a four-way head that factors the label by dialect register {safe, unsafe} x {MSA, other}, we augment the training fold with MSA prompts from AraSafe, and decode with a plain argmax to score safe/unsafe. For B2 we fine-tune the same backbone as a seven-way classifier and average the softmax outputs of five random seeds. Both submissions ranked first on their track, with macro-F1 0.984 on B1 and 0.790 on B2. We report results by dialect and by class: on B1 the model is accurate across all six dialects (93.6--99.8%) and its predicted unsafe rate tracks the true rate closely; on B2 six of the seven classes reach F1 between 0.73 and 0.98, with the residual error concentrated in the Bully class, which overlaps semantically with harassment and harm-to-others.
We present ArabicDialectSafety, a human-curated Arabic safety dataset of 25,071 prompts covering six Arabic varieties: Modern Standard Arabic, Syrian, Egyptian, Algerian, Palestinian, and Moroccan. The dataset is annotated with dialect labels and seven fine-grained harm categories. We introduce a dual-task evaluation framework for binary safe/unsafe detection and granular harm classification across dialects. Benchmarking seven supervised and generative models, we find that fine-tuned MARBERTv2 achieves the strongest performance, with Macro-F1 scores of 0.95 for binary classification and 0.90 for granular classification, substantially outperforming prompted frontier LLMs, including Arabic-specialized models. Our analyses show that dialect conditioning is most effective when integrated at the representation level, while significant performance gaps remain for low-resource Maghrebi dialects. We further evaluate seven frontier LLMs as response generators on harmful dialectal Arabic prompts and observe unsafe generation rates below 5 percent across models. We release the dataset and code upon acceptance to support future research on dialect-aware Arabic safety evaluation. Warning: This paper contains examples of harmful and potentially offensive content included solely for research purposes.