cs.CLOct 18, 2026
SaveSZED at ArGuard Shared Tasks: Multi-Task Label-Attention and Cross-Modal Fusion for Arabic Hateful Meme Detection
Organizations: ENSIA, Algeria
Abstract
We present SZED's submission to ArGuard 2026 Track A: Arabic hateful meme detection. The task has two parts: binary classification (A1) and fine-grained multi-label tagging (A2). For A1, we fuse MARBERTv2 and CLIP ViT-L/14 embeddings, comparing concatenation, self-attention, and ensemble heads. For A2, we use a multi-task label-attention architecture with per-class query vectors and an auxiliary head for cascade calibration. We reach macro-F1 = 0.7597 on A1 and 0.3142 on A2 (averaged over 10 active classes). We report reproducibility across runs, per-class error patterns, and the effect of cascade calibration on the long-tailed A2 label space.
Figures & tables
| Split | # examples | # Hateful | % Hateful |
|---|---|---|---|
| Train | 3,500 | 1,324 | 37.8 |
| Dev | 500 | 189 | 37.8 |
| Dev-test | 500 | 189 | 37.8 |
| Test (official) | 500 | 148 | 29.6 |
Table 1: ArGuard Task 1 split sizes and class balance. Train, dev, and dev-test share an identical 37.8% Hateful rate; the official test split is more balanced at 29.6% Hateful. The dev-test split is used in the backbone-comparison experiments (Appendix A ).
Figure 1: System overview. (a) A1: shared encoders feed two fusion heads combined via a probability-averaged ensemble. (b) A2: full token/patch sequences feed a fusion transformer, read by ten label-attention queries and, separately, an auxiliary binary head used for cascade calibration.
| Run | Macro-F1 | Accuracy |
|---|---|---|
| Dev (leaderboard) | 0.7381 | 0.7620 |
| Test, run 1 | 0.7556 | 0.7880 |
| Test, run 2 (official) | 0.7597 | 0.7940 |
Table 2: A1 results across independent runs of the identical architecture and hyperparameters. Test, run 2 is our official leaderboard submission.
| Variant | MF1 | Acc. | F1 H | F1 NH |
|---|---|---|---|---|
| Ensemble | 0.7340 | 0.754 | 0.6612 | 0.8069 |
| Transformer | 0.7297 | 0.750 | 0.6556 | 0.8038 |
| Concat | 0.7285 | 0.746 | 0.6595 | 0.7974 |
| Cross-modal | 0.6878 | 0.704 | 0.6166 | 0.7590 |
Table 3: A1 ablation on dev set (one run per variant, seed 42; see Section 5.2 for a caveat on single-run comparisons). MF1 = macro-F1; F1 H / F1 NH = per-class F1. Best threshold per variant applied. The incongruity signal carries no separating power: mean CLIP cosine similarity is 0.236 for Hateful and 0.238 for Not Hateful on training data.
| Metric | Dev a | Test a | Test b |
|---|---|---|---|
| Macro-F1 (raw @0.5) | 0.3276 | – | 0.2969 |
| Macro-F1 (calibrated) | 0.3450 | 0.3142 | 0.3092 |
| Micro-F1 | – | 0.4557 | 0.4395 |
| Hamming loss | – | 0.1992 | 0.2112 |
| Subset accuracy | – | 0.1460 | 0.1340 |
Table 4: A2 results across two runs of the identical architecture. a Submitted/official run (dev and official leaderboard test scores). b An independent rerun, evaluated on test only. Raw: sigmoid outputs at a global 0.5 threshold, no cascade calibration. Calibrated: cascade-scaled probabilities with per-class thresholds. All macro-F1 figures are averaged over the 10 active classes (Section 3 ). The calibrated test macro-F1 gap between the rerun (0.3092) and the official submission (0.3142) reflects run-to-run variance, consistent with the effect discussed for A1 in Section 4.1.
Figure 2: Official test-set per-class F1 for A2 (macro-F1 = 0.3142 over 10 active classes), with support (n), sorted descending.
Appendix figures & tables2 assets
Supplementary material from the paper’s appendix.
Appendix
| Model Backbone | Test Macro-F1 |
|---|---|
| AraBERTv0.2 | 0.7610 |
| MARBERTv2 | 0.7597 |
| CAMeLBERT-Mix | 0.7560 |
| CAMeLBERT-DA | 0.7550 |
Table 5: A1 methodology on different backbone models (test set). All models fall within a narrow range of one another.
| Model Backbone | Test Macro-F1 |
|---|---|
| CAMeLBERT-Mix | 0.3296 |
| MARBERTv2 | 0.3142 |
| CAMeLBERT-DA | 0.3112 |
| AraBERTv0.2 | 0.3040 |
Table 6: A2 methodology on different backbone models (test macro-F1). CAMeLBERT-Mix leads clearly on this split. The MARBERTv2 row reflects our most recent run (the official leaderboard submission, 0.3142).