cs.CLFeb 21, 2026

Yor-Sarc: A gold-standard dataset for sarcasm detection in a low-resource African language

Authors: Toheeb Aduramomi Jimoh, Tabea De Wille, Nikola S. Nikolov

Organizations: Department of Computer Science and Information Systems, University of Limerick, Castletroy, V94 T9PX, Limerick, Ireland

Abstract

Sarcasm detection poses a fundamental challenge in computational semantics, requiring models to resolve disparities between literal and intended meaning. The challenge is amplified in low-resource languages where annotated datasets are scarce or nonexistent. We present Yor-Sarc, the first gold-standard dataset for sarcasm detection in Yorùbá, a tonal Niger-Congo language spoken by over 5050 million people. The dataset comprises 436 instances annotated by three native speakers from diverse dialectal backgrounds using an annotation protocol specifically designed for Yorùbá sarcasm by taking culture into account. This protocol incorporates context-sensitive interpretation and community-informed guidelines and is accompanied by a comprehensive analysis of inter-annotator agreement to support replication in other African languages. Substantial to almost perfect agreement was achieved (Fleiss' κ=0.7660κ= 0.7660; pairwise Cohen's κ=0.6732κ= 0.6732--0.87430.8743), with 83.3%83.3\% unanimous consensus. One annotator pair achieved almost perfect agreement (κ=0.8743κ= 0.8743; 93.8%93.8\% raw agreement), exceeding a number of reported benchmarks for English sarcasm research works. The remaining 16.7%16.7\% majority-agreement cases are preserved as soft labels for uncertainty-aware modelling. Yor-Sarc\footnote{https://github.com/toheebadura/yor-sarc} is expected to facilitate research on semantic interpretation and culturally informed NLP for low-resource African languages.

Figures & tables

Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. CHARM: Charge Calibration and Acoustic Rescue for LLM-based Multimodal Sarcasm Detection

    Jul 13, 2026Qiyang Sun, Yi Chang, Yupei Li +3Language Model CalibrationMultimodal Sentiment Analysis

  2. YoNER: A New Yorùbá Multi-domain Named Entity Recognition Dataset

    Apr 7, 2026Peace Busola Falola, Jesujoba O. Alabi, Solomon O. Akinola +3Named Entity RecognitionLow-Resource Language Processing

  3. SarcasTraps: Effect of Sarcasm on Turkish Hate Speech Detection

    Oct 12, 2026Niyazi Ahmet Metin, Dilara KeküllüoğluHate Speech Detection