U4RASD at KnowledgeGraphEval-2026: Ontology-Constrained Encoder and LLM Systems for Arabic Relation Extraction

Ahmad Abbas; Hussein Mazeh; Chadi Abou Chakra; Aya Jouni; Mohammad Ballout; Fadi A. Zaraket

cs.CLOct 8, 2026

U4RASD at KnowledgeGraphEval-2026: Ontology-Constrained Encoder and LLM Systems for Arabic Relation Extraction

Authors: Ahmad Abbas, Hussein Mazeh, Chadi Abou Chakra, Aya Jouni, Mohammad Ballout, Fadi A. Zaraket

Organizations: Arab Center for Research and Policy Studies, Doha · American University of Beirut, Beirut, Lebanon

Abstract

This paper details our submission to KnowledgeGraphEval 2026 Subtask 2 on directed Arabic relation extraction. We explored encoder-based and large language model approaches for predicting one of 40 relations or no_relation from an Arabic sentence and a subject-object pair. Our encoder systems use typed entity markers and separate relation-existence and relation-classification heads, while our LLM systems fine-tune Yehia and Qwen3.5 with QLoRA and ontology-constrained prediction. Our best system, an ensemble of Yehia-RE, Qwen-RE, and TypedPair, achieved a micro-F1 score of 94.06 and ranked 1st in Subtask 2. Yehia-RE (full-vocabulary) achieved 92.98, while the best encoder and Qwen-RE achieved 92.69 and 91.94 respectively. Our results highlight the importance of constrained prediction and complementary modeling approaches for Arabic relation extraction.

Relation Extraction

Relation extraction identifies and classifies relationships between entities or events in text and multimodal data.

See all 82 papers

HeRE: A Novel Benchmark for Hebrew Relation Extraction via LLM-Guided Denoising of Knowledge Graph Alignments
Oct 15, 2026Ron Keinan, Amir David Nissan Cohen, Reut TsarfatyKG Entity Alignment Relation Extraction
Bar-Ilan University · Origin AI
Cross-lingual Relation Extraction with Large Language Models: Zero-Shot, Few-Shot, and Fine-Tuned Evaluation on Romanian
Jun 30, 2026Dragos-Mitrut Vasile, Elena-Simona Apostol, Stefan-Adrian Toma +2Relation Extraction LLM Fine-Tuning
National University of Science and Technology POLITEHNICA Bucharest, Splaiul Independent,ei 313, Bucures,ti 060042, Romania · Military Technical Academy ‘Ferdinand I’, Bulevardul George Cos,buc 39-49, 050141, Bucharest, Romania · Freie Universität Berlin, Arnimallee 14, Berlin, 14195, Germany +2
AraSSM: A bidirectional state-space encoder for Arabic masked language modeling
Aug 8, 2026Ahmed Amine Aliane, Hassina Aliane, Nasredine SemmarLanguage Model Pretraining Arabic NLP
Arabic Institute for Translation, Algiers, Algeria · CERIST, Algeria · CEA LIST, Paris-Saclay, France

Oct 15, 2026J/K move · Enter open · S save

HeRE: A Novel Benchmark for Hebrew Relation Extraction via LLM-Guided Denoising of Knowledge Graph Alignments

Ron Keinan, Amir David Nissan Cohen, Reut Tsarfaty

Bar-Ilan University · Origin AI

KG Entity Alignment Relation Extraction Low-Resource Language Processing LLM Information Extraction

Open paper AskPDF arXiv

While LLMs demonstrate strong capabilities in English Relation Extraction (RE) tasks, a significant gap remains in assessing their RE performance in low-resource languages, due to a lack of standardized benchmarks. Creating native, language-specific annotated RE benchmarks is expensive and requires human expertise, while relying on translation yields suboptimal, inaccurate datasets that introduce English-centric biases. To tackle this, we propose a scalable, language-agnostic methodology to create RE data by denoising naturally occurring yet noisy alignments of knowledge bases and corresponding texts. We define a proper alignment via a Natural Language Inference (NLI) task, where a relation triplet is entailed by an input text. We empirically assess five denoising strategies, and demonstrate that multilingual models outperform language-specific models in the few-shot LLM setting, while Hebrew-specific encoders remain competitive under NLI fine-tuning and are strongest in the label-free cross-training setting. We further find that prompting in the native language achieves performance on par with or slightly exceeding English prompts, confirming that the pipeline operates naturally in the target language. The best open-weights model, Gemma-4-31B-it (F1,=,0.915), matches the best API model, GPT-5.4 (0.921), and reaches F1,=,0.844 on a fully held-out, dual-annotated test set. Applying the optimized pipeline, we release HeRE: a novel, massive, 1.5-million-instance silver dataset for Hebrew RE, designed to support data-intensive experimentation and model fine-tuning. HeRE bridges a critical gap in standard Hebrew RE evaluation, while providing the NLP community with a reproducible, language-agnostic pipeline for RE dataset construction in other low-resource settings.

Abstract

Relation Extraction

Explore similar work

HeRE: A Novel Benchmark for Hebrew Relation Extraction via LLM-Guided Denoising of Knowledge Graph Alignments

Cross-lingual Relation Extraction with Large Language Models: Zero-Shot, Few-Shot, and Fine-Tuned Evaluation on Romanian

AraSSM: A bidirectional state-space encoder for Arabic masked language modeling