Effective QA-driven Annotation of Predicate-Argument Relations Across Languages

Jonathan Davidov; Aviv Slobodkin; Shmuel Tomi Klein; Reut Tsarfaty; Ido Dagan; Ayal Klein

cs.CLFeb 26, 2026

Effective QA-driven Annotation of Predicate-Argument Relations Across Languages

Authors: Jonathan Davidov, Aviv Slobodkin, Shmuel Tomi Klein, Reut Tsarfaty, Ido Dagan, Ayal Klein

Organizations: Bar-Ilan University · Ariel University

Abstract

Explicit representations of predicate-argument relations form the basis of interpretable semantic analysis, supporting reasoning, generation, and evaluation. However, attaining such semantic structures requires costly annotation efforts and has remained largely confined to English. We leverage the Question-Answer driven Semantic Role Labeling (QA-SRL) framework -- a natural-language formulation of predicate-argument relations -- as the foundation for extending semantic annotation to new languages. To this end, we introduce a cross-linguistic projection approach that reuses an English QA-SRL parser within a constrained translation and word-alignment pipeline to automatically generate question-answer annotations aligned with target-language predicates. Applied to Hebrew, Russian, and French -- spanning diverse language families -- the method yields high-quality training data and fine-tuned, language-specific parsers that outperform strong multilingual LLM baselines (GPT-4o, LLaMA-Maverick). By leveraging QA-SRL as a transferable natural-language interface for semantics, our approach enables efficient and broadly accessible predicate-argument parsing across languages.

Figures & tables

Both were shot in the confrontation with police and have been recovering in hospital since the massive attack.
QA-SRL	1	When was someone shot?	in the confrontation ; the attack
	2	Who was shot?	Both
	3	Who shot someone?	police
	4	Where has someone been recovering?	in hospital
	5	How long was someone recovering from something?	since the attack
	6	Who was recovering from something?	Both

Table 1: An example sentence annotated with QA-SRL and QANom.

Language	Sentence	Question	Answer	English gloss
Hebrew	\hebrew המדיניות את אישרה הוועדה \hebrew החדשה	\hebrew משהו? אישר מי	\hebrew הוועדה	Who approved something? → The committee
		\hebrew אישר? מישהו מה	\hebrew החדשה המדיניות את	What did someone approve? → the new policy
Russian	Комитет одобрил новую политику	Кто что одобрил?	Комитет	Who approved something? → The committee
		что кто-то одобрил?	новую политику	What did someone approve? → the new policy
French	Le comité a approuvé la nouvelle politique	qui a approuvé quelque chose?	Le comité	Who approved something? → The committee
		qu’est-ce que quelqu’un a approuvé?	la nouvelle politique	What did someone approve? → the new policy

Table 2: Examples of model predictions in different target languages.

Figure 1: A schematic overview of our proposed methodology for transferring QA-SRL to new languages based on the English QA-SRL infrastructure.

	Train			Dev			Test
Language	Sents	Pred.	QAs	Sents	Pred.	QAs	Sents	Pred.	QAs
Hebrew	13,233	33,275	80,210	64	146	368	116	342	793
Russian	18,804	42,518	102,197	98	258	689	228	603	1,582
French	18,181	38,157	97,336	51	124	259	144	288	616

Table 3: Number of sentences, predicates, and QA pairs in the projected QA-SRL datasets for Hebrew, Russian, and French.

	Unlabeled AD			Labeled AD
Language	P	R	F1	P	R	F1
Hebrew	67.8	94.7	79.0	57.4	93.8	71.2
Russian	70.3	87.9	78.1	47.3	83.3	60.3
French	81.3	93.5	87.0	60.8	91.6	73.1

Table 4: Manual evaluation results of the projected training sets.

Model	Metric	Hebrew	Russian	French
GPT-4o (ICL)	Unlabeled Match	66.3	67.9	63.2
	Exact Match	10.5	14.1	10.2
	Semantic Match	38.3	47.0	41.8
LLaMA-4-Maverick (ICL)	Unlabeled Match	61.6	60.1	52.6
	Exact Match	12.0	12.8	7.5
	Semantic Match	38.8	45.0	30.7

Table 5: F1 scores of baseline and fine-tuned models across Hebrew, Russian, and French. Double horizontal rules delineate three blocks: (1) The upper block lists strong large-scale baseline systems (GPT-4o and LLaMA-4-Maverick). (2) The middle block reports a mid-scale multilingual model before and after fine-tuning on our projected dataset. (3) The lower block presents language-specific models that are comparable in scale to the middle block, comparing their in-context and fine-tuned variants per language.

Appendix figures & tables2 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 2: F1 score as a function of the IOU threshold

Figure 3: ROC curve for span matching decisions

Explore similar work

Revisiting Semantic Role Labeling: Efficient Structured Inference with Dependency-Informed Analysis
May 4, 2026Sangpil Youm, Leah Jones, Bonnie J. DorrStructured Prediction
University of Florida, Gainesville, Florida
Cross-lingual Relation Extraction with Large Language Models: Zero-Shot, Few-Shot, and Fine-Tuned Evaluation on Romanian
Jun 30, 2026Dragos-Mitrut Vasile, Elena-Simona Apostol, Stefan-Adrian Toma +2Relation Extraction LLM Fine-Tuning
National University of Science and Technology POLITEHNICA Bucharest, Splaiul Independent,ei 313, Bucures,ti 060042, Romania · Military Technical Academy ‘Ferdinand I’, Bulevardul George Cos,buc 39-49, 050141, Bucharest, Romania · Freie Universität Berlin, Arnimallee 14, Berlin, 14195, Germany +2
The Multilingual FrameNet Corpus
Aug 24, 2026Beatrice Fiumanò, Nicolas Lazzari, Simone Paolo Ponzetto +1Semantic Parsing Cross-Lingual Transfer
University of Bologna, Italy · University of Pisa, Italy · University of Mannheim, Germany

May 4, 2026J/K move · Enter open · S save

Revisiting Semantic Role Labeling: Efficient Structured Inference with Dependency-Informed Analysis

Sangpil Youm, Leah Jones, Bonnie J. Dorr

University of Florida, Gainesville, Florida

Structured Prediction

Open paper AskPDF arXiv

Semantic Role Labeling (SRL) provides an explicit representation of predicate-argument structure, capturing linguistically grounded relations such as who did what to whom. While recent NLP progress has been dominated by large language models (LLMs), these systems often rely on implicit semantic representations, often lacking explicit structural constraints and systematic explanatory mechanisms. Traditionally, SRL systems have often relied on AllenNLP; however, the framework entered maintenance mode in December 2022, limiting compatibility with evolving encoder architectures and modern inference requirements. We revisit structured SRL modeling, introducing a modernized encoder-based framework that preserves explicit predicate-argument structure while enabling inference 10 times faster. Using BERT-base, the model attains comparable predictive performance, and RoBERTa and DeBERTa further improve F1 performance within the same framework. We adopt a dependency-informed diagnostic methodology to characterize span-level inconsistencies and conduct a representation-level analysis of LLM behavior under dependency-informed structural signals. Results indicate that dependency cues primarily improve structural stability. Finally, we illustrate how the framework's explicit predicate-argument structure can support multilingual SRL projection as a downstream application.

Abstract

Explore similar work

Revisiting Semantic Role Labeling: Efficient Structured Inference with Dependency-Informed Analysis

Cross-lingual Relation Extraction with Large Language Models: Zero-Shot, Few-Shot, and Fine-Tuned Evaluation on Romanian

The Multilingual FrameNet Corpus