LLM-based Atomic Propositions help weak extractors: Evaluation of a Propositioner for triplet extraction

Luc Pommeret; Thomas Gerald; Patrick Paroubek; Sahar Ghannay; Christophe Servan; Sophie Rosset

cs.CLApr 3, 2026

LLM-based Atomic Propositions help weak extractors: Evaluation of a Propositioner for triplet extraction

Authors: Luc Pommeret, Thomas Gerald, Patrick Paroubek, Sahar Ghannay, Christophe Servan, Sophie Rosset

Organizations: STL · Université Paris-Saclay, CNRS, LISN, 91400, Orsay, France · LISN · STL, AMIAD ·

Abstract

Knowledge Graph construction from natural language requires extracting structured triplets from complex, information-dense sentences. In this paper, we investigate if the decomposition of text into atomic propositions (minimal, semantically autonomous units of information) can improve the triplet extraction. We introduce MPropositionneur-V2, a small multilingual model covering six European languages trained by knowledge distillation from Qwen3-32B into a Qwen3-0.6B architecture, and we evaluate its integration into two extraction paradigms: entity-centric (GLiREL) and generative (Qwen3). Experiments on SMiLER, FewRel, DocRED and CaRB show that atomic propositions benefit weaker extractors (GLiREL, CoreNLP, 0.6B models), improving relation recall and, in the multilingual setting, overall accuracy. For stronger LLMs, a fallback combination strategy recovers entity recall losses while preserving the gains in relation extraction. These results show that atomic propositions are an interpretable intermediate data structure that complements extractors without replacing them.

Figures & tables

Figure 1: The upper schema depicts the pipeline’s stages: at stage 1, we extract atomic propositions from the source text; at stage 2, we extract triplets using either a dependency parser or generative LLMs; and finally, we build the knowledge graph from the entities and relations retrieved. For evaluation, we limit to the triplet entity-relation benchmark.

		GLiREL			Qwen3-0.6B			Qwen3-4B
Benchmark	Pipeline	Acc	e-rec	r-rec	Acc	e-rec	r-rec	Acc	e-rec	r-rec
SMiLER	English – Direct	49.8	99.0	50.3	35.7	100.0	35.7	71.4	100.0	71.4
	English – Prop	43.7	86.6	50.4	35.1	86.6	40.5	59.6	86.6	68.8
	English – Comb	51.5 †	99.1	51.9	–	–	–	–	–	–
	French – Direct	65.2	99.8	65.3	32.2	100.0	32.2	81.8	100.0	81.8
	French – Prop	48.9	76.7	63.8	39.7	76.7	51.7	64.6	76.7	84.3

Table 1: Results on the SMiLER and FewRel benchmarks.

\dagger

indicates a statistically significant improvement of Comb over Direct (bootstrap test,

p<0.05

). Atomization significantly improves relation recall. But entity recall decreases. The accuracy increases for the small LLM (Qwen3-0.6B). Bold indicates the best result per extractor column for each benchmark/language.

	GLiREL			Qwen3-0.6B			Qwen3-4B
Pipeline	P	R	F1	P	R	F1	P	R	F1
Sentence	3.7	9.4	5.3	22.6	8.3	12.1	36.6	13.3	19.5
Document	1.4	14.8	2.5	20.3	15.7	17.7	32.0	24.4	27.7
Proposition	5.9	6.9	6.3	25.9	6.7	10.6	36.5	9.4	14.9
Sentence+Proposition	3.8	12.3	5.8	–	–	–	–	–	–
Document+Proposition	1.6	18.1	2.9	–	–	–	–	–	–

Table 2: Results on the DocRED benchmark. Atomization significantly improves precision and F1 when using a weaker extractor. However, when using LLM extraction, performance decreases.

	CoreNLP				Qwen3-0.6B				Qwen3-4B
Pipeline	P	R	F1	AUC	P	R	F1	AUC	P	R	F1	AUC
Direct	17.6	24.3	20.4	0.144	29.4	7.9	12.4	0.051	46.2	33.3	38.7	0.244
Prop	16.4	31.3	21.5	0.182	24.0	11.1	15.2	0.069	31.6	35.0	33.2	0.230
Union	–	–	–	–	23.7	14.7	18.1	0.091	30.8	43.6	36.1	0.285

Table 3: Results on the CaRB benchmark. Atomization significantly improves recall and F1 when a weaker extractor is used. However, when using LLM extraction, performance decreases.

Figure 5: The Knowledge Graph built with triplets extracted by GLiREL on the atomic propositions for the above sentence.

KG Construction

Knowledge graph construction turns text, tables, images, and databases into structured entities and relationships that machines can query and reason over.

See all 195 papers

LLMs Underperform Graph-Based Parsers on Supervised Relation Extraction for Complex Graphs
Apr 9, 2026Paolo Gajo, Domenic Rosati, Hassan Sajjad +1Relation Extraction LLM Information Extraction
University of Bologna · Dalhousie University
GLiNER-Relex: A Unified Framework for Joint Named Entity Recognition and Relation Extraction
May 11, 2026Ihor Stepanov, Oleksandr Lukashov, Mykhailo Shtopko +1Relation Extraction KG Construction
Knowledgator Engineering, Kyiv, Ukraine · Baldor Technologies Pvt. Ltd. (IDfy), Mumbai, India
A Multi-granularity Interaction Network for Relational Triple Extraction
Oct 18, 2026Xiaocheng Luo, Yanping Chen, Zaifu ZhanRelation Extraction Feature Interaction Modeling
School of Computer Science and Technology, Soochow University, Suzhou, 215006, Jiangsu, PR China · State Key Laboratory of Public Big Data, College of Computer Science and Technology, Guizhou University, Guiyang, 550025, Guizhou, PR China · Department of Electrical and Computer Engineering, University of Minnesota, Minneapolis, 55455, MN, USA

Apr 9, 2026J/K move · Enter open · S save

LLMs Underperform Graph-Based Parsers on Supervised Relation Extraction for Complex Graphs

Paolo Gajo, Domenic Rosati, Hassan Sajjad, Alberto Barrón-Cedeño

University of Bologna · Dalhousie University

Relation Extraction LLM Information Extraction LLM Evaluation

Open paper AskPDF arXiv

Relation extraction represents a fundamental component in the process of creating knowledge graphs, among other applications. Large language models (LLMs) have been adopted as a promising tool for relation extraction, both in supervised and in-context learning settings. However, in this work we show that their performance still lags behind much smaller architectures when the linguistic graph underlying a text has great complexity. To demonstrate this, we evaluate four LLMs against a graph-based parser on six relation extraction datasets with sentence graphs of varying sizes and complexities. Our results show that the graph-based parser increasingly outperforms the LLMs, as the number of relations in the input documents increases. This makes the much lighter graph-based parser a superior choice in the presence of complex linguistic graphs.

Abstract

KG Construction

Explore similar work

LLMs Underperform Graph-Based Parsers on Supervised Relation Extraction for Complex Graphs

GLiNER-Relex: A Unified Framework for Joint Named Entity Recognition and Relation Extraction

A Multi-granularity Interaction Network for Relational Triple Extraction