THIVLVC: Retrieval Augmented Dependency Parsing for Latin

Luc Pommeret; Thibault Wagret; Jules Deret

cs.CLApr 7, 2026

THIVLVC: Retrieval Augmented Dependency Parsing for Latin

Authors: Luc Pommeret, Thibault Wagret, Jules Deret

Organizations: STL · Université Paris-Saclay, CNRS, LISN, 91400, Orsay, France · ENS de Lyon, HiSoMA · École Normale Supérieure de Lyon, HISOMA, 69007, Lyon, France

Abstract

We describe THIVLVC, a two-stage system for the EvaLatin 2026 Dependency Parsing task. Given a Latin sentence, we retrieve structurally similar entries from the CIRCSE treebank using sentence length and POS n-gram similarity, then prompt a large language model to refine the baseline parse from UDPipe using the retrieved examples and UD annotation guidelines. We submit two configurations: one without retrieval and one with retrieval (RAG). On poetry (Seneca), THIVLVC improves CLAS by +17 points over the UDPipe baseline; on prose (Thomas Aquinas), the gain is +1.5 CLAS. A double-blind error analysis of 300 divergences between our system and the gold standard reveals that, among unanimous annotator decisions, 53.3% favour THIVLVC, showing annotation inconsistencies both within and across treebanks.

Figures & tables

Figure 1: Overview of the THIVLVC pipeline. The input sentence is processed in parallel by the structural retriever (which selects similar examples from CIRCSE) and by UDPipe (which produces a baseline dependency parse). Both outputs, together with the UD guidelines, are passed to the LLM for refinement.

Dataset	Strategy	Length Diff	POS Overlap
Prose	TF-IDF	$13.24\pm 12.09$	$0.421\pm 0.150$
	Morphological	$12.86\pm 14.78$	$\mathbf{0.519}\pm 0.138$
	Structural	$\mathbf{0.76}\pm 1.07$	$0.512\pm 0.142$
Poetry	TF-IDF	$11.60\pm 15.95$	$0.454\pm 0.177$
	Morphological	$14.51\pm 20.63$	$0.556\pm 0.163$
	Structural	$\mathbf{1.13}\pm 3.02$	$\mathbf{0.601}\pm 0.196$

Table 1: Retrieval strategies comparison. Knowledge base: CIRCSE train (762 sentences).

k=5

retrievals per query.

		With subtypes		No subtypes
System	Genre	CLAS	LAS	CLAS	LAS
UDPipe	Poetry	56.94	57.22	57.24	59.74
UDPipe	Prose	79.41	82.17	83.07	85.21
THIVLVC_1	Poetry	72.71	70.36	76.00	76.97
THIVLVC_1	Prose	74.04	75.72	81.52	82.78
THIVLVC_2	Poetry	74.03	72.88	76.08	77.60

Table 2: THIVLVC system comparison. THIVLVC_1 = LLM + UDPipe + UD Guidelines. THIVLVC_2 = THIVLVC_1 + RAG on CIRCSE.

Verdict	Ann. 1		Ann. 2
	$n$	%	$n$	%
Gold (human)	137	45.7	110	36.7
THIVLVC (system)	126	42.0	143	47.7
Both wrong	7	2.3	11	3.7
Undecidable	11	3.7	21	7.0
Don’t know	19	6.3	15	5.0

Table 3: Blind evaluation verdicts per annotator (300 items).

Appendix figures & tables8 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 2: Annotation interface: overview. The Latin sentence is displayed at the top; two anonymized parse options (A and B) are shown side by side with their ID, word form, head, and relation columns.

Figure 3: Annotation interface: verdict buttons. Divergent rows are highlighted. Annotators choose among five categories.

Verdict	$n$	%
THIVLVC better than Gold	89	53.3
Gold better than THIVLVC	78	46.7
Decided	167
Undecidable	3	1.7
Don’t know	3	1.7
Total	173

Appendix

Table 4: Unanimous consensus: items where both annotators agree (173/300, 57.7%).

Error type	Gold err.		THIVLVC err.
	$n$	%	$n$	%
Wrong head + relation	26	29.2	14	17.9
Subtype confusion	23	25.8	29	37.2
Wrong head only	20	22.5	8	10.3
Wrong relation only	17	19.1	24	30.8
Head + subtype	3	3.4	3	3.8

Appendix

Table 5: Taxonomy of errors by source (Gold vs. THIVLVC ), based on 167 unanimous decided cases.

Gold errors ( THIVLVC correct)			THIVLVC errors (Gold correct)
Wrong	Correct	$n$	Wrong	Correct	$n$
obl	obl:arg	6	amod	acl	8
obl	obl:lmod	6	advcl	advcl:cmp	8
amod	root	4	advmod	advmod:tmod	7
obl	nummod	4	advmod:lmod	advmod	7
conj	parataxis	3	advmod	discourse	4

Appendix

Table 6: Most frequent label confusions by error source.

Genre	Gold	THIVLVC	Total	THIVLVC %
Poetry	49	50	99	50.5
Prose	29	39	68	57.4
All	78	89	167	53.3

Appendix

Table 7: Consensus results by genre (167 decided cases).

Metric	All categories	Gold/ THIVLVC only
Items evaluated	300	224
Observed agreement ( $p_{o}$ )	0.577	0.746
Expected agreement ( $p_{e}$ )	0.374	0.499
Cohen’s $\kappa$	0.324	0.493

Appendix

Table 8: Inter-annotator agreement on 300 blind evaluation items.

\kappa

is Cohen’s kappa.

	Gold	THIVLVC	Both	Undec.	DK	Total
Gold	78	37	6	8	8	137
THIVLVC	20	89	3	10	4	126
Both	3	4	0	0	0	7
Undec.	3	5	0	3	0	11
DK	6	8	2	0	3	19
Total	110	143	11	21	15	300

Appendix

Table 9: Confusion matrix between annotators (300 items). Rows = Ann. 1, Columns = Ann. 2.

Dependency Parsing

Dependency parsing identifies grammatical relationships between words and represents sentence structure as a tree.

See all 24 papers

Correction as Annotation: Bootstrapping a Dependency Parser for Documentary Medieval Latin
Sep 5, 2026Gabriel H. PizzornoDependency Parsing Low-Resource Language Processing
Harvard University · Department of History | Robinson Hall, Office 210 | 35 Quincy Street, Cambridge, MA 02138
ThaiTrees: Thai Syntactic Dependency Trees Across Domains
Sep 23, 2026Attapol T. Rutherford, Papatchol ThientongCorpus Linguistics Low-Resource Language Processing
Department of Linguistics Chulalongkorn University
A Reproducible Universal Dependencies-Style Pipeline for Katharevousa Greek Parliamentary Text
May 21, 2026George Mikros, Fotios FitsilisLow-Resource Language Processing Dependency Parsing
Hamad Bin Khalifa University · Universidad Austral

Sep 5, 2026J/K move · Enter open · S save

Correction as Annotation: Bootstrapping a Dependency Parser for Documentary Medieval Latin

Gabriel H. Pizzorno

Harvard University · Department of History | Robinson Hall, Office 210 | 35 Quincy Street, Cambridge, MA 02138

Dependency Parsing Low-Resource Language Processing Human-in-the-Loop Annotation

Open paper AskPDF arXiv

Medieval documentary sources remain inadequately served by existing natural language processing tools. None of the five readily available Latin treebank models attains usable performance on a collection of 160 inventories compiled in Marseille between 1258 and 1446. The best labelled attachment score is 0.62 and the best morphology-aware score is 0.24. Performance does not correlate with either genre or period proximity. To address this shortfall, in-domain training data was generated as a by-product of using these inadequate models. In each of nine iterations, a model pre-annotated 200 sentences; an expert corrected the annotations; and the corrected sentences were used to train the subsequent model, with batches sampled independently of model state, without active-learning selection. Thirty-three hours of annotation effort over 1,804 sentences increased universal part-of-speech accuracy from 0.80 to 0.98 and labelled attachment from 0.48 to 0.92, outperforming all baselines on the reported metrics while using 97% less training data than the largest one of them. Annotator effort declined from 54% of tokens to a plateau of 14-18%, an operational progress metric that requires no separate gold standard and can serve as a stopping criterion.

Abstract

Dependency Parsing

Explore similar work

Correction as Annotation: Bootstrapping a Dependency Parser for Documentary Medieval Latin

ThaiTrees: Thai Syntactic Dependency Trees Across Domains

A Reproducible Universal Dependencies-Style Pipeline for Katharevousa Greek Parliamentary Text