Worth Their Weights? Evaluating Generative Models for Discourse Parsing

Luke Gessler; Amir Zeldes

cs.CLOct 13, 2026

Worth Their Weights? Evaluating Generative Models for Discourse Parsing

Authors: Luke Gessler, Amir Zeldes

Organizations: Indiana University Bloomington Department of Linguistics · Georgetown University Department of Linguistics

Abstract

Rhetorical Structure Theory (RST) parsing has historically been dominated by discriminative models based on pretrained encoder-only language models. The recent availability of high-quality generative language models has made it unclear whether RST parsing is better approached as a generative task. In this work, we investigate whether generative language models are better at parsing RST than traditional discriminative ones. We find that at matched parameter counts, discriminative parsers perform much better than generative ones, though much larger generative models sometimes outperform them. Generative models hold a particular advantage in elementary discourse unit (EDU) segmentation, and we report new state-of-the-art results on the RST-DT benchmark in end-to-end parsing as well as segmentation.

Figures & tables

Figure 1: RST tree fragment from the GUM corpus: The most central EDU $[$ 69 $]$ has a less central purpose satellite, forming unit $[$ 69-70 $]$ which has a concession satellite, itself a temporal sequence of two EDUs.

RST tree: EDUs: $e_{1}{=}$ The cat sat. , $e_{2}{=}$ It was tired. , $e_{3}{=}$ So it slept. Structure: $\big((e_{1}\ e_{2})_{\textsc{sn}:\text{elaboration}}\ \ e_{3}\big)_{\textsc{ns}:\text{background}}$ .
Fine-tuned generative parsers
Shift-reduce ( sr ): The cat sat . <shift> It was tired . <shift> <reduce_sn_elaboration> So it slept . <shift> <reduce_ns_background>
S-expression ( sexp ): { { { The cat sat . } { It was tired . } SN elaboration <label_end> } { So it slept . } NS background <label_end> }
In-context learning
Segmentation output: The cat sat . \|\| It was tired . \|\| So it slept .
Parsing output: The cat sat . <shift> It was tired . <shift> SN elaboration <label_end> So it slept . <shift> NS background <label_end>

Figure 2: A three-EDU tree and its serializations. For the fine-tuned generative parsers, every element in < > is a single special token added to the PLM’s tokenizer. ICL takes two calls: the model marks EDU boundaries in the raw text, then parses its own segmented output. For ICL, < > encloses ordinary strings.

Discriminative (LoRA fine-tuned encoder)
		RST-DT (coarse)					GUM (fine)
System	PLM	Seg	S	N	R	F	Seg	S	N	R	F
dmrst	Ettin-150M	96.0	72.5	60.9	49.9	48.6	93.7	66.6	52.8	40.5	40.3
dmrst	Ettin-400M	96.7	75.2	64.6	54.3	52.5	94.7	72.3	58.4	46.7	46.5
dmrst	Ettin-1B	96.5	75.7	64.5	53.5	52.0	94.8	71.1	58.0	46.5	46.3
Generative (LoRA fine-tuned LM)

Table 1: End-to-end parsing results on RST-DT and GUM.

Discriminative (LoRA fine-tuned encoder)
		RST-DT (coarse)				GUM (fine)
System	PLM	S	N	R	F	S	N	R	F
TopDown	Ettin-400M	78.3	67.4	56.2	54.8	76.9	63.1	50.3	50.0
ShiftReduce	Ettin-400M	77.0	66.5	54.7	53.5	76.0	62.7	50.7	50.3
dmrst	Ettin-150M	76.3	65.2	54.2	52.6	72.5	59.5	47.1	46.8
dmrst	Ettin-400M	78.0	68.1	57.3	55.5	76.4	63.7	51.5	51.3

Table 2: Gold EDU parsing results on RST-DT and GUM.

	RST-DT						GUM
	dmrst (enc.)			DecOnly-sr (dec.)			dmrst (enc.)			DecOnly-sr (dec.)
Ettin	Seg	E2E	Gold	Seg	E2E	Gold	Seg	E2E	Gold	Seg	E2E	Gold
150M	96.0	48.6	52.6	92.6	31.5	38.3	93.7	40.3	46.8	89.6	19.2	26.1
400M	96.7	52.5	55.5	94.5	35.3	41.3	94.7	46.5	51.3	91.9	22.5	27.6
1B	96.5	52.0	56.5	95.7	39.6	44.4	94.8	46.3	51.4	93.0	27.9	32.5

Table 3: Parsing results using matched Ettin encoders and decoders: segmentation

F_{1}

(Seg), and Full end to end (E2E) and with gold EDUs (Gold).

Basque (RST Basque TreeBank)
		End-to-end					Gold EDU
System	PLM	Seg	S	N	R	F	S	N	R	F
dmrst	XLM-R base	88.1	62.5	42.3	28.2	27.4	71.1	50.1	33.4	33.1
EncDec-sr	T5Gemma 2 1B-1B	89.4	52.5	32.8	22.2	21.9	58.1	34.6	24.3	24.0
DecOnly-sr	Qwen 3.6 27B	92.7	58.8	31.8	19.2	18.6	61.3	33.4	19.4	19.1
German (PCC)

Table 4: Parsing results on non-English languages.

Figure 3: Label confusion matrices for top Ettin decoder and encoder models on RST-DT.

Panel of Figure 4 — Figure 4: Attachment accuracy on GUM with gold EDUs, by relation explicitness (bars, 95% CIs) and modality (lines). icl-f covers the 17 documents it parsed.

Appendix figures & tables8 assets

Supplementary material from the paper’s appendix.

Appendix

	TopDown	ShiftReduce	dmrst
Optimizer	AdamW
Weight decay	0.01
LoRA rank	16
LoRA $\alpha$	32
LoRA dropout	0.05
LoRA targets	all linear

Appendix

Table 5: Discriminative parser settings for the main table (LoRA) runs. No LoRA bias terms are trained and DoRA is off. † dmrst dropout: 0.5 (encoder, decoder, labeler), 0.2 (document GRU), 0.3 (segmenter).

TopDown (XLM-R large)
	S	N	R	F
FFT	78.6	66.5	56.5	54.3
LoRA	77.4	66.2	56.3	54.1
ShiftReduce (XLM-R large)
FFT	76.5	66.2	55.2	53.5
LoRA	76.9	66.6	56.4	54.2

Appendix

Table 6: Discriminative parsers under full fine-tuning (FFT) and under the LoRA recipe of Table 5 , on RST-DT, gold EDU.

Setting	Value
Optimizer	Adafactor ‡
Learning rate	$3\times 10^{-4}$
LR schedule	warmup–stable–decay (see text)
Weight decay	0.05
Label smoothing	0.1
Gradient clip	1.0

Appendix

Table 7: Hyperparameters shared by all generative parsers. ‡ Adafactor with relative_step and scale_parameter disabled, so the external schedule sets the rate. † On the multimodal Gemma 4 31B, the adapter regex targets the text stack only, excluding the vision and audio towers.

System	PLM	greedy	beam	greedy	beam	$\Delta$ F
		Seg		F
EncDec-sr	T5Gemma 2 1B-1B	97.8	97.9	51.4	52.2	$+0.8$
EncDec-sexp	T5Gemma 2 1B-1B	95.1	96.0	46.0	47.5	$+1.5$
DecOnly-sr	Gemma 3 1B	96.7	96.6	45.7	45.2	$-0.5$
DecOnly-sexp	Gemma 3 1B	92.5	94.0	34.4	34.3	$-0.1$
DecOnly-sr	Gemma 4 31B	98.2	98.2	55.1	54.8	$-0.4$

Appendix

Table 8: Beam search ablation within the generative family: RST-DT end-to-end, identical checkpoints decoded greedily and with legality-constrained beam search (beam size 6). Greedy is the setting used in Tables 1 and 2 . Segmentation and Full are shown per decoding, with

\Delta

F the beam-minus-greedy difference computed before rounding.

S_{0,0}\rightarrow

The cat sat . <shift>

S_{1,1}

S_{1,1}\rightarrow

It was tired . <shift>

S_{2,2}

S_{2,2}\rightarrow

So it slept . <shift>

S_{3,3}

\mid

label

S_{2,1}

S_{2,1}\rightarrow

So it slept . <shift>

S_{3,2}

S_{3,3}\rightarrow

label

S_{3,2}

S_{3,2}\rightarrow

label

S_{3,1}

S_{3,1}\rightarrow\varepsilon

Appendix

Figure 6: The structure prediction grammar compiled for the three-EDU document of Figure 2 . Nonterminal

S_{s,d}

is the parser state after

s

shifts with

d

subtrees on the stack, the start symbol is

S_{0,0}

, and derivations terminate at

S_{n,1}

. Shift productions carry the text of the EDU they consume, and label expands to one string per nuclearity–relation pair.

Figure 7: Label confusion matrices for top ICL models on RST-DT.

	RST-DT			GUM
	sent.	punct.	none	sent.	punct.	none
dmrst 400M	99.9	95.9	93.4	99.8	94.2	92.6
dmrst 1B	99.8	97.7	92.8	99.9	92.7	91.4
Ettin-1B dec.	99.7	96.8	91.0	99.9	89.6	86.5
Gemma 3 1B	99.7	97.3	93.6	99.7	91.3	89.6
T5Gemma 2	99.8	98.8	96.7	99.9	92.8	92.6

Appendix

Table 9: End-to-end segmentation recall by the token preceding each gold boundary: sentence-final punctuation (sent.), other punctuation (punct.), or none. Predictions are projected onto gold tokens as in Section 4.3 , the generative rows are sr parsers, the Ettin-1B decoder (dec.) is the DecOnly-sr parser of Table 3 , and

n

is the number of gold boundaries.

	RST-DT						GUM
Width (EDUs)	2	3–4	5–8	9–16	17–32	33+	2	3–4	5–8	9–16	17–32	33+
dmrst 400M	92.7	85.8	70.7	52.9	47.6	29.1	89.6	80.6	71.4	61.8	52.7	46.6
dmrst 1B	93.8	85.7	70.7	58.7	54.5	33.0	90.0	80.6	71.7	58.5	44.7	44.7
Ettin-1B dec.	91.9	76.7	46.7	24.0	16.1	32.0	84.0	68.5	41.1	26.7	15.4	28.1
Gemma 3 1B	91.3	78.8	52.8	32.7	24.5	33.0	84.4	71.4	46.2	22.8	16.0	32.8
T5Gemma 2	93.2	84.9	61.2	42.3	37.8	26.2	90.4	77.9	58.3	36.2	23.4	37.2

Appendix

Table 10: Gold EDU span recall by constituent width in EDUs, excluding each document’s root. Rows are as in Table 9 , and

n

is the number of gold constituents.

Language Modeling

Language modeling develops systems that learn patterns in text and other sequences to understand, predict, and generate language.

See all 519 papers

RAG-RST: Retrieved Exemplars as In-Context Discourse Schema for RST Parsing
Oct 15, 2026Mohammadreza Sediqin, Shlomo Engelson ArgamonRetrieval-Augmented ICL
Illinois Institute of Technology New York, USA · Touro University New York, USA
Japanese Discourse Structure Analysis Based on Segmented Discourse Representation Theory Using LLMs
Oct 27, 2026Satomu Tomisaki, Yosuke Mikami, Amane Watahiki +1Multilingual Language Model Evaluation
The University of Tokyo · RIKEN · Tohoku University
Scoring Both Directions: LLMs realize the MRS they cannot reliably parse
Sep 24, 2026Soham DanLLM Evaluation Semantic Parsing
Scale AI

Oct 15, 2026J/K move · Enter open · S save

RAG-RST: Retrieved Exemplars as In-Context Discourse Schema for RST Parsing

Mohammadreza Sediqin, Shlomo Engelson Argamon

Illinois Institute of Technology New York, USA · Touro University New York, USA

Retrieval-Augmented ICL

Open paper AskPDF arXiv

In Rhetorical Structure Theory (RST), each edge of a discourse tree carries a nuclearity label, marking which span holds the central content, and a relation drawn from a closed inventory of functional categories. Neither label is a surface property: the same two clauses can be Elaboration or Background depending on rhetorical intent, and the boundary between them is fixed by annotation convention rather than by lexical cues. Predicting these labels therefore requires a formal discourse schema, and exposure to discourse-rich pretraining text does not supply it: given only a target EDU pair, a strong LLM recovers nuclearity at 0.794 F1 and relation at 0.738 F1. We propose RAG-RST, a retrieval-augmented framework that supplies this schema as in-context demonstration rather than through parameter updates, raising nuclearity to 0.846 F1 and relation to 0.788 F1 with no training. Two configurations are compared: a precomputed strategy that labels all EDU pairs before parsing, and a dynamic strategy that retrieves at each reduce step, so that the exemplars reaching the generator depend on the reductions the parser has already committed to. Both outperform existing RST parsers across span, nuclearity, relation, and full tree metrics, with the dynamic variant consistently strongest. Two properties of the retrieved schema emerge: its utility is bounded in number, saturating and then declining beyond fifty exemplars, and it depends on structural position. Both indicate that a formal linguistic schema can be supplied to a foundation model as a demonstration rather than trained into its parameters.

Abstract

Language Modeling

Explore similar work

RAG-RST: Retrieved Exemplars as In-Context Discourse Schema for RST Parsing

Japanese Discourse Structure Analysis Based on Segmented Discourse Representation Theory Using LLMs

Scoring Both Directions: LLMs realize the MRS they cannot reliably parse