Coupling Local Context and Global Semantic Prototypes via a Hierarchical Architecture for Rhetorical Roles Labeling

Anas Belfathi; Nicolas Hernandez; Laura Monceaux; Warren Bonnard; Mary Catherine Lavissiere; Christine Jacquin; Richard Dufour

cs.CLMar 4, 2026

Coupling Local Context and Global Semantic Prototypes via a Hierarchical Architecture for Rhetorical Roles Labeling

Authors: Anas Belfathi, Nicolas Hernandez, Laura Monceaux, Warren Bonnard, Mary Catherine Lavissiere, Christine Jacquin, Richard Dufour

Organizations: Nantes Université, École Centrale Nantes, CNRS, LS2N, UMR 6004, F-44000 Nantes, France · University of Lorraine, France

Abstract

Rhetorical Role Labeling (RRL) identifies the functional role of each sentence in a document, a key task for discourse understanding in domains such as law and medicine. While hierarchical models capture local dependencies effectively, they are limited in modeling global, corpus-level features. To address this limitation, we propose two prototype-based methods that integrate local context with global representations. Prototype-Based Regularization (PBR) learns soft prototypes through a distance-based auxiliary loss to structure the latent space, while Prototype-Conditioned Modulation (PCM) constructs corpus-level prototypes and injects them during training and inference. Given the scarcity of RRL resources, we introduce SCOTUS-Law, the first dataset of U.S. Supreme Court opinions annotated with rhetorical roles at three levels of granularity: category, rhetorical function, and step. Experiments on legal, medical, and scientific benchmarks show consistent improvements over strong baselines, with 4 Macro-F1 gains on low-frequency roles. We further analyze the implications in the era of Large Language Models and complement our findings with expert evaluation.

Figures & tables

Figure 1: Example of a document segments in SCOTUS-Law , annotated with discursive categories , rhetorical functions , and attributes , which together form the step annotation (full hierarchy in Figure 6 ).

Figure 2: Illustration of our methods for injecting global representations into hierarchical architectures. PBR (left) learns soft prototypes jointly with the model to regularize the latent space. PCM (right) dynamically injects precomputed role prototypes during encoding via modulation mechanisms.

Corpus-level statistics
Statistic	Train	Dev	Test
# Documents	144	18	18
Total # Sentences	21,396	2,450	2,481
Avg. # Sentences / Doc	148.58	136.11	137.83
Avg. # Tokens / Sentence	22.95	21.43	22.15
Sentence distribution by rhetorical function

Table 1: Descriptive statistics for the SCOTUS-Law dataset at the rhetorical function level.

	Legal										Medical		Scientific
	Scotus Category		Scotus RF		Scotus Steps		LegalEval		DeepRhole		PubMed		CS-Abstracts
	mF1	wF1	mF1	wF1	mF1	wF1	mF1	wF1	mF1	wF1	mF1	wF1	mF1	wF1
Baseline (HSLN)	82.22	88.35	61.36	78.81	46.70	63.21	78.82	90.94	44.24	50.51	87.01	91.09	68.55	75.08
Mind T.y.s.s. et al. (2024)	83.46	89.20	62.67	79.07	45.24	62.78	79.80	91.25	45.30	50.93	87.67	91.86	69.19	76.91
PBR	83.69 ‡	89.75 ‡	65.75 ‡	80.31 ‡	50.48 ‡	65.73 ‡	82.50 ‡	93.17 ‡	44.96 †	51.11 †	88.86 ‡	92.91 ‡	71.10 ‡	78.09 ‡
PCM (Full Sampling)	83.96 ‡	89.80 ‡	67.53 ‡	80.64 ‡	54.03 ‡	67.54 ‡	81.41 ‡	91.21	47.13 ‡	55.54 ‡	87.19	91.89	69.84	76.66

Table 2: Macro-F1 and Weighted-F1 scores across domains for the baseline, PBR, and PCM (with different sampling strategies). An gold prototypes experiment is also included, selecting the optimal prototype post-hoc for each sentence. † and ‡ indicate statistical significance over the baseline at

0.05

and

0.01

, respectively.

Rhetorical Function	%	Baseline	+PCM	$\Delta$
Accepting arguments/a reasoning	$0.4$	$15.40$	$57.15$	$\uparrow$ 41.75
Announcing	$1.3$	$68.98$	$76.93$	$\uparrow$ 7.95
Citing	$2.4$	$85.99$	$89.92$	$\uparrow$ 3.93
Describing	$3.6$	$61.04$	$61.41$	$\uparrow$ 0.37
Evaluating the impact of the decision	$0.2$	$0.00$	$0.00$	0.00
Giving instructions to competent courts	$0.4$	$52.18$	$56.01$	$\uparrow$ 3.83

Table 3: Role-wise F1 on SCOTUS RF . The % column indicates the proportion of each rhetorical function in the corpus.

Panel of Figure 3 — Figure 3: t-SNE projection of sentence embeddings under baseline, PBR, and PCM.

	Legal		Medical		Scientific
	SCOTUS RF		PubMed		CS-Abstracts
	mF1	wF1	mF1	wF1	mF1	wF1
DeepSeek-70B	65.20	75.20	81.03	86.67	64.92	72.60
Meta-Llama3-8B	66.78	75.09	82.51	87.63	67.39	74.59
Mistral-7B	70.29	76.61	81.86	87.20	62.24	70.61
Qwen3-8B	69.36	75.53	81.73	87.26	65.93	74.93

Table 4: Performance of LLMs Fine-Tuned with QLoRA vs. Prototype-Based Methods (Ours).

Input Excerpt	Confused Role Pair	Gold Label	Baseline Prediction	PCM Prediction	Error Reduction	Expert Assessment
As we have explained , failure to comply with §262(l)(2)(A) is not an act of artificial infringement.	Recalling $\leftrightarrow$ Stating the Court’s reasoning	Recalling	Stating the Court’s reasoning	Recalling	19.75%	This case is ambiguous because “As we have explained” signals a backward reference, while the remainder conveys a definitive judicial conclusion, typical of Stating the Court’s reasoning.
Thus, in Los Angeles Cloak Joint Board ILGWU (Helen Rose Co.), 127 N. L. R. B. 1543 (1960), the Board held that §8(b)(1)(B) barred a union from picketing a company in an attempt to force the employer to…	Describing $\leftrightarrow$ Recalling	Recalling	Describing	Recalling	6.56%	The difficulty arises because “the Board held that” points to recalling a precedent, yet the remainder paraphrases substantive content neutrally, resembling Describing.
The American Bar Association recommends defense counsel “promptly communicate and explain to the defendant all plea offers made by the prosecuting attorney,” ABA Standards for Criminal Justice, Pleas of Guilty 14–3.2(a) (3d ed. 1999),	Quoting $\leftrightarrow$ Recalling	Recalling	Quoting	Recalling	33.33%	Ambiguity stems from combining an attribution verb (“recommends”) with a direct quotation, blending narrative recall with verbatim citation and blurring the distinction between Recalling and Quoting.

Table 5: Expert evaluation of model predictions on ambiguous rhetorical role pairs. PCM reduces baseline errors by better resolving overlaps between semantically similar functions.

Model	Annotation Difficulty (Likert Scale)
	Easy	Rather easy	Rather difficult	Difficult
PCM (Best)	100	79.62	61.53	16.66

Table 6: Accuracy on hardness Likert scale annotated by experts.

Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

Panel of Figure 4 — Figure 4: Topical, Temporal, and Authorial Diversity in our annotated corpus.

Figure 5: Distribution of Rhetorical Functions by Relative Position, revealing a structured rhetorical flow in judicial reasoning—from the initial announcement to the final resolution.

Figure 6: The final coding scheme is composed of 5 categories (ovals with orange background), 13 rhetorical functions (green rectangles) and 24 attributes (types in blue rectangles, target in the yellow rectangle, and author in the purple rectangle. The scheme reads from top to bottom: A step label is constructed by first choosing a category, then a rhetorical function, then if required, by combining attributes to complete the discursive information provided by the rhetorical function.

Figure 7: The hierarchical architecture.

Dataset	Source	Domain	Language	# Docs	# Sents	Labels
Scotus Category	Ours	Legal (U.S.)	English	180	26,327	5
Scotus RF	Ours	Legal (U.S.)	English	180	26,327	13
Scotus Steps	Ours	Legal (U.S.)	English	180	26,327	35
LegalEval	Kalamkar et al. (2022)	Legal (India)	English	214	31,865	13
DeepRhole	Bhattacharya et al. (2023a)	Legal (India)	English	50	9,380	7
PubMed	Dernoncourt and Lee (2017)	Medical	English	20,000	227,000	5

Appendix

Table 8: Evaluation datasets used in our experiments. Scotus is annotated at three hierarchical levels: category, rhetorical function, and steps.

Method	Scotus RF	LegalEval	PubMed
Linear Fusion	80.89	91.62	91.91
Conditional Layer Norm	78.11	87.49	92.74
Cross-Attention Fusion	79.30	87.74	92.20
Feature-wise Linear Mod.	74.71	76.74	92.74
Gated Residual Addition	79.58	89.06	92.79

Appendix

Table 9: W-F1 scores for prototype injection strategies. All variants share the same hierarchical encoder with PCM integration.

Panel of Figure 8 — Figure 8: Effect of PBR hyperparameters on w-F1 at the SCOTUS RF Dashed lines indicate the baseline without prototypes.

Prototype Learning

Prototype learning represents data with representative examples or feature centers to support classification, adaptation, anomaly detection, and interpretable predictions.

See all 80 papers

Semantic Reranking at Inference Time for Hard Examples in Rhetorical Role Labeling
May 18, 2026Anas Belfathi, Nicolas Hernandez, Laura Monceaux +2LLM Reranking
Nantes Université, École Centrale Nantes, CNRS, LS2N, UMR 6004, F-44000 Nantes, France · University of Lorraine, France
HKJudge: A Legal Discourse-Annotated Corpus for Interpreting What Courts Find, How They Reason, and What They Rule
Jun 4, 2026Xi Xuan, Wenxin Zhang, Yufei Zhou +2Legal NLP Document Information Extraction
City University of Hong Kong · University of Chinese Academy of Sciences
A Systematic Survey of Semantic Role Labeling in the Era of Pretrained Language Models
Feb 9, 2025Huiyao Chen, Meishan Zhang, Jing Li +4
1Harbin Institute of Technology (Shenzhen) 2 Shenzhen Loop Area Institute (SLAI) · 1Harbin Institute of Technology (Shenzhen) · University of Oslo +1

May 18, 2026J/K move · Enter open · S save

Semantic Reranking at Inference Time for Hard Examples in Rhetorical Role Labeling

Anas Belfathi, Nicolas Hernandez, Laura Monceaux, Warren Bonnard+1

Nantes Université, École Centrale Nantes, CNRS, LS2N, UMR 6004, F-44000 Nantes, France · University of Lorraine, France

LLM Reranking

Open paper AskPDF arXiv

Rhetorical Role Labeling (RRL) assigns a functional role to each sentence in a document and is widely used in legal, medical, and scientific domains. While language models (LMs) achieve strong average performance, they remain unreliable on hard examples, where prediction confidence is low. Existing approaches typically handle uncertainty implicitly and treat labels as discrete identifiers, overlooking the semantic information encoded in label names. We introduce RISE, an inference-time semantic reranking framework that leverages label semantics to refine predictions on hard instances. RISE automatically identifies low-confidence predictions and reranks model outputs using contrastively learned label representations, without retraining or modifying the underlying model. Experiments on eight domain-specific RRL datasets with seven LMs, including encoder-based and causal architectures, show an average gain of +9.15 macro-F1 points on hard examples. For explainability, we further propose manual hardness annotations to study difficulty from both model and human perspectives, revealing a moderate agreement with Cohen's kappa = 0.40.

Abstract

Prototype Learning

Explore similar work

Semantic Reranking at Inference Time for Hard Examples in Rhetorical Role Labeling

HKJudge: A Legal Discourse-Annotated Corpus for Interpreting What Courts Find, How They Reason, and What They Rule

A Systematic Survey of Semantic Role Labeling in the Era of Pretrained Language Models