Rhetorical Role Labeling (RRL) identifies the functional role of each sentence in a document, a key task for discourse understanding in domains such as law and medicine. While hierarchical models capture local dependencies effectively, they are limited in modeling global, corpus-level features. To address this limitation, we propose two prototype-based methods that integrate local context with global representations. Prototype-Based Regularization (PBR) learns soft prototypes through a distance-based auxiliary loss to structure the latent space, while Prototype-Conditioned Modulation (PCM) constructs corpus-level prototypes and injects them during training and inference. Given the scarcity of RRL resources, we introduce SCOTUS-Law, the first dataset of U.S. Supreme Court opinions annotated with rhetorical roles at three levels of granularity: category, rhetorical function, and step. Experiments on legal, medical, and scientific benchmarks show consistent improvements over strong baselines, with 4 Macro-F1 gains on low-frequency roles. We further analyze the implications in the era of Large Language Models and complement our findings with expert evaluation.
Figures & tables
Figure 1: Example of a document segments in SCOTUS-Law , annotated with discursive categories , rhetorical functions , and attributes , which together form the step annotation (full hierarchy in Figure 6 ).
Figure 2: Illustration of our methods for injecting global representations into hierarchical architectures. PBR (left) learns soft prototypes jointly with the model to regularize the latent space. PCM (right) dynamically injects precomputed role prototypes during encoding via modulation mechanisms.
Corpus-level statistics
Statistic
Train
Dev
Test
# Documents
144
18
18
Total # Sentences
21,396
2,450
2,481
Avg. # Sentences / Doc
148.58
136.11
137.83
Avg. # Tokens / Sentence
22.95
21.43
22.15
Sentence distribution by rhetorical function
Table 1: Descriptive statistics for the SCOTUS-Law dataset at the rhetorical function level.
Legal
Medical
Scientific
Scotus Category
Scotus RF
Scotus Steps
LegalEval
DeepRhole
PubMed
CS-Abstracts
mF1
wF1
mF1
wF1
mF1
wF1
mF1
wF1
mF1
wF1
mF1
wF1
mF1
wF1
Baseline (HSLN)
82.22
88.35
61.36
78.81
46.70
63.21
78.82
90.94
44.24
50.51
87.01
91.09
68.55
75.08
Mind T.y.s.s. et al. (2024)
83.46
89.20
62.67
79.07
45.24
62.78
79.80
91.25
45.30
50.93
87.67
91.86
69.19
76.91
PBR
83.69 ‡
89.75 ‡
65.75 ‡
80.31 ‡
50.48 ‡
65.73 ‡
82.50 ‡
93.17 ‡
44.96 †
51.11 †
88.86 ‡
92.91 ‡
71.10 ‡
78.09 ‡
PCM (Full Sampling)
83.96 ‡
89.80 ‡
67.53 ‡
80.64 ‡
54.03 ‡
67.54 ‡
81.41 ‡
91.21
47.13 ‡
55.54 ‡
87.19
91.89
69.84
76.66
Table 2: Macro-F1 and Weighted-F1 scores across domains for the baseline, PBR, and PCM (with different sampling strategies). An gold prototypes experiment is also included, selecting the optimal prototype post-hoc for each sentence. † and ‡ indicate statistical significance over the baseline at 0.05 and 0.01 , respectively.
Rhetorical Function
%
Baseline
+PCM
Δ
Accepting arguments/a reasoning
0.4
15.40
57.15
↑ 41.75
Announcing
1.3
68.98
76.93
↑ 7.95
Citing
2.4
85.99
89.92
↑ 3.93
Describing
3.6
61.04
61.41
↑ 0.37
Evaluating the impact of the decision
0.2
0.00
0.00
0.00
Giving instructions to competent courts
0.4
52.18
56.01
↑ 3.83
Table 3: Role-wise F1 on SCOTUS RF . The % column indicates the proportion of each rhetorical function in the corpus.
Figure 3: t-SNE projection of sentence embeddings under baseline, PBR, and PCM.
Legal
Medical
Scientific
SCOTUS RF
PubMed
CS-Abstracts
mF1
wF1
mF1
wF1
mF1
wF1
DeepSeek-70B
65.20
75.20
81.03
86.67
64.92
72.60
Meta-Llama3-8B
66.78
75.09
82.51
87.63
67.39
74.59
Mistral-7B
70.29
76.61
81.86
87.20
62.24
70.61
Qwen3-8B
69.36
75.53
81.73
87.26
65.93
74.93
Table 4: Performance of LLMs Fine-Tuned with QLoRA vs. Prototype-Based Methods (Ours).
Input Excerpt
Confused Role Pair
Gold Label
Baseline Prediction
PCM Prediction
Error Reduction
Expert Assessment
As we have explained , failure to comply with §262(l)(2)(A) is not an act of artificial infringement.
Recalling ↔ Stating the Court’s reasoning
Recalling
Stating the Court’s reasoning
Recalling
19.75%
This case is ambiguous because “As we have explained” signals a backward reference, while the remainder conveys a definitive judicial conclusion, typical of Stating the Court’s reasoning.
Thus, in Los Angeles Cloak Joint Board ILGWU (Helen Rose Co.), 127 N. L. R. B. 1543 (1960), the Board held that §8(b)(1)(B) barred a union from picketing a company in an attempt to force the employer to…
Describing ↔ Recalling
Recalling
Describing
Recalling
6.56%
The difficulty arises because “the Board held that” points to recalling a precedent, yet the remainder paraphrases substantive content neutrally, resembling Describing.
The American Bar Association recommends defense counsel “promptly communicate and explain to the defendant all plea offers made by the prosecuting attorney,” ABA Standards for Criminal Justice, Pleas of Guilty 14–3.2(a) (3d ed. 1999),
Quoting ↔ Recalling
Recalling
Quoting
Recalling
33.33%
Ambiguity stems from combining an attribution verb (“recommends”) with a direct quotation, blending narrative recall with verbatim citation and blurring the distinction between Recalling and Quoting.
Table 5: Expert evaluation of model predictions on ambiguous rhetorical role pairs. PCM reduces baseline errors by better resolving overlaps between semantically similar functions.
Model
Annotation Difficulty (Likert Scale)
Easy
Rather easy
Rather difficult
Difficult
PCM (Best)
100
79.62
61.53
16.66
Table 6: Accuracy on hardness Likert scale annotated by experts.
Appendix figures & tables7 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 4: Topical, Temporal, and Authorial Diversity in our annotated corpus.
Figure 5: Distribution of Rhetorical Functions by Relative Position, revealing a structured rhetorical flow in judicial reasoning—from the initial announcement to the final resolution.
Figure 6: The final coding scheme is composed of 5 categories (ovals with orange background), 13 rhetorical functions (green rectangles) and 24 attributes (types in blue rectangles, target in the yellow rectangle, and author in the purple rectangle. The scheme reads from top to bottom: A step label is constructed by first choosing a category, then a rhetorical function, then if required, by combining attributes to complete the discursive information provided by the rhetorical function.
Figure 7: The hierarchical architecture.
Dataset
Source
Domain
Language
# Docs
# Sents
Labels
Scotus Category
Ours
Legal (U.S.)
English
180
26,327
5
Scotus RF
Ours
Legal (U.S.)
English
180
26,327
13
Scotus Steps
Ours
Legal (U.S.)
English
180
26,327
35
LegalEval
Kalamkar et al. (2022)
Legal (India)
English
214
31,865
13
DeepRhole
Bhattacharya et al. (2023a)
Legal (India)
English
50
9,380
7
PubMed
Dernoncourt and Lee (2017)
Medical
English
20,000
227,000
5
Appendix
Table 8: Evaluation datasets used in our experiments. Scotus is annotated at three hierarchical levels: category, rhetorical function, and steps.
Method
Scotus RF
LegalEval
PubMed
Linear Fusion
80.89
91.62
91.91
Conditional Layer Norm
78.11
87.49
92.74
Cross-Attention Fusion
79.30
87.74
92.20
Feature-wise Linear Mod.
74.71
76.74
92.74
Gated Residual Addition
79.58
89.06
92.79
Appendix
Table 9: W-F1 scores for prototype injection strategies. All variants share the same hierarchical encoder with PCM integration.
Figure 8: Effect of PBR hyperparameters on w-F1 at the SCOTUS RF Dashed lines indicate the baseline without prototypes.
Prototype learning represents data with representative examples or feature centers to support classification, adaptation, anomaly detection, and interpretable predictions.
Rhetorical Role Labeling (RRL) assigns a functional role to each sentence in a document and is widely used in legal, medical, and scientific domains. While language models (LMs) achieve strong average performance, they remain unreliable on hard examples, where prediction confidence is low. Existing approaches typically handle uncertainty implicitly and treat labels as discrete identifiers, overlooking the semantic information encoded in label names. We introduce RISE, an inference-time semantic reranking framework that leverages label semantics to refine predictions on hard instances. RISE automatically identifies low-confidence predictions and reranks model outputs using contrastively learned label representations, without retraining or modifying the underlying model. Experiments on eight domain-specific RRL datasets with seven LMs, including encoder-based and causal architectures, show an average gain of +9.15 macro-F1 points on hard examples. For explainability, we further propose manual hardness annotations to study difficulty from both model and human perspectives, revealing a moderate agreement with Cohen's kappa = 0.40.
Anas Belfathi, Nicolas Hernandez, Laura Monceaux +2
Nantes Université, École Centrale Nantes, CNRS, LS2N, UMR 6004, F-44000 Nantes, France · University of Lorraine, France
Court judgments are central to legal practice and jurisprudence, yet discourse analysis of Hong Kong judgments has received limited attention, owing largely to the absence of expert-annotated corpora. We introduce the Hong Kong Judgment Discourse Dataset (HKJudge), the first sentence-level expert-annotated legal discourse corpus. HKJudge includes criminal judgments across all five levels of HK's court hierarchy, comprising ∼290k sentences and ∼6.5 million tokens, fully annotated by legal linguistics experts. We design a two-tier discourse schema that captures what facts a court finds, how it reasons, and what it rules. At the sentence level, each sentence is assigned one of 26 rhetorical roles. At the span level, sentences are further annotated with three sentencing elements (charge, imprisonment term, fine). Ten legal linguistics annotators produced the annotations with an inter-annotator agreement of κ=0.8. We formulate two tasks on HKJudge, termed rhetorical role classification and legal element extraction, and provide the first benchmark evaluation of four BERT-based models, two open-source LLMs under zero-shot and fine-tuning settings, and four commercial LLMs on both tasks. Our work demonstrates the value of sentence-level discourse annotation for modeling the structure of HK judgments and provides a rich data foundation for future work on legal judgment prediction. The HKJudge dataset and code are available at https://github.com/xuanxixi/HKJudge.
Xi Xuan, Wenxin Zhang, Yufei Zhou +2
City University of Hong Kong · University of Chinese Academy of Sciences
Semantic role labeling (SRL) is a central natural language processing task for understanding predicate-argument structures within texts and enabling downstream applications. Despite extensive research, comprehensive surveys that critically synthesize the field from a unified perspective remain lacking. This survey makes several contributions beyond organizing existing work. We propose a unified four-dimensional taxonomy that categorizes SRL research along model architectures, syntax feature modeling, application scenarios, and multimodal extensions. We provide a critical analysis of when and why syntactic features help, identifying conditions under which syntax-aided approaches provide consistent gains over syntax-free counterparts. We offer the first systematic treatment of SRL in the era of large language models, examining the complementary roles of LLMs and specialized SRL systems and identifying directions for hybrid approaches. We extend the scope of SRL surveys to cover multimodal settings including visual, video, and speech modalities, and analyze structural differences in evaluation across these modalities. Literature was collected through systematic searches of the ACL Anthology, IEEE Xplore, the ACM Digital Library, and Google Scholar, covering publications from 2000 to 2025 and applying explicit inclusion and exclusion criteria to yield approximately 200 primary references. SRL benchmarks, evaluation metrics, and paradigm modeling approaches are discussed alongside practical applications across domains. Future research directions are analyzed, addressing the evolving role of SRL with large language models and broader NLP impact.
Huiyao Chen, Meishan Zhang, Jing Li +4
1Harbin Institute of Technology (Shenzhen) 2 Shenzhen Loop Area Institute (SLAI) · 1Harbin Institute of Technology (Shenzhen) · University of Oslo +1