It Depends on the Task: Rethinking Rhetorical Segmentation of Dutch Judicial Decisions
Organizations: Maastricht University, the Netherlands.
Abstract
Classifying the material facts, submissions, judicial reasoning, and the court's decision (rhetorical roles) in case law is a prerequisite for high-quality performance across virtually all NLP-in-law tasks, including legal information retrieval, summarisation, and LLM-based analysis. These rhetorical roles often intertwine and shift sentence by sentence, making reliable classification difficult. This paper introduces the first sentence-level annotated dataset of Dutch judicial decisions, labelled across four legally meaningful categories: material facts, submissions, judicial reasoning, and the court's decision, alongside non-annotated (None) content, and compares several classification strategies, including a staged pipeline that separates None detection from the rhetorical roles, and a Bayesian update that leverages information about the section-header a sentence belongs to. Our results show that no single strategy dominates on both precision and recall: header priors help most when the None sentences are excluded and offer little benefit for None detection itself. These trade-offs matter in practice. For instance, extractive summarisation depends more on recall so it does not miss relevant facts, whereas outcome prediction depends more on precision regarding decision and facts sentences so that the predicted ruling is not contaminated by noise. We use these trade-offs and map the approaches tested in this study onto a set of representative legal NLP use cases.
Figures & tables
| Label | Count | Proportion |
|---|---|---|
| None | 3,060 | 36.5% |
| beoordeling | 2,293 | 27.3% |
| materiële feiten | 1,529 | 18.2% |
| proceshandelingen | 1,263 | 15.1% |
| beslissing | 246 | 2.9% |
| Total | 8,391 | 100.0% |
| Model | Setting | Subset | N | Before Bayes | After Bayes | ||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Acc. | P | R | F1 | W-F1 | Acc. | P | R | F1 | W-F1 | ||||
| BERTje | 5-way | incl. None | 1502 | 0.6458 | 0.6774 | 0.6398 | 0.6561 | 0.6442 | 0.6551 | 0.6890 | 0.6416 | 0.6615 | 0.6528 |
| Pipeline A | incl. None | 1502 | 0.6258 | 0.6466 | 0.5997 | 0.6164 | 0.6235 | 0.6505 | 0.6835 | 0.6300 | 0.6496 | 0.6487 | |
| Pipeline B | incl. None | 1502 | 0.6272 | 0.6530 | 0.6028 | 0.6212 | 0.6256 | 0.6585 | 0.6900 | 0.6474 | 0.6625 | 0.6573 | |
| Stage 1 | incl. None (binary) | 1502 | 0.7783 | 0.7595 | 0.7589 | 0.7592 | 0.7782 | 0.7783 | 0.7596 | 0.7581 | 0.7588 | 0.7780 | |
| Stage 2 4-way | excl. None | 961 | 0.7138 | 0.7484 | 0.7128 | 0.7272 | 0.7130 | 0.7638 | 0.7996 | 0.7619 | 0.7781 | 0.7626 | |
| Formulation | Class | Support | Before (plain) | After (tuned ) | ||||
|---|---|---|---|---|---|---|---|---|
| P | R | F1 | P | R | F1 | |||
| 5-way | None | 541 | 0.7033 | 0.7579 | 0.7295 | 0.6933 | 0.7689 | 0.7292 |
| beoordeling | 389 | 0.6355 | 0.6992 | 0.6659 | 0.6548 | 0.7069 | 0.6799 | |
| beslissing | 69 | 0.8947 | 0.7391 | 0.8095 | 0.9074 | 0.7101 | 0.7967 | |
| materiële feiten | 297 | 0.6423 | 0.5320 | 0.5820 | 0.6639 | 0.5387 | 0.5948 | |
| proceshandelingen | 206 | 0.5798 | 0.5291 | 0.5533 | 0.6150 | 0.5583 | 0.5852 | |
| # | Task | Objective | Best fits | Unsuitable |
|---|---|---|---|---|
| 1 | Extractive summarization of facts | Recall on materiële feiten | Stage 2 alone (0.677 R) if pre-filtered; Pipeline B (0.572 R) end-to-end | Fused/precision-tuned OvR config (0.892 P / 0.583 R after pre-filtering) |
| 2 | Provision retrieval from facts | Precision on feiten | OvR materiële feiten detector alone (0.892 P after pre-filtering) | Five-way / Pipeline A (0.664 / 0.649 P) |
| 3 | Case-based retrieval on facts | Balanced, recall-leaning | Stage 2 alone (0.732 F1) if pre-filtered; Pipeline B (0.610 F1) end-to-end | Precision-tuned OvR config |
| 4 | Outcome prediction | Precision on beslissing | Stage 2/OvR alone (0.984/0.969 P); Pipeline B (0.906 P) end-to-end | Pipeline A (0.844 P) |
| 5 | Argument mining | Recall on beoordeling | Stage 2 alone (0.851 R); Pipeline A (0.756 R) end-to-end | Precision-tuned configs |
| 6 | Reading aid (highlighting roles in the text) | Balanced, no class favoured | Pipeline B, or fused five-way (0.681 / 0.677 Macro F1) | — |
Legal NLP
Legal natural language processing (NLP) develops methods for analyzing and generating statutes, cases, contracts, and legal advice.
See all 117 papers