LAURA: Knowledge Distillation for Interpretable Ambiguous Clause Identification in Legal Contracts
Organizations: School of Computer Science and Engineering University of New South Wales (UNSW), Sydney
Abstract
Legal contracts contain ambiguities that expose enterprises to financial and legal risks. Some ambiguities allow flexible interpretation without triggering disputes, while others lead to significant legal conflicts. This makes identification alone insufficient, and interpretable rationale analysis essential. We propose LAURA, a post-training framework for interpretable ambiguous clause identification. LAURA leverages knowledge distillation with an IRAC-Unlearning prompting technique to transfer knowledge from a teacher LLM to an open-weight student model (<=1B parameters), which is then trained using a joint objective combining classification and rationale generation losses. The framework supports both legal and non-legal stakeholders in making informed decisions about which ambiguities require further attention. Extensive experiments across 7 baselines and 7 open-weight models demonstrate that LAURA with Flan-T5 (250M) delivers state-of-the-art interpretability over all interpretable baselines while matching the identification performance of the best-performing opaque baseline.
Figures & tables
| App. | Models | P | R | F1 | A | I |
| Majority | – | 0.52 | 1.0 | 0.68 | 0.52 | ✗ |
| Direct | GPT-4o | 0.53 | 0.97 | 0.69 | 0.54 | ✓ |
| CoT | GPT-4o | 0.53 | 0.98 | 0.69 | 0.53 | ✓ |
| 3-Shot | GPT-4o | 0.69 | 0.32 | 0.44 | 0.57 | ✓ |
| 5-Shot | GPT-4o | 0.63 | 0.30 | 0.40 | 0.54 | ✓ |
| SFT | BERT | 0.71 | 0.60 | 0.65 | 0.66 | ✗ |
Appendix figures & tables2 assets
Supplementary material from the paper’s appendix.
Appendix
| Ambiguity Types | Examples |
| Lexical | Enable a user to electronically record, modify, and retrieve a patient’s active medication list as well as medication history for longitudinal care. |
| Syntactic | Supplier will request Distributor immediately to assign its rights in the property. |
| Semantic | The contractor must deliver the project on time. |
| Vagueness | Data Processor shall keep Personal Information sufficiently isolated from other data on the server in an appropriate manner to prevent it from being misused. |
| Incompleteness | The Service Provider shall notify the Customer in writing of any changes to the Service Level Agreement. |
| Referential | The Service Provider shall provide support to the Customer according to the applicable provisions in the Agreement. |
| Correctness | Completeness | Conciseness |
| Cor1: Factually incorrect: The rationale does not accurately represent the prediction. There are significant factual errors or contradictions within the rationale that misalign with the predicted label. | Com1: Misses key points or explanations: The rationale omits essential aspects or critical explanations related to the predicted label, making it incomplete and inadequate. | Con1: Overtly verbose and hard to follow: The rationale is excessively long or includes irrelevant information, making it difficult to follow and understand the reasoning to the predicted label. |
| Cor2: Mostly correct with minor misinterpretation: The rationale does not contradict the predicted label, but contains minor misinterpretations that may lead to incorrect or misleading details, which do not adequately support the prediction. | Com2: Covers important aspects but omits details: The rationale addresses the key points related to the predicted label but leaves out minor details, resulting in an incomplete explanation. | Con2: Contains unnecessary elaboration: The rationale includes unnecessary elaboration or details extraneous to the understanding of the predicted label. This makes the rationale less concise but relatively clear. |
| Cor3: Acceptable: The rationale is fully aligned with the predicted outcome, providing a clear and accurate explanation for the prediction, and it faithfully supports the model’s prediction. | Com3: Acceptable: The rationale fully covers all important aspects of the predicted label, providing a complete and detailed explanation with all essential information. | Con3: Acceptable: The rationale is succinct, providing relevant information without unnecessary elaboration. It effectively supports the predicted label without being overly verbose or lacking in clarity. |
Legal NLP
Legal natural language processing (NLP) develops methods for analyzing and generating statutes, cases, contracts, and legal advice.
See all 109 papers