Compact Prompting in Instruction-tuned LLMs for Joint Argumentative Component Detection
Organizations: Université Côte d’Azur, I3S, CNRS, Inria (Marianne), France
Abstract
Argumentative component detection (ACD) is a core subtask of Argument(ation) Mining (AM) and one of its most challenging aspects, as it requires jointly delimiting argumentative spans and classifying them into components such as claims and premises. While research on this subtask remains relatively limited compared to other AM tasks, most existing approaches formulate it as a simplified sequence labeling problem, component classification, or a pipeline of component segmentation followed by classification. In this paper, we propose a novel approach based on instruction-tuned Large Language Models (LLMs) using compact instruction-based prompts, and reframe ACD as a language generation task, enabling arguments to be identified directly from plain text without relying on pre-segmented components. Experiments on standard benchmarks show that our approach achieves higher performance compared to state-of-the-art systems. To the best of our knowledge, this is one of the first attempts to fully model ACD as a generative task, highlighting the potential of instruction tuning for complex AM problems.
Figures & tables
| Dataset | Argument Components | |
|---|---|---|
| Claim | Premise | |
| USElecDeb60To16 Haddadan et al. (2019) | 29k | 26k |
| Persuasive Essays Stab and Gurevych (2017) | 2257 | 3832 |
| Web Discourse Habernal and Gurevych (2017) | 195 | 538 |
| Merge | 31.5k | 30.3k |
| Model/Dataset | Macro F1 | Acc |
|---|---|---|
| Human Upper Bound Stab and Gurevych (2017) - PE | 0.8860 | - |
| CRF Goudas et al. (2014) | 0.4237 | - |
| Heuristic Baseline Stab and Gurevych (2017) - PE | 0.642 | - |
| CRF with features Stab and Gurevych (2017) - PE | 0.8670 | - |
| MT-all Morio et al. (2022) - PE | 0.7566 | - |
| OPT-6.7B - PE | 0.8518 | 0.8856 |
| Gold | And so <premise> if you believe the same thing </premise> , <claim> you just don’t want to raise taxes on people </claim> . And <claim> the reality is it’s not just wealthy people </claim> |
| OPT-6.7B ACD | And so <premise> if you believe the same thing </premise> , <claim> you just don’t want to raise </claim> taxes on people. And <premise> the reality is it’s not just wealthy people </premise> |
| Gold | <claim> We will do what we do best</claim>. <claim> It’s a strategy that we’ve been working on for a couple of years </claim> . <claim> It is going to take us to much better advantage in conventional forces </claim> |
| OPT-6.7B ACD | <claim> We will do what we did best </claim> . <premise> It’s a strategy that we’ve been working on for a few years </premise> . <claim> It is going to take us to much better advantage in conventional forces </claim> |
| Gold | <claim> Maybe we need to do a better job in mental clinics to help them </claim> . Because <premise> there is a major problem there </premise> |
| OPT-6.7B ACD | <claim> Maybe we need to do better job in mental clinics </claim> <premise> to help them </premise> . Because <premise> there is a major problem there </premise> . |
| Gold | <premise> She’s been doing this for 30 years </premise> |
| OPT-6.7B ACD | <premise> She’s been doing this job for 30 years </premise> |
| Model | B-C | I-C | B-P | I-P | O | F1-Macro |
| OPT-6.7B (PE) | 0.78 | 0.76 | 0.87 | 0.88 | 0.97 | 0.85 |
| GPT-2-1.5B (PE) | 0.78 | 0.76 | 0.87 | 0.89 | 0.96 | 0.85 |
| Llama-3-8B (PE) | 0.84 | 0.80 | 0.90 | 0.88 | 0.98 | 0.88 |
| DeBERTa-v3 (PE) | 0.6 | 0.74 | 0.58 | 0.79 | 0.82 | 0.71 |
| RoBERTa (PE) | 0.59 | 0.68 | 0.59 | 0.78 | 0.19 | 0.69 |
| OPT-1.3B (Merge) | 0.75 | 0.73 | 0.71 | 0.73 | 0.92 | 0.77 |
| Dataset | O | B-Premise | I-Premise | B-Claim | I-Claim |
|---|---|---|---|---|---|
| USElecDeb60To16 | 566492 | 26055 | 350079 | 29624 | 338941 |
| Persuasive Essays | 35946 | 2257 | 29828 | 3832 | 59652 |
| Web Discourse | 61414 | 195 | 3491 | 538 | 20566 |
| Gold | As a former educator, I believe that every child deserves equal access to education in the least restrictive environment possible. As a parent, I support that concept as long as the inclusion doesn’t come at the expense of the other students in class. Inclusion can be successful rewarding for all involved but it also has drawbacks. |
| OPT-6.7B ACD | <claim> As a former educator, </claim> <claim> I believe that every child deserves equal access to education in the less restrictive environment possible. </claim><claim> As a parent, </claim> <premise> I support that concept as long as the inclusion doesn’t come at the expense of the other children in class. </premise> <claim> Inclusion can be successful rewarding for all involved but also has drawbacks. </claim> |
| Gold | <premise> I’m not a politician </premise> , but <claim> I think I could go to Washington in a week and get everybody holding hands and get this bill signed because I talk to the Democratic leaders and they want it </claim> |
| OPT-6.7B ACD | <premise> I’m not a politician </premise> , but <claim> I think I could go to Washington in a week and get Everybody holding hands and get this Bill signed </claim> <premise> because I talk to the Democratic Leaders and they want it </premise> |
Argument Mining
Argument mining identifies claims, premises, and their relations in text to reveal how people construct and contest arguments.
See all 39 papers