Towards Full-Text Scholarly Argument Mining using Entity-Anchored Claim Extraction
Organizations: Text Technology Lab (TTLab), Goethe University Frankfurt, Germany · Fakultät für Germanistik, Technische Universität Chemnitz, Germany
Abstract
Argument mining in full-text scholarly articles remains an important yet challenging task. This difficulty stems not only from the need to model long-distance dependencies but also from the substantial effort required to annotate sentence-level argumentative components throughout lengthy documents. To address these challenges, we introduce a novel approach that leverages scientific entity mentions (e.g., tasks, datasets, and models) to identify and structure claims in scholarly literature. We validate our approach through a case study involving 50 full-text research papers on sentiment analysis. We first establish a reliable human gold standard (Krippendorff's ), identifying 113 claim-bearing sentences and extracting 199 structured claim records from 300 candidate sentences. To address the annotation bottleneck, we employ a majority-voting ensemble of large language models (LLMs) to generate a silver-standard dataset comprising 1,487 claims. Encoders fine-tuned exclusively on the silver-standard dataset outperform those trained on a small human-annotated gold-standard dataset and demonstrate strong zero-shot transfer to NLP papers outside the training corpus.
Figures & tables
| Model | P | R | F1 | [95% CI] |
|---|---|---|---|---|
| Qwen3.6-27B | .962 | .673 | .792 | .70 [.61, .78] |
| Gemma-4-31B | .771 | .956 | .854 | .75 [.67, .82] |
| Mistral-3.2-24B | .832 | .832 | .832 | .73 [.64, .81] |
| maj3 | .888 | .841 | .864 | .78 [.71, .85] |
| SciBERT † | .926 | .811 | .865 | .79 [.71, .85] |
| DeBERTa-v3-large † | .939 | .823 | .877 | .81 [.75, .88] |
| Set | Papers | Sentences | Claim Ratio |
|---|---|---|---|
| Core | 32 | 60 | 38.3% |
| Gold | 43 | 294 | 38.4% |
| Silver | 50 | 9,073 | 16.4% |
| ClaimFlow | 388 | 14,418 | 14.8% |
Appendix figures & tables6 assets
Supplementary material from the paper’s appendix.
Appendix
| Entity Type | Keywords |
|---|---|
| Datasets | IMDb, SST, SST-2, SST-5, Stanford Sentiment Treebank, SemEval-2014, SemEval-2015, SemEval-2016, Sentiment140, Yelp, Amazon Reviews, Twitter US Airline, MR dataset, Rotten Tomatoes, Laptop14, Restaurant14 |
| Methods | SVM, Naïve Bayes, LSTM, BiLSTM, Bi-LSTM, GRU, CNN, RNN, BERT, RoBERTa, XLNet, DistilBERT, ALBERT, Transformer, Word2Vec, GloVe, FastText, TF-IDF, Attention, Memory Network, Capsule Network |
| Property | Value |
|---|---|
| Papers | 50 |
| Publication years | 2013-2023 |
| Full-text length (median words) | 4,533 |
| Citations received (median) | 191 |
| Papers citing each seed benchmark | |
| SemEval-14 ABSA | 30 |
| Label | Use | Mentions |
| Method | entity | 5,035 |
| ModelArchitecture | entity | 2,849 |
| Task | entity | 2,656 |
| MLModel | entity | 2,080 |
| Dataset | entity | 980 |
| Datasource | entity | 120 |
| Statistic | Value |
|---|---|
| Specific entity mentions | 13,720 |
| Per paper (median; range) | 250; 94-605 |
| Body sentences with 1 entity | 6,220 |
| Distinct entities (normalised) | 3,627 |
| Method / Task | 2,027 / 488 |
| MLModel / ModelArchitecture | 330 / 587 |
| Sections | Pool | Sampled | Rate | Core |
|---|---|---|---|---|
| Results onward † | 1,293 | 175 | 13.5% | 35 |
| Abs. + Conc. | 381 | 75 | 19.7% | 15 |
| Intro. + Rel. Work | 1,101 | 35 | 3.2% | 7 |
| Method + Exp. | 1,396 | 15 | 1.1% | 3 |
| Total | 4,171 | 300 | 7.2% | 60 |
| Predicate | Definition | Constraint |
|---|---|---|
| outperforms | Subject achieves better task performance than the object on a shared task, dataset, or metric. | object |
| underperforms | Subject achieves worse task performance than the object on a shared task, dataset, or metric. | object |
| achieves | Subject attains a stated performance level, with no named comparison object. | no object |
| improves | Subject raises performance relative to a prior state, baseline, or component it modifies. | object |
| degrades | Subject lowers performance relative to a prior state, baseline, or component it modifies. | object |
| generalizes-to | Subject transfers to the object domain, task, language, or dataset while retaining performance. | object |
Argument Mining
Argument mining identifies claims, premises, and their relations in text to reveal how people construct and contest arguments.
See all 40 papers