Overview

Text summarization studies how to condense one or more documents into shorter accounts that preserve important information for a reader or task. Extractive methods select source sentences, while abstractive methods generate new wording; current systems also use large language models, hierarchical processing, retrieval, and structured evidence to handle long or specialized sources. Benchmarks such as CNN/DailyMail, XSum, and Multi-News support comparison, but overlap scores like ROUGE may not capture factual accuracy, coverage, or usefulness. A central challenge is balancing concise, readable summaries with faithful evidence, especially for long documents, multiple viewpoints, and high-stakes domains.

Latest papers

151 papers10 papers in the last four weeks, down 23% on the four weeks before. 0.1% of all new papers.

May 3, 2024

OARelatedWork: A Large-Scale Dataset of Related Work Sections with Full-texts from Open Access Sources

This paper introduces OARelatedWork: a dataset for related work generation from open-access sources. It is the first large-scale multi-document summarization dataset for related work generation, containing whole related work sections and full texts of cited papers. Its validation and test splits are constructed so that every source paper is available in full text, enabling controlled evaluation of full-text related work generation. The dataset includes 94 450 papers from multiple domains. With OARelatedWork, we aim to shift the field from generating parts of related work sections from abstracts only to generating entire related work sections from all available content. We (i) benchmark a wide spectrum of models, highlighting that synthesizing massive full-text contexts remains challenge for tested cost-effective Large Language Models (LLMs): under our statement-level judge, GPT-4o-mini's evidence-grounded True rate drops from 92.9% with abstracts to 83.8% with full texts. We (ii) empirically analyze human writing behavior through a human evaluation over 40 papers and 408 factual statements, revealing that authors frequently introduce abstractive claims ungrounded in localized source texts; consequently, tested LLMs actually surpass human baselines in strict, evidence-grounded factuality. Finally, we (iii) conduct a fine-grained meta-evaluation, revealing that standard reference-based metrics are inadequate for evaluating such long-form structured outputs, and introduce a robust statement-level evaluation framework to address this gap.