Summarization Evaluation

Ask about this topic

Momentum

4 papers in the last four weeks, against 1 the four weeks before. 0.0% of all new papers.

Jul 13Week of Sep 28
View on map

Latest papers 24

All topics
CardsList

  1. A Comparative Study of Evaluation Metrics for Long-Document Financial Narrative Summarization with Transformers

    Oct 7, 2026Nadhem Zmandar, Mo El-Haj, Paul RaysonText SummarizationSummarization Evaluation

    UCREL NLP group, School of Computing and Communications, Lancaster University, UK

  2. Whose Voice Survives the Summary? A Voice-Retention Audit of LLM Employee Listening

    Sep 30, 2026Thilo Tamme, Anton Hantel, Bijan Khosrawi-RadText SummarizationSummarization Evaluation

    Technical University of Munich · Massachusetts Institute of Technology · Leuphana University Lüneburg

  3. CHI: A Composite Hallucination Index Unifying Entity, Relation, and Quantity Dimensions for Summarization Evaluation

    Sep 27, 2026Praveenkumar Katwe, Rakesh Chandra Balabantaray, Kali Prasad VittalaHallucination in Language ModelsFactual Consistency Evaluation

    International Institute of Information Technology Bhubaneswar, India · Salesforce India Pvt Ltd, Bengaluru, India

  4. Can Classical Semantic-Extractive Summarization Be Evaluated in Hindi? A Replication Study

    Sep 24, 2026Showket Ahmad Khan, Mudasir Mohd, Nasrullah Sheikh +4Text SummarizationSummarization Evaluation

    Department of Computer Science, South Campus, University of Kashmir, Anantnag, India · IBM Research, San Jose, CA, USA · Manipal University Jaipur, Dehmi Kalan, Jaipur 303007, Rajasthan, India

  5. Preserving What Matters: Semantic Scaffolds Beyond Saturation in Summarization Evaluation

    Sep 18, 2026Nikhil Reddy Pottanigari, Ramin Fahimi, Noah Bolger +2LLM EvaluationText Summarization

    ServiceNow Canada

  6. Beyond Scores: Understanding LLM-as-a-Judge Mechanisms in Summarization Evaluation

    Sep 1, 2026Himil Vasava, Ming JiangLLM-as-a-JudgeLanguage Model Generation Evaluation

    University of Wisconsin-Madison

  7. Averaging Bias: Human Faithfulness Annotations are not Locally Faithful

    Jul 31, 2026Huajian Zhang, Yiyang Feng, Jiawei ZhouFactual Consistency EvaluationSummarization Evaluation

    Stony Brook University

  8. Reexamining zero-shot summarization: Empirical investigation of trustworthiness of LLM-summarizers

    Jul 23, 2026Vasudha Bhatnagar, Purnima Bindal, Vikas Kumar +1LLM EvaluationText Summarization

    Department of Computer Science, University of Delhi, Delhi, 110007, Delhi, India · Department of Statistics, Ramjas College, University of Delhi, Delhi, 110007, Delhi, India

  9. Abstractiveness Metrics for Evaluating Text Summarization: A Refined Formulation with Empirical Validation

    Jul 12, 2026Praveenkumar Katwe, Rakesh Chandra Balabantaray, Kali Prasad VittalaText SummarizationSummarization Evaluation

    Department of Computer Science and Engineering, International Institute of Information Technology, Bhubaneswar, India · Salesforce India Pvt Ltd, Bengaluru, India

  10. A Large-Scale Multi-Dimensional Empirical Study of LLMs for Conversation Summarization

    Jun 14, 2026Weixiao Zhou, Gengyao Li, Xianfu Cheng +3LLM EvaluationLong-Context Language Model Evaluation

    CCSE, Beihang University · Fanyu AI Laboratory · MAIS, CASIA

  11. Summarization is Not Dead Yet

    Jun 6, 2026Dongqi Liu, Chenxi Whitehouse, Zheng Zhao +3LLM EvaluationText Summarization

    Saarland University, Max Planck Institute for Informatics · Tencent YouTu Research · University of Cambridge +2

  12. Thinking Through Signs: PEEL as a Semiotic Scaffolding for Epistemically Accountable AI-Enabled Research

    Jun 2, 2026Clarisse de Souza, Gabriel Barbosa, Simone Diniz Junqueira Barbosa +3LLM EvaluationAI Accountability

    PUC-Rio, PUC-Behring Institute of Artificial Intelligence, Brazil · PUC-Rio, Brazil

  13. Plans for Evaluating Structured Generative Search Summaries

    May 26, 2026Tetsuya Sakai, Jina Lee, Hanpei Fang +1Language Model Generation EvaluationSummarization Evaluation

    Waseda University/Naver Corporation · Tokyo, Japan · Waseda University +2

  14. Measuring What Matters Beyond Text: Evaluating Multimodal Summaries by Quality, Alignment, and Diversity

    May 12, 2026Abid Ali, Diego Molla-Aliod, Usman NaseemLLM EvaluationMultimodal Large Language Models

    School of Computing, Macquarie University, Sydney, Australia

  15. LLM-ReSum: A Framework for LLM Reflective Summarization through Self-Evaluation

    Apr 28, 2026Huyen Nguyen, Haoxuan Zhang, Yang Zhang +2LLM EvaluationLLM Self-Refinement

    Cigna Group Evernorth Health Services Austin, Texas, USA · dept. of Information Science University of North Texas Denton, Texas, USA · dept. of Data Science University of North Texas Denton, Texas, USA

  16. LongSumEval: Question-Answering Based Evaluation and Feedback-Driven Refinement for Long Document Summarization

    Apr 28, 2026Huyen Nguyen, Haoxuan Zhang, Yang Zhang +2Language Model Generation EvaluationFactual Consistency Evaluation

    Cigna Group Evernorth Health Services Austin, Texas, USA · dept. of Information Science University of North Texas Denton, Texas, USA · dept. of Data Science University of North Texas Denton, Texas, USA

  17. QEVA: A Reference-Free Evaluation Metric for Narrative Video Summarization with Multimodal Question Answering

    Apr 27, 2026Woojun Jung, Junyeong KimVideo SummarizationSummarization Evaluation

    Department of Artificial Intelligence, Chung-Ang University

  18. SCURank: Ranking Multiple Candidate Summaries with Summary Content Units for Enhanced Summarization

    Apr 21, 2026Bo-Jyun Wang, Ying-Jia Lin, Hung-Yu KaoLearning to RankText Summarization

    Department of Computer Science and Information Engineering, National Cheng Kung University · Department of Artificial Intelligence, Chang Gung University · Artificial Intelligence Research Center, Chang Gung University +1

  19. Calibrating Model-Based Evaluation Metrics for Summarization

    Apr 19, 2026Hongye Liu, Dhanajit Brahma, Ricardo HenaoLanguage Model CalibrationSummarization Evaluation

    Duke University

  20. Diagnosing LLM Judge Reliability: Conformal Prediction Sets and Transitivity Violations

    Apr 16, 2026Manan Gupta, Dhruv KumarLLM EvaluationLLM-as-a-Judge

    1BITS Pilani, Pilani Campus, India

  21. Gavel: Agent Meets Checklist for Evaluating LLMs on Long-Context Legal Summarization

    Jan 7, 2026Yao Dou, Benjamin Mamut, Wei XuLegal NLPLLM Evaluation

    Georgia Institute of Technology

  22. MAPLE: Medical Aspect-Based Summarization with Phrase-Level Evidence

    Jan 6, 2026Bohao Chu, Hendrik Damm, Tabea M. G. Pakull +5HealthcareText Summarization

    University of Duisburg-Essen · University of Applied Sciences and Arts Dortmund · University Hospital Essen +1

  23. VIBE: Annotation-Free Video-to-Text Information Bottleneck Evaluation for TL;DR

    May 23, 2025Shenghui Chen, Po-han Li, Sandeep Chinchali +1VLM EvaluationVideo Summarization

    The University of Texas at Austin

  24. Evaluating the Evaluator: Summarization Metrics and LLM-Judges beyond English

    Mar 21, 2025Jeremy Barnes, Naiara Perez, Alba Bonet-Jover +1Summarization EvaluationAutomated Evaluation

    HiTZ Center, University of the Basque Country (UPV/EHU) · Department of Software and Computing Systems, University of Alicante, Spain · GOI Institute, Basque Summer University (UEU)