Radiology Report Evaluation

Ask about this topic

Overview

The field develops ways to score report-generation and report-review systems for clinical accuracy, completeness, and harmful errors—not just similarity to a reference report. Approaches include finding-level extraction and matching, clinical graphs, question answering and entailment, severity-aware LLM judges, and direct radiologist review. Benchmarks span chest X-rays, CT, PET, MRI, and longitudinal reports, with growing attention to spatial grounding, report-writing conventions, and whether findings are supported by images. A central tension is that fluent reports and high text-overlap scores can conceal omissions or unsupported claims, while more clinically aligned metrics still need to be transparent, consistent, and validated across settings.

Latest papers

CardsList

32 papers4 papers in the last four weeks, down 33% on the four weeks before. 0.0% of all new papers.

  1. Dissociating Answerability from Extraction in LLM Question Answering over Radiology Reports

    Oct 13, 2026Priyanshu Sinha, Kelsey McDonaldMedical QARadiology Report Evaluation

    Labcorp, Durham, NC

  2. From Sampled Evidence to Radiology Reports: Objectives, Constraints, and Realization

    Oct 12, 2026Kun Zhao, Haoteng Tang, Chenghua Lin +1Radiology Report GenerationEvidence-Grounded Generation

    University of Pittsburgh · University of Texas Rio Grande Valley · The University of Manchester

  3. Unmentioned Checklist Findings Change How Reinforcement Learning Appears to Improve Chest Radiograph Report Checking

    Oct 4, 2026Ali Vosoughi, Akhil Kasturi, Chenliang Xu +1Radiology VLMsRadiology Report Evaluation

    Department of Computer Science, University of Rochester, Rochester, NY 14627, USA. · Department of Electrical and Computer Engineering, University of Rochester, Rochester, NY 14627, USA. · Department of Imaging Sciences, University of Rochester Medical Center, Rochester, NY 14642, USA. +1

  4. Autonomous Structuring of Radiology Reports Across Modalities at Archive Scale Using an Open-Weight Large Language Model

    Oct 3, 2026Friedrich Puttkammer, Fabian Drexel, Marlene Fritzsche +19Medical Report GenerationRadiology Report Generation

    Department of Radiology, Charité – Berlin University of Medicine, corporate member of Freie Universität Berlin and Humboldt Universität zu Berlin, Berlin, Germany · Department of Diagnostic and Interventional Radiology, School of Medicine and Health, TUM Klinikum Rechts der Isar, Technical University of Munich, Munich, Germany · Chair for AI in Healthcare and Medicine, Technical University of Munich (TUM) and TUM University Hospital, Munich, Germany +9

  5. QureRadEmbed: Structuring Radiological Similarity through Attribute and Reasoning Supervision

    Sep 27, 2026Janhavi Prabhu, Sahil, Shivam Ashok Shukla +1Clinical NLPContrastive Learning

    Research & Development, Qure.ai

  6. Can Jev Judge Radiology Reports? Evaluating a System One Model for Clinical Factuality

    Sep 23, 2026Jiaju Huang, Hao Yang, Xinyu Ma +6Radiology Report GenerationLLM-as-a-Judge

    Intelligent Medical Computing Laboratory, Faculty of Applied Sciences, Macao Polytechnic University · Department of Radiology, Netherlands Cancer Institute, Amsterdam, the Netherlands · Department of Radiology and Nuclear Medicine, Radboud University Medical Center, Nijmegen, the Netherlands

  7. Reporting Practice Matters: The Impact of Reference Choice on Chest X-ray Report Evaluation

    Sep 16, 2026Daniel P. Jeong, Charles Q. Li, Hossein Hosseiny +5Radiology Report GenerationGenerative AI Evaluation

    Machine Learning Department, Carnegie Mellon University · Department of Radiology and Imaging Sciences, Allegheny Health Network · Data Science R&D, Highmark Health Enterprise Data & Analytics +1

  8. Clinical Reasoning Under a Partially Observed Objective in Cone Beam CT Report Generation

    Sep 2, 2026Ajo Babu George, Govind Arun, Sidharth N Krishna +1CT Report GenerationRadiology Report Generation

    DiceMed, Cuttack, Odisha, India · University of Maryland, College Park, Maryland, USA · Indira Gandhi National Open University, New Delhi, India +1

  9. RadMatch: Auditable Radiology Report Evaluation via Finding-Level Matching

    Sep 1, 2026Charles Corbière, Léo Machado, Aubin Charley +3LLM-as-a-JudgeMedical Image Benchmarks

    Raidium, Paris, France · Department of Radiology, Fondation Ophtalmologique Adolphe de Rothschild, Paris, France · Service de Radiologie Interventionnelle, Département d’Anesthésie, Chirurgie et Imagerie Interventionnelle (DACI), Gustave Roussy, Villejuif, France

  10. A multicenter benchmark and clinically structured metric for coronary CTA report generation

    Sep 1, 2026Zhiyu Ye, Yue Sun, Limiao Zou +7VLM EvaluationRadiology Report Generation

    State Key Laboratory of Biomedical Imaging Science and System, Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen, China · Pengcheng Laboratory, Shenzhen, China · University of Chinese Academy of Sciences, Beijing, China +1

  11. Error Detection for PET/CT Radiology Reports: Domain-Specific vs Large Language Models

    Aug 30, 2026Hermione Warr, Harry Anthony, Lilli J Freischem +3LLM EvaluationPET/CT Imaging

    Department of Engineering Science, University of Oxford, UK · Department of Physics, University of Oxford, UK · Department of Oncology, University of Oxford, UK +1

  12. Multi-Agent AI System for Radiology Report Structuring and Quality Assurance with Independent Radiologist Evaluation

    Aug 18, 2026Iryna Hartsock, Cesar Lam, Christopher Otteni +4Radiology Report EvaluationMulti-Agent Systems

    Department of Machine Learning, H. Lee Moffitt Cancer Center & Research Institute, Tampa, FL, USA · Department of Diagnostic Imaging and Interventional Radiology, H. Lee Moffitt Cancer Center & Research Institute, Tampa, FL, USA · Department of Neuro-Oncology, H. Lee Moffitt Cancer Center & Research Institute, Tampa, FL, USA +1

  13. Beyond Fluency: A Clinical Benchmark and Anomaly-Enhanced Baseline for Spine MRI Report Generation

    Aug 7, 2026Bruno Palau, Franziska Vogt, Daria Laslo +4VLM EvaluationRadiology Report Generation

    ETH Zurich, Switzerland · 2Swiss Institute of Bioinformatics (SIB), Switzerland

  14. Prospective clinical indication, post-hoc report leakage, and fusion design in multi-image chest radiograph classification: a patient-clustered evaluation

    Jul 15, 2026Kamran Shahid, Muhammad Munwar IqbalData LeakageChest X-Ray Classification

    Department of Computer Science, University of Engineering and Technology, Taxila, Pakistan

  15. Harrison.Rad 1.5 Technical Report: A radiology foundation model that can draft reports from images, priors and clinical context

    Jul 7, 2026Suneeta Mall, Vladimir Nekrasov, Ashnil Kumar +5Radiology Report GenerationRadiology VLMs

  16. SHOVIR: A Benchmark for Evaluating Vision Shortcut Learning in Radiology Report Generation

    Jun 29, 2026Filippo Ruffini, Marco Salmé, Rosa Sicilia +2VLM EvaluationRadiology Report Generation

    Unit of Artificial Intelligence and Computer Systems, Department of Engineering, Università Campus Bio-Medico di Roma, Rome, Italy · Department of Diagnostics and Intervention, Radiation Physics, Biomedical Engineering, Umeå University, Umeå, Sweden · UniCamillus-Saint Camillus International University of Health Sciences, Rome, Italy

  17. Beyond Scalar Scores: Exploring LLM-based Metrics for Clinical Significance Evaluation in Radiology Reports

    Jun 17, 2026Qingyu Lu, Ruochen Li, Liang Ding +3Clinical Language Model EvaluationLanguage Model Robustness

    Nanyang Technological University · Technical University of Munich · Alibaba +2

  18. ReportQA: QA-Based Radiology Report Evaluation

    Jun 13, 2026Yiming Shi, Shaoshuai Yang, Xi Chen +10Radiology Report GenerationMedical VLMs

    Department of Electronic Engineering, Tsinghua University · Beijing Electronic Digital & Intelligence · College of AI, Tsinghua University +1

  19. RadOT-Eval: Auditable Structured-Evidence Transport for Radiology Report Evaluation

    Jun 7, 2026Weixin Liu, Juming Xiong, Yang Li +5Radiology Report GenerationAutomated Evaluation

    Vanderbilt University, Nashville, TN, USA · Vanderbilt University Medical Center, Nashville, TN, USA · Virginia Tech, Blacksburg, VA, USA

  20. RadSEM: A Finding-by-Finding Metric for Clinical Consistency in Radiology Reports

    Jun 3, 2026Zhenhong Yang, Zhuoyun Liu, Jintao Fei +4Factual Consistency EvaluationAutomated Evaluation

    JDH Algo, JD Health International Inc., China · Department of Big Data in Health Science, The First Affiliated Hospital of Wenzhou Medical University, China · Zhejiang Engineering Research Center for Hospital Emergency and Process Digitization, Wenzhou, China +1

  21. CT-FineBench: A Diagnostic Fidelity Benchmark for Fine-Grained Evaluation of CT Report Generation

    Apr 27, 2026Ruifeng Yuan, Wanxing Chang, Weiwei Cao +4Radiology Report EvaluationMedical Report Generation

    DAMO Academy, Alibaba Group, China · Hupan Lab, 310023, China · Fudan University, China +2

  22. CXRMate-2: Structured Multimodal Temporal Embeddings and Tractable Reinforcement Learning for Clinically Acceptable Chest X-ray Radiology Report Generation

    Apr 21, 2026Aaron Nicolson, Elizabeth J. Cooper, Hwan-Jin Yoon +7Radiology Report GenerationRadiology VLMs

    Australian e-Health Research Centre, CSIRO Health and Biosecurity, Brisbane & Melbourne, Australia · Princess Alexandra Hospital, Metro South Health, Brisbane, Australia · Queensland Digital Health Centre, University of Queensland, Brisbane, Australia +2

  23. Enhancing Fine-Grained Spatial Grounding in 3D CT Report Generation via Discriminative Guidance

    Apr 12, 2026Chenyu Wang, Weicheng Dai, Han Liu +2Radiology Report GenerationCT Report Generation

    Boston University, Boston, MA, USA · Siemens Healthineers, Princeton, NJ, USA

  24. Towards Better Aligned LLM Judges for Radiology Report Evaluation across Modalities and Anatomies

    Apr 3, 2026Federica Bologna, Jean-Philippe Corbeil, Matthew Wilkens +1LLM-as-a-JudgeRadiology Report Evaluation

    Cornell University, Ithaca, NY, USA · Microsoft Healthcare & Life Sciences, Montr´eal, QC, Canada · Cornell University, Ithaca, NY +1

  25. Blinded Radiologist and LLM-Based Evaluation of LLM-Generated Japanese Translations of Chest CT Reports: Comparative Study

    Apr 2, 2026Yosuke Yamagishi, Atsushi Takamatsu, Yasunori Hamaguchi +4Radiology Report EvaluationMT Evaluation

    Division of Radiology and Biomedical Engineering, Graduate School of Medicine, The University of Tokyo, Tokyo, Japan · Department of Radiology, The University of Tokyo Hospital, Tokyo, Japan · Department of Radiology, Kanazawa University Graduate School of Medical Sciences, Ishikawa, Japan +2

  26. Ran Score: a LLM-based Evaluation Score for Radiology Report Generation

    Mar 24, 2026Ran Zhang, Yucong Lin, Zhaoli Su +13Radiology Report EvaluationRadiology Report Generation

    School of Optics and Photonics, Beijing Institute of Technology, Beijing 100081, China · Zhengzhou Research Institute, Beijing Institute of Technology, Zhengzhou 450003, China · School of Medical Technology, Beijing Institute of Technology, Beijing 100081, China +7

  27. Bridging the Skill Gap in Clinical CBCT Interpretation with CBCTRepD

    Mar 11, 2026Qinxin Wu, Fucheng Niu, Hengchuan Zhu +9Radiology Report GenerationHuman-AI Collaboration

    Stomatology Hospital, School of Stomatology, Zhejiang University School of Medicine, Zhejiang Provincial Clinical Research Center for Oral Diseases, Zhejiang Key Laboratory of Oral Biomedical, Hangzhou, China. · College of Computer Science and Technology, Zhejiang University-University of Illinois Urbana-Champaign Institute, Zhejiang University, Hangzhou 310027, Zhejiang, China.

  28. PET-F2I: A Comprehensive Benchmark and Parameter-Efficient Fine-Tuning of LLMs for PET/CT Report Impression Generation

    Mar 11, 2026Yuchen Liu, Wenbo Zhang, Liling Peng +6Radiology Report GenerationMedical LLMs

    Fudan University · Shanghai Academy of Artificial Intelligence for Science · Shanghai Universal Medical Imaging Diagnostic Center +1

  29. CRIMSON: A Clinically-Grounded LLM-Based Metric for Generative Radiology Report Evaluation

    Mar 6, 2026Mohammed Baharoon, Thibault Heintz, Siavash Raissi +6Radiology Report EvaluationLLM Evaluation

    Department of Biomedical Informatics, Harvard Medical School, Boston, MA · Department of Radiation Oncology, Mass General Brigham, Boston, MA · King Fahad Hospital, Al-Ahsa Health Cluster, Al Hofuf, Saudi Arabia +2

  30. RADAR: A Multimodal Benchmark for 3D Image-Based Radiology Report Review

    Mar 4, 2026Zhaoyi Sun, Minal Jagtiani, Wen-wai Yim +4Radiology Report EvaluationMedical Image Benchmarks

    University of Washington, Seattle, WA 98195, USA · Microsoft Health AI, Redmond, WA 98052, USA