Medical Image Benchmarks

Ask about this topic

Overview

Medical image benchmarks provide datasets, annotations, and shared protocols for comparing systems on tasks such as segmentation, diagnosis, image synthesis, and report generation across modalities including CT, MRI, X-rays, ultrasound, and pathology. They evaluate methods ranging from U-Net and nnU-Net models to vision transformers, foundation models, and multimodal language models, using measures such as Dice, AUROC, calibration, and expert review. A central challenge is whether strong results transfer across patients, hospitals, scanners, populations, and difficult cases; external testing, patient-level splits, and audits for leakage and shortcuts help expose gaps hidden by average scores. The field is moving toward workflow-based and evidence-grounded evaluation, but there is still tension between standardized, easily comparable scores and measures that reflect clinical safety, uncertainty, and real-world usefulness.

Latest papers

CardsList

339 papers31 papers in the last four weeks, up 7% on the four weeks before. 0.2% of all new papers.

  1. A Systematic Benchmark of GAN Architectures for MRI-to-CT Synthesis

    Mar 13, 2026Alessandro Pesci, Valerio Guarrasi, Marco Alì +2Medical Image SynthesisGenerative Adversarial Networks

    Unit of Artificial Intelligence and Computer Systems, Department of Engineering, Università Campus Bio-Medico di Roma, Italy · CDI Centro Diagnostico Italiano S.p.A., Milan, Italy · University of Milan-Bicocca, Milan, Italy +1

  2. GP-VM×\timesSMA: Benchmarking General-Purpose Vision Models and Specialized Architectures for 2D Medical Image Segmentation

    Mar 13, 2026Vanessa Borst, Anna Riedmann, Samuel KounevImage SegmentationVision Foundation Models

    Julius-Maximilians-University Würzburg, 97070 Würzburg, Germany

  3. Unmasking Biases and Reliability Concerns in Convolutional Neural Networks Analysis of Cancer Pathology Images

    Mar 12, 2026Michael Okonoda, Eder Martinez, Abhilekha Dalal +1Dataset BiasConvolutional Neural Networks

    Department of Computer Science, Kansas State University, Manhattan, KS 66506, USA

  4. FL-MedSegBench: A Comprehensive Benchmark for Federated Learning on Medical Image Segmentation

    Mar 12, 2026Meilu Zhu, Zhiwei Wang, Axiu Mao +4Federated LearningMedical Image Benchmarks

    Department of Electrical and Computer Engineering, The University of Hong Kong, Hong Kong, China · School of Communication Engineering, Hangzhou Dianzi University, Hang Zhou, China · Department of Radiation Oncology, Stanford University, California, USA +1

  5. Prompting with the human-touch: evaluating model-sensitivity of foundation models for musculoskeletal CT segmentation

    Mar 11, 2026Caroline Magg, Maaike A. ter Wee, Johannes G. G. Dobbe +4Medical Image SegmentationMedical Imaging Foundation Models

    Quantitative Healthcare Analysis (QurAI) Group, University of Amsterdam, Science Park 900, Amsterdam, 1098 XH, The Netherlands · Department Biomedical Engineering and Physics, Amsterdam UMC, Meibergdreef 9, Amsterdam, 1105 AZ, The Netherlands · Department Orthopaedics, Amsterdam UMC, Meibergdreef 9, Amsterdam, 1105 AZ, The Netherlands

  6. MedQ-Deg: A Multidimensional Benchmark for Evaluating MLLMs Across Medical Image Quality Degradations

    Mar 8, 2026Jiyao Liu, Junzhi Ning, Chenglong Ma +14MLLM EvaluationMedical VLMs

    Fudan University, Shanghai, China · Shanghai AI Lab, Shanghai, China · Shanghai Jiaotong University, Shanghai, China +1

  7. OPTED: Open Preprocessed Trachoma Eye Dataset Using Zero-Shot SAM 3 Segmentation

    Mar 6, 2026Kibrom Gebremedhin, Hadush Hailu, Bruk GebregziabherOphthalmic Image AnalysisMedical Image Analysis

    Department of Computer Science, Mekelle University, Mekelle, Ethiopia · Department of Computer Science, Maharishi International University, Fairfield, IA, USA · Signal Technologies, Germany

  8. CoRe-BT: A Multimodal Radiology-Pathology-Text Benchmark for Robust Brain Tumor Typing

    Mar 4, 2026Juampablo E. Heras Rivera, Daniel K. Low, Xavier Xiong +5Brain Tumor ClassificationMedical Image Benchmarks

    University of Washington, Seattle, Washington 98105, USA · University of Washington School of Medicine, Seattle, Washington 98105, USA · Microsoft Health AI, Redmond, Washington 98052-6399, USA

  9. RADAR: A Multimodal Benchmark for 3D Image-Based Radiology Report Review

    Mar 4, 2026Zhaoyi Sun, Minal Jagtiani, Wen-wai Yim +4Radiology Report EvaluationMedical Image Benchmarks

    University of Washington, Seattle, WA 98195, USA · Microsoft Health AI, Redmond, WA 98052, USA

  10. Designing UNICORN: a Unified Benchmark for Imaging in Computational Pathology, Radiology, and Natural Language

    Mar 3, 2026Michelle Stegeman, Lena Philipp, Fennie van der Graaf +20Computational PathologyMedical Imaging Foundation Models

    and on behalf of the UNICORN consortium

  11. CTForensics: A Comprehensive Dataset and Method for AI-Generated CT Image Detection

    Mar 2, 2026Yiheng Li, Zichang Tan, Guoqing Xu +3Image Forgery DetectionAI-Generated Image Detection

    School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing, China · MAIS, Institute of Automation, Chinese Academy of Sciences, Beijing, China · Sangfor Technologies Inc., Shenzhen, China +1

  12. The MAMA-MIA Challenge: Advancing Generalizability and Fairness in Breast MRI Tumor Segmentation and Treatment Response Prediction

    Mar 1, 2026Lidia Garrucho, Smriti Joshi, Kaisar Kushibar +43Medical Image SegmentationMedical Image Benchmarks

  13. GroundedSurg: A Multi-Procedure Benchmark for Language-Conditioned Surgical Tool Segmentation

    Mar 1, 2026Tajamul Ashraf, Abrar Ul Riyaz, Wasif Tak +4Medical Image GroundingReferring Image Segmentation

    King Abdullah University of Science and Technology (KAUST), Saudi Arabia · Gaash Research Lab, National Institute of Technology Srinagar, India · Thapar Institute of Engineering and Technology, India +1

  14. Specializing Foundation Models via Mixture of Low-Rank Experts for Comprehensive Head CT Analysis

    Feb 28, 2026Youngjin Yoo, Han Liu, Bogdan Georgescu +14Medical Imaging Foundation ModelsLow-Rank Adaptation

    Digital Technology and Innovation, Siemens Healthineers, Princeton, NJ USA · Department of Computed Tomography, Siemens Healthineers, Forchheim, Germany · Department of Radiology, University of Maryland Medical Center, Baltimore, MD USA +4

  15. MM-NeuroOnco: A Multimodal Benchmark and Instruction Dataset for MRI-Based Brain Tumor Diagnosis

    Feb 26, 2026Feng Guo, Jiaxiang Liu, Yang Li +2Medical Image BenchmarksBrain MRI

    Guangdong Institute of Intelligence Science and Technology, Hengqin, China · Center for Brain-Inspired Computing Research (CBICR), Department of Precision Instrument, Tsinghua University, Beijing, China

  16. GazeXPErT: An Expert Eye-tracking Dataset for Interpretable and Explainable AI in Oncologic FDG-PET/CT Scans

    Feb 26, 2026Joy T Wu, Daniel Beckmann, Sarah Miller +15Explainable Medical Image AnalysisMedical Image Benchmarks

    Stanford University/Stanford Health Care, Radiology Department, Stanford, CA 94305, USA · University of Münster, Computer Vision and Machine Learning Systems, Heisenbergstraße 2, Münster 48149, Germany · University of Illinois Urbana-Champaign, School of Information Sciences, Champaign, IL 61820, USA +1

  17. Overview of the CXR-LT 2026 Challenge: Multi-Center Long-Tailed and Zero Shot Chest X-ray Classification

    Feb 25, 2026Hexin Dong, Yi Lin, Pengyu Zhou +7Chest X-Ray ClassificationMedical Image Benchmarks

    Weill Cornell Medicine · Peking Union Medical College · University of Minnesota +2

  18. Virtual Biopsy for Intracranial Tumors Diagnosis on MRI

    Feb 25, 2026Xinzhe Luo, Shuai Shao, Yan Wang +3Brain Tumor ClassificationBrain MRI

    School of Artificial Intelligence and Data Science, University of Science and Technology of China

  19. DM4CT: Benchmarking Diffusion Models for Computed Tomography Reconstruction

    Feb 20, 2026Jiayang Shi, Daniel M. Pelt, K. Joost BatenburgComputed TomographyDiffusion-Based Inverse Problems

    LIACS, Leiden University · Centrum Wiskunde en Informatica

  20. Cholec80-port: A Geometrically Consistent Trocar Port Segmentation Dataset for Robust Surgical Scene Understanding

    Feb 19, 2026Shunsuke Kikuchi, Atsushi Kouno, Hiroki MatsuzakiMedical Image SegmentationMedical Image Benchmarks

    Jmees Inc., Kashiwa-city, Chiba, Japan

  21. OmniCT: Towards a Unified Slice-Volume LVLM for Comprehensive CT Analysis

    Feb 18, 2026Tianwei Lin, Zhongwei Qiu, Wenqiao Zhang +12Radiology VLMsMedical VLMs

    Zhejiang University · DAMO Academy, Alibaba Group · Hupan Lab

  22. CT-Bench: A Benchmark for Multimodal Lesion Understanding in Computed Tomography

    Feb 16, 2026Qingqing Zhu, Qiao Jin, Tejas S. Mathai +10Medical Image BenchmarksComputed Tomography

  23. MedAD-R1: Consistency-Reinforced Policy Optimization for Interpretable Medical Anomaly Detection

    Feb 1, 2026Haitao Zhang, Yingying Wang, Jiaxiang Wang +5HealthcareGroup Relative Policy Optimization

    School of Informatics, Xiamen University · Institute of Artificial Intelligence, Xiamen University · Zhejiang Expressway Co., Ltd. +3

  24. M3CoTBench: Benchmark Chain-of-Thought of MLLMs in Medical Image Understanding

    Jan 13, 2026Juntao Jiang, Jiangning Zhang, Yali Bi +7Medical VLMsMultimodal CoT Reasoning

    1Zhejiang University · University of Science and Technology of China · 3East China Normal University +2

  25. Region-Specific Calibration Achieves Excellent Inter-Device Reliability for Smartphone Dermatology: A Multi-Device Benchmark on Korean Facial Skin

    Dec 26, 2025Sungwoo Kang, Jong-Kook KimMedical DiagnosisDermatology

    Department of Electrical and Computer Engineering, Korea University, Seoul, Republic of Korea · School of Electrical Engineering, Korea University, Seoul, Republic of Korea

  26. MedInsightBench: Evaluating Medical Analytics Agents Through Multi-Step Insight Discovery in Multimodal Medical Data

    Dec 15, 2025Zhenghao Zhu, Chuxue Cao, Sirui Han +4Data Analysis AgentsMultimodal Model Evaluation

    The Hong Kong University of Science and Technology, Hong Kong, China · ByteDance, China

  27. 6 Fingers, 1 Kidney: Natural Adversarial Medical Images Reveal Critical Weaknesses of Vision-Language Models

    Dec 3, 2025Leon Mayer, Piotr Kalinowski, Caroline Ebersbach +6VLM EvaluationVLM Robustness

    German Cancer Research Center (DKFZ) Heidelberg, Division of Intelligent Medical Systems, Germany. · Medical Faculty, Heidelberg University, Germany. · Faculty of Mathematics and Computer Science, Heidelberg University, Germany. +9

  28. MedGEN-Bench: A Contextually Entangled Benchmark for Open-ended Multimodal Medical Generation

    Nov 17, 2025Junjie Yang, Yuhao Yan, Gang Wu +12HealthcareMultimodal Generation

    South China University of Technology · Sun Yat-sen University · Hangzhou Dianzi University +10

  29. MediRound: Multi-Round Entity-Level Reasoning Segmentation in Medical Images

    Nov 15, 2025Qinyue Tong, Ziqian Lu, Jun Liu +3Interactive Medical Image SegmentationText-Guided Image Segmentation

    Zhejiang University, Hangzhou, Zhejiang, China · Zhejiang Sci-Tech University, Hangzhou, Zhejiang, China · National University of Singapore, Singapore

  30. Scaling Vision Transformers for Functional MRI with Flat Maps

    Oct 15, 2025Connor Lane, Mihir Tripathy, Leema Krishna Murali +15fMRI Foundation ModelsMasked Autoencoders

    MedARC · Sophont · Baylor College of Medicine +3