Diagnostic Benchmark

Follow this topic Ask about this topic

Recent momentum

-59%

14 papers in the last 28 days · 0.2% of indexed attention

Twelve weeks of publication activity for this topic as it is defined today.

Weekly history

Recent digests

What was published in this topic, kept on the site without email delivery.

Period ending 2026-09-21

4 new papers

A weekly snapshot of new work published in Diagnostic Benchmark.

Period ending 2026-09-14

5 new papers

A weekly snapshot of new work published in Diagnostic Benchmark.

Period ending 2026-09-07

4 new papers

A weekly snapshot of new work published in Diagnostic Benchmark.

179 papers

Latest in Diagnostic Benchmark

Browse all topics →

JMed48k: A Multi-Profession Japanese Medical Licensing Benchmark for Vision-Language Model Evaluation
May 21, 2026Yue Xun, Junyu Liu, Qian Niu +10Medical Visual Question Answering Examiners
Active Evidence-Seeking and Diagnostic Reasoning in Large Language Models for Clinical Decision Support
May 21, 2026Chen Zhan, Xihe Qiu, Xiaoyu Tan +8Clinical Decision Support Diagnostic Benchmark
MedExpMem: Adapting Experience Memory for Differential Diagnosis
May 20, 2026Qianhan Feng, Zhongzhen Huang, Yakun Zhu +4Clinical Reasoning Diagnostic Benchmark
What Do Biomedical NER and Entity Linking Benchmarks Measure? A Corpus-Centric Diagnostic Framework
May 19, 2026Robert Leaman, Rezarta Islamaj, Zhiyong LuMedical Ontology Named-Entity Recognition
POLAR-Bench: A Diagnostic Benchmark for Privacy-Utility Trade-offs in LLM Agents
May 18, 2026Qiaoyuan Zheng, Yiqu Yang, Qi Gao +1Adversarial Evaluation Privacy-Utility Trade-Offs
How Good LLMs Are at Answering Bangla Medical Visual Questions? Dataset and Benchmarking
May 18, 2026Rafid Ahmed, Intesar Tahmid, Mir Sazzat Hossain +3Medical Visual Question Answering Bangla
VCG-Bench: Towards A Unified Visual-Centric Benchmark for Structured Generation and Editing
May 15, 2026Xiaoyan Su, Peijie Dong, Zhenheng Tang +8Diagrams Diagnostic Benchmark
MI-CXR: A Benchmark for Longitudinal Reasoning over Multi-Interval Chest X-rays
May 15, 2026Sunghwan Steve Cho, Yunseok Han, Jaeyoung DoChest X-Ray Chest Radiography
RxEval: A Prescription-Level Benchmark for Evaluating LLM Medication Recommendation
May 14, 2026Shuhao Chen, Weisen Jiang, Changmiao Wang +4Medication Recommendation Framework Clinical Reasoning
A Multi-Probe Audit of Clinical-Interview Depression Detection Benchmarks
May 13, 2026Takehiro Ishikawa, Jon DukeDepression Detection Diagnostic Benchmark
AcuityBench: Evaluating Clinical Acuity Identification and Uncertainty Alignment
May 12, 2026Robin Linzmayer, Georgianna Lin, Di Coneybeare +22Triage Diagnostic Benchmark
ABRA: Agent Benchmark for Radiology Applications
May 11, 2026Bulat Maksudov, Vladislav Kurenkov, Kathleen M. Curran +1Radiology Report Generation Agentic Benchmarks
ClinicalBench: Stress-Testing Assertion-Aware Retrieval for Cross-Admission Clinical QA on MIMIC-IV
May 11, 2026Alex StinardClinical Reasoning Diagnostic Benchmark
Verification Mirage: Mapping the Reliability Boundary of Self-Verification in Medical VQA
May 11, 2026Ruinan Jin, Beidi Zhao, Myeongkyun Kang +2Medical Visual Question Answering Verifier
Med-StepBench: A Hierarchical Reasoning Framework for Evaluating Hallucinations in Medical Vision-Language Models
May 11, 2026Minh Khoi Nguyen, Dai Lam Le, Amir Reza Jafari +8Medical Vision-Language Models Robust Hallucination Detection
TOC-Bench: A Temporal Object Consistency Benchmark for Video Large Language Models
May 11, 2026Junzhe Chen, Siyuan Meng, Yuxi Chen +3Video Understanding Spatio-Temporal Coherence
DeepTumorVQA: A Hierarchical 3D CT Benchmark for Stage-Wise Evaluation of Medical VLMs and Tool-Augmented Agents
May 10, 2026Yixiong Chen, Wenjie Xiao, Pedro R. A. S. Bassi +7Medical Vision-Language Models Textvqa
EpiGraph: Building Generalists for Evidence-Intensive Epilepsy Reasoning in the Wild
May 10, 2026Yuyang Dai, Zheng Chen, Jathurshan Pradeepkumar +4Reasoning Graphs Knowledge Graphs
MedVIGIL: Evaluating Trustworthy Medical VLMs Under Broken Visual Evidence
May 8, 2026Hanqi Jiang, Junhao Chen, Mingyu Kang +12Medical Visual Question Answering Medical Vision-Language Models
MedExAgent: Training LLM Agents to Ask, Examine, and Diagnose in Noisy Clinical Environments
May 8, 2026Yicheng Gao, Xiaolin Zhou, Yahan Li +2Clinical Reasoning Diagnostic Benchmark
CXR-ContraBench: Benchmarking Negated-Option Attraction in Medical VLMs
May 7, 2026Zhengru Fang, Yanan Ma, Yu Guo +5Chest X-Ray Medical Vision-Language Models
MRI-Eval: A Tiered Benchmark for Evaluating LLM Performance on MRI Physics and GE Scanner Operations Knowledge
May 6, 2026Perry E. RadauDiagnostic Benchmark Large Language Model Evaluation
Ask Before You Diagnose: Safe-Psych, a Sequential Evaluation Benchmark for LLMs in Psychiatry
May 6, 2026Oriana Presacan, Andreea Grama, Larisa Irimină +6Psychiatric Reasoning Diagnostic Benchmark
DALPHIN: Benchmarking Digital Pathology AI Copilots Against Pathologists on an Open Multicentric Dataset
May 5, 2026Carlijn Lems, Sander Moonemans, Natálie Klubíčková +53Computational Pathology Diagnostic Benchmark
PhysicianBench: Evaluating LLM Agents in Real-World EHR Environments
May 4, 2026Ruoqi Liu, Imran Q. Mohiuddin, Austin J. Schoeffler +10Real-World Clinical Workflows Electronic Health Records
ISAAC: Auditing Causal Reasoning in Deep Models for Drug-Target Interaction
May 3, 2026Barbara Tarantino, Sun Kim, Yijingxiu Lu +1Clinical Prediction Diagnostic Benchmark
DiagramNet: An End-to-End Recognition Framework and Dataset for Non-Standard System-Level Diagrams
May 2, 2026Jincheng Lou, Ruohan Xu, Jiapeng Li +6Diagrams Diagnostic Benchmark
WildTableBench: Benchmarking Multimodal Foundation Models on Table Understanding In the Wild
May 1, 2026Junzhe Huang, Xiaoxiao Sun, Yan Yang +6Tabular Data Multimodal Benchmarks
MedMosaic: A Challenging Large Scale Benchmark of Diverse Medical Audio
May 1, 2026Harshit Rajgarhia, Shuubham Ojha, Asif Shaik +4Multimodal Clinical Data Diagnostic Benchmark
From ML Predictions to Informed Diagnostic Assistance Using the Toulmin Model of Argumentation
May 1, 2026Anca Marginean, Adrian GrozaDiagnostic Benchmark Biomarker
Are Multimodal LLMs Ready for Clinical Dermatology? A Real-World Evaluation in Dermatology
May 1, 2026Roy Jiang, Hyunjae Kim, Zhenyue Qin +8Multimodal Clinical Data Skin Lesion Classification
DIAGRAMS: A Review Framework for Reasoning-Level Attribution in Diagram QA
Apr 29, 2026Anirudh Iyengar Kaniyar Narayana Iyengar, Tampu Ravi Kumar, Manan Suri +4Diagrams Reasoning Graphs
DRAGON: A Benchmark for Evidence-Grounded Visual Reasoning over Diagrams
Apr 28, 2026Anirudh Iyengar Kaniyar Narayana Iyengar, Tampu Ravi Kumar, Gaurav Najpande +4Diagrams Visual Reasoning
MultiDx: A Multi-Source Knowledge Integration Framework towards Diagnostic Reasoning
Apr 27, 2026Yimin Deng, Zhenxi Lin, Yejing Wang +9Clinical Reasoning Diagnostic Benchmark
CT-FineBench: A Diagnostic Fidelity Benchmark for Fine-Grained Evaluation of CT Report Generation
Apr 27, 2026Ruifeng Yuan, Wanxing Chang, Weiwei Cao +4Diagnostic Benchmark
Agri-CPJ: A Training-Free Explainable Framework for Agricultural Pest Diagnosis Using Caption-Prompt-Judge and LLM-as-a-Judge
Apr 26, 2026Wentao Zhang, Qi Zhang, Mingkun Xu +6Agricultural Landscapes Remote Sensing Image Change Captioning
VeriLLMed: Interactive Visual Debugging of Medical Large Language Models with Knowledge Graphs
Apr 25, 2026Yurui Xiang, Xingyi Mao, Rui Sheng +7Clinical Reasoning Precise Debugging
Avionic Main Fuel Pump Simulation and Fault-Diagnosis Benchmark
Apr 23, 2026Felix Leonhard Janzen, Lukas Moddemann, Alexander Diedrich +1Anomaly Detection Cyber-Physical System
Thinking Like a Botanist: Challenging Multimodal Language Models with Intent-Driven Chain-of-Inquiry
Apr 22, 2026Syed Nazmus Sakib, Nafiul Haque, Shahrear Bin Amin +4Diagnostic Benchmark Visual Reasoning
X-PCR: A Benchmark for Cross-modality Progressive Clinical Reasoning in Ophthalmic Diagnosis
Apr 22, 2026Gui Wang, Zehao Zhong, YongSong Zhou +6Multimodal Clinical Data Ophthalmology
DistortBench: Benchmarking Vision Language Models on Image Distortion Identification
Apr 21, 2026Divyanshu Goyal, Akhil Eppa, Vanya Bannihatti KumarDistortion Multimodal Benchmarks
MedPRMBench: A Fine-grained Benchmark for Process Reward Models in Medical Reasoning
Apr 19, 2026Lingyan Wu, Xiang Zheng, Weiqi Zhai +5Clinical Reasoning Reasoning Benchmark
Evaluating Multimodal LLMs for Inpatient Diagnosis: Real-World Performance, Safety, and Cost Across Ten Frontier Models
Apr 18, 2026Bruce A. Bassett, Amy Rouillard, Sitwala Mundia +8Multimodal Clinical Data Large Language Model Performance
MADE: A Living Benchmark for Multi-Label Text Classification with Uncertainty Quantification of Medical Device Adverse Events
Apr 16, 2026Raunak Agarwal, Markus Wenzel, Simon Baur +3Multi-Label Classification Diagnostic Benchmark
Can LLMs Accurately Score Medical Diagnoses and Clinical Reasoning?
Apr 16, 2026Amy Rouillard, Sitwala Mundia, Linda Camara +8Clinical Decision Support Clinical Reasoning
MedLayBench-V: A Large-Scale Benchmark for Expert-Lay Semantic Alignment in Medical Vision Language Models
Apr 7, 2026Han Jang, Junhyeok Lee, Heeseong Eum +1Medical Vision-Language Models Multimodal Benchmarks
LIBERO-Para: A Diagnostic Benchmark and Metrics for Paraphrase Robustness in VLA Models
Mar 30, 2026Chanyoung Kim, Minwoo Kim, Minseok Kang +2Vision-Language-Action Models Large Vision Language Models
RA-QA: A Benchmarking System for Respiratory Audio Question Answering Under Real-World Heterogeneity
Feb 4, 2026Gaia A. Bertolino, Yuwei Zhang, Tong Xia +2Respiratory Sound Classification Large Audio Language Models
Think Like a Doctor: Conversational Diagnosis through the Exploration of Diagnostic Knowledge Graphs
Feb 2, 2026Jeongmoon Won, Seungwon Kook, Yohan JoSimulated Patients Diagnostic Benchmark
M3CoTBench: Benchmark Chain-of-Thought of MLLMs in Medical Image Understanding
Jan 13, 2026Juntao Jiang, Jiangning Zhang, Yali Bi +7Medical Visual Question Answering Diagnostic Benchmark
EpiQAL: Benchmarking Large Language Models in Epidemiological Question Answering and Reasoning
Jan 6, 2026Mingyang Wei, Dehai Min, Zewen Liu +8Epidemic Forecasting Clinical Reasoning
6 Fingers, 1 Kidney: Natural Adversarial Medical Images Reveal Critical Weaknesses of Vision-Language Models
Dec 3, 2025Leon Mayer, Piotr Kalinowski, Caroline Ebersbach +6Medical Vision-Language Models Multimodal Benchmarks
MedRepBench: A Comprehensive Benchmark for Medical Report Interpretation
Aug 21, 2025Fangxin Shang, Yuan Xia, Dalu Yang +2Medical Visual Question Answering Medical Report Generation
EndoCogniAgent: Closed-Loop Agentic Reasoning with Self-Consistency Validation for Endoscopic Diagnosis
Aug 10, 2025Yi Tang, Kai-Ni Wang, Liang-Peng Pu +3Video Capsule Endoscopy Diagnostic Benchmark
Causality for Tabular Data Synthesis: A High-Order Structure Causal Benchmark Framework
Jun 12, 2024Zineb Senane, Axel Karlsson, Lele Cao +6Synthetic Tabular Data Structural Causal Model
Timely Clinical Diagnosis through Active Test Selection
Date pendingSilas Ruhrberg Estévez, Nicolás Astorga, Mihaela van der SchaarDiagnostic Benchmark Model Selection
MedGEN-Bench: A Contextually Entangled Benchmark for Open-ended Multimodal Medical Generation
Date pendingJunjie Yang, Yuhao Yan, Gang Wu +12Medical Visual Question Answering Medical Vision-Language Models
VeriSim: A Configurable Framework for Stress-Testing Medical AI Under Patient Communication Noise
Date pendingSina Mansouri, Mohit Marvania, Vibhavari Ashok Shihorkar +5Simulated Patients Diagnostic Benchmark
Solve the Missing First Step: Can VLMs Standardize Raw Heterogeneous Medical Data?
Date pendingXin Chen, Dongliang Xu, Cunhao Zhu +5Diagnostic Benchmark Multimodal Clinical Data

May 21, 2026 · cs.CVJ/K move · Enter open · S save

JMed48k: A Multi-Profession Japanese Medical Licensing Benchmark for Vision-Language Model Evaluation

Yue Xun, Junyu Liu, Qian Niu, Xinyi Wang+9

We introduce JMed48k, a multi-profession Japanese healthcare licensing benchmark for evaluating vision-language models. Built from official PDF materials released by the Japanese Ministry of Health, Labour and Welfare, JMed48k contains 48,862 exam questions and 20,142 images from 11 national licensing examinations between 2005 and 2025, with visual content annotated under an 8-type taxonomy. From this corpus, we derive JMed48k-Eval, a recent five-year evaluation subset with 12,484 scored questions, including 9,905 text-only questions and 2,579 questions with images. We evaluate 21 proprietary, open-source, and medical-specific models, reporting text-only and with-image performance separately. Because these subsets contain different questions, we further introduce a paired image-removal audit that evaluates questions with images before and after removing visual content to explore four answer-transition states. The audit shows that proprietary and open source models gain substantially from images, whereas medical-specific systems show limited observable use of visual evidence, with many correct answers persisting after image removal. Even among proprietary models, the net image-removal effect varies sevenfold across professions, from +5.7 points on Physician questions to +39.8 points on Public Health Nurse questions. We release JMed48k to support reproducible, profession-stratified evaluation of vision-language models in medical licensing settings.

Medical Visual Question Answering Examiners Diagnostic Benchmark Vision-Language

Open paper Ask about it PDF