Diagnostic Benchmark

Ask about this topic

Momentum

9 papers in the last four weeks, down 18% on the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28
View on map

Latest papers 116

All topics
CardsList

  1. MS-Exam-Gen: Source-Grounded Benchmark Construction for Evaluating LLMs on Textual Multiple Sclerosis MRI Knowledge

    Oct 5, 2026Abdul Basit, Muhammad Abdullah Hanif, Muhammad ShafiqueMultiple SclerosisDiagnostic Benchmark

    eBRAIN Lab, Division of Engineering, New York University Abu Dhabi (NYUAD), Abu Dhabi, UAE

  2. R-GroundBench: A Diagnostic Benchmark for R-Group Groundingin Markush Molecular Editing

    Sep 30, 2026Xin Wang, Zichuan Ying, Xinna Lin +8Chemical StructuresMoleculenet

    Westlake University · The University of Hong Kong · Shanghai Innovation Institute +5

  3. KlinikeBench: Evaluating Language Models Beyond Diagnostic Accuracy

    Sep 29, 2026Xueting Fang, Zehui Li, Yang Yang +5Clinician TrustDiagnostic Benchmark

    Zhejiang University · Imperial College London · Nanchang University +2

  4. Beyond Legibility: Benchmarking Visual Text Rendering and In-Place Editing in Unified Video Generation

    Sep 29, 2026Ziying Zhang, Litao Li, Junchao Liao +4Text-To-Video Generation ModelFew-Shot Font Generation

    Alibaba Group · Shanghai Jiao Tong University · Fudan University

  5. AgentHop: A Diagnostic Benchmark for Agentic Multi-Hop Scientific Question Answering

    Sep 28, 2026Chanhee Park, Jeongho Yoon, Sungbin Han +2Agentic BenchmarksDiagnostic Benchmark

    Korea University · Sookmyung Women’s University

  6. PhysAlign: A Benchmark for Evidence-Grounded Role Alignment in Multimodal Physics Reasoning

    Sep 27, 2026Kecheng Liang, Haoyang Liu, Zexin Chen +5DiagramsMultimodal Reasoning

    Sun Yat-sen University · Dalian University of Technology · Xi’an Jiaotong-Liverpool University

  7. EyeVQA: Benchmarking Ophthalmic Vision-Language Models from Recognition to Spatial Grounding

    Sep 26, 2026Gujie Shao, Zixun Xie, Xuechun Xing +7OphthalmologyDiagnostic Benchmark

  8. LegendBench: A Diagnostic Benchmark for Legend Understanding with Counterfactual Interventions

    Sep 21, 2026Xinnuo Zhang, Zhike Tang, Jing Xu +2ChartDiagnostic Benchmark

    The Hong Kong University of Science and Technology (Guangzhou)

  9. Same Words, Different Actions: Paired Turn-Taking Evaluation under Rewritten Dialogue Contexts

    Sep 15, 2026Shuofeng Zhao, Hongwei Cai, Wenke Fan +9Turn-TakingDialogue Benchmarks

    Zuoyebang Education Technology, Beijing, China

  10. MedRoundsQA: A Persona and Difficulty Aware Evaluation for Multi-Turn Medical Consultations

    Sep 14, 2026Youssef Mohamed, Ahmed Heakl, Qinrong Cui +13Diagnostic Benchmark

    MBZUAI · Cairo University · Ain Shams University +2

  11. ExBind: A Controlled Diagnostic Benchmark for Visual-to-Executable Correspondence

    Sep 1, 2026Ziqian Wang, Yuxiao Cheng, Tingxiong Xiao +1Diagnostic Benchmark

    Department of Automation, Tsinghua University Beijing 100084, China

  12. Benchmarking Vision-Language Models for Automated Pathology Diagnosis and Report Generation

    Sep 1, 2026Yumi Lee, Harim Oh, Hyoryung Kim +52Medical Vision-Language ModelsWhole-Slide Images

  13. Reliable Benchmarking of Artifact Detection in Computational Pathology: A Reproducibility and Uncertainty Analysis

    Aug 31, 2026Konstantinos Moutselos, Ilias MaglogiannisWhole-Slide ImagesComputational Pathology

    Department of Digital Systems, University of Piraeus, Piraeus, Greece

  14. Polish Medical Visual Question Answering: Vision-Language Models Underutilize Visual Evidence

    Aug 13, 2026Jakub Pokrywka, Łukasz Grzybowski, Antoni Lasik +3Medical Visual Question AnsweringMultiple-Choice Visual Question Answering

    Adam Mickiewicz University · ARAAI Poland · NASK National Research Institute +2

  15. Diagram-MMU: A Multi-Modal Benchmark for Scientific Diagrams

    Aug 12, 2026Weihao Bo, Shan Zhang, Yanpeng Sun +7DiagramsDiagnostic Benchmark

    Nanjing University of Science and Technology · Baidu Inc · AIML, Adelaide University +4

  16. A corpus-specific clinical RAG system matches or outperforms newer frontier LLMs on HealthBench

    Aug 12, 2026Praveen Reddy, Charuta Mandke, Suvrankar Datta +4Clinical Reasoning TrainingDiagnostic Benchmark

  17. Chartography: A Benchmark for Professional Chart Understanding

    Aug 11, 2026Suhaas Garre, Chris Mutty, Sushant Mehta +1ChartDiagnostic Benchmark

    Surge AI*

  18. FormStruct-Bench:A Hierarchical and Diagnostic Benchmark for Table-Form Document Structure Recognition

    Aug 11, 2026Lujie Ban, Jiangtao Zhu, Yuanheng Yu +2Structured ContextDocument

    The Chinese University of Hong Kong, Shenzhen · University of Washington

  19. From Diagnosis to Correction: Benchmarking and Improving Real-World Table Parsing

    Aug 10, 2026Jutao Xiao, Yuan Qu, Dongsheng Ma +7Document ParsingTabular Data

    Zhejiang University · Shanghai Artificial Intelligence Laboratory · 1Zhejiang University, 2Shanghai Artificial Intelligence Laboratory +3

  20. VideoVIBE: A Video-Grounded Diagnostic Benchmark for One-Shot Interactive Website Generation

    Aug 10, 2026Jiajun Xu, Yanghao Zhou, Jingyun Liao +6Long-Video BenchmarksVibe Coding

    University of Technology Sydney · Beijing Institute of Technology · Hunan University +4

  21. KVDiagnosis: A Diagnostic Benchmark for KV-Cache Compression in Long-Context Language Models

    Aug 10, 2026Chen Qiu, Ziwu Liu, Chao Fei +2Key-Value Cache CompressionDiagnostic Benchmark

    KAUST Thuwal, Saudi Arabia

  22. Math-Vision Diagrams: A Comprehensive Benchmark for Evaluating LLM Mathematical Diagram Generation Capabilities

    Aug 9, 2026Harish Kashyap, Kiran Byadarhaly, Sriram Chakaravarthy +2DiagramsDiagnostic Benchmark

    Pandita AI Inc., 2540 N First St, San Jose CA 95131, USA

  23. TomaMMU: A Comprehensive Multimodal Understanding Benchmark for Tomato Leaf Diseases

    Aug 9, 2026Gia-Han Truong, Khang Nguyen Quoc, Luyl-Da QuachLeaf Disease ClassificationTomato Diseases

    Department of Information Technology, FPT University, Can Tho, Vietnam · School of Electrical Engineering, Korea University, Seoul, South Korea

  24. PluginEval: A Diagnostic Benchmark for Fine-Grained Error Attribution in Function Calling

    Aug 9, 2026Dongjie Xu, Julius, Hanchi Dong +6Large Language Model EvaluationDiagnostic Benchmark

    Soochow University · Tencent Inc. · Beijing University of Posts and Telecommunications +4

  25. DiagChain: A Diagnostic Benchmark for Evaluating LLM Agents on Evidence-Grounded Attack Chain Reconstruction

    Aug 4, 2026Xuyang Liu, Yibin Han, Zhenwei Zhang +8Diagnostic Benchmark

    KLISS, BNRist, School of Software, Tsinghua University, Beijing, China · CRRC Corporation Limited, Beijing, China

  26. LDU-Bench: Multimodal LLM Evaluation for Lithography Defect Understanding under Layout-Varying Circuit Backgrounds

    Aug 4, 2026Huanglong Ji, Botong Zhao, Shujing Lv +1LithographyDiagnostic Benchmark

  27. WorldExam: Benchmarking World Models from Apparent Appearance to Inherent Reactivity

    Aug 3, 2026Yuxue Yang, Shuyao Shang, Jiahe Wang +13Video World ModelsWorld Models

    1CASIA · 4AMAP · 5THU +2

  28. AcoustiTrace: When Plausible Sound Violates Physics

    Aug 3, 2026Shiyang Li, Yuewen Cao, Yihao Liu +4Audio-Video GenerationAcoustic

    Zhejiang University · Shanghai Artificial Intelligence Laboratory · Shanghai Jiao Tong University +3

  29. TreeProbe : A Tibetan Medicine Benchmark for Cultural Bias in LLMs

    Aug 1, 2026Jin Zhang, Linyu Li, Weili Jiang +7Multilingual Healthcare Q&AClinical Reasoning Training

    University of Electronic Science and Technology of China · Peking University · Southwest Jiaotong University +1

  30. Knowledge before Reasoning: EC-Reason-Bench, a Training-Free Diagnostic Benchmark for LLM Enzyme Classification

    Jul 29, 2026Linyu Li, Zhi Jin, Yichi Zhang +6Reasoning BenchmarkDiagnostic Benchmark

    School of Computer Science, Peking University, Beijing, China · College of Computer Science and Technology, Zhejiang University, Hangzhou, China · School of Computer Science, Tibet University, Lhasa, China

  31. Polistemics: Evaluating LLMs as Information Mediators in Politics & Elections

    Jul 28, 2026Baran Peters, Gabor Hollbeck, Robert Jakob +1Political DiscourseEpistemic State Replication

    ETH Agentic Systems Lab · CORDA

  32. Forensic Reproducibility Audit of a Radiology Vision-Language Model Benchmark: From Intended Protocol to Released Artifact

    Jul 28, 2026Mateusz KozłowskiMedical Vision-Language ModelsRadiology Report Generation

    Independent Researcher, Kraków, Poland

  33. PatientAgentBench: A Benchmark Framework for Evaluating Patient-Facing Health AI Agents

    Jul 28, 2026Korosh Vatanparvar, Ashutosh Joshi, Maria Xenochristou +11PatientsDiagnostic Benchmark

    Amazon Health AI

  34. When Does Deep Representation Learning Help Single-Cell Clustering? A Sensitivity-Aware Diagnostic Benchmark for Biomedical AI Pipelines

    Jul 28, 2026Nguyen Thanh Phong, Truong Viet Vu, Nguyen Ha Thu +4Single-Cell RnaClustering

    Faculty of Information Technology, Van Lang School of Technology, Van Lang University, Ho Chi Minh City, Vietnam · Faculty of Law, Van Lang University, Ho Chi Minh City, Vietnam

  35. ERUnderstand: Evaluating Vision-Language Models on Structured ER Diagrams

    Jul 27, 2026Ali Ansari, Yasmin Mohammadi, Farnoush Nili +3DiagramsSchema

    Temple University · Department of Computer and Information Sciences · Philadelphia, PA, USA

  36. GraphRareBench: An Auditable Graph-Evidence Benchmark for Phenotype-Driven Rare-Disease Diagnosis

    Jul 27, 2026Guiling Guo, Jia Yang, Jiahao Xu +3Rare DiseaseBiomedical Knowledge Graph

    National Institute for Data Science in Health and Medicine, Xiamen University · Institute of Artificial Intelligence, Xiamen University · School of Medicine, Xiamen University +1

  37. LA-RL: Label-Aware Self-Reflection for Reinforcement Learning in Information Extraction

    Jul 26, 2026Xiao You, Tianwei Yan, Zixu Shan +2Information ExtractionRelation Extraction

    Hefei University of Technology, Hefei, China · Chongqing Jiaotong University, Chongqing, China

  38. CRAG-MM-Diagnostics: Enabling Stage-Wise Analysis of Knowledge-Intensive VQA

    Jul 23, 2026Hanseok Oh, Parishad BehnamGhader, Benno Krojer +4Knowledge-Based Visual Question AnsweringDiagnostic Benchmark

    New York University, New York NY, USA · Mila - Quebec AI Institute, Montreal, Canada · McGill University, Montreal, Canada +3

  39. PhenSPINE: A Standardized Benchmark for Spine Pathology Diagnosis

    Jul 22, 2026Duong Ngoc Vu, Hai Son Nguyen, Trong-Nghia Nguyen +4Magnetic Resonance Imaging DatasetMulti Organ Computed Tomography Segmentation

    Business AI Lab, College of Technology, National Economics University, Vietnam · Medical Imaging & Radiological Technology Department, Faculty of Medical Technology, Phenikaa School of Medicine & Pharmacy, Phenikaa University, Vietnam · Radiology and Functional Exploration Center, Phenikaa University Hospital, Vietnam +1

  40. PathAgentBench: Benchmarking Evidence-Seeking Vision-Language Models on Whole-Slide Pathology Image

    Jul 21, 2026Dankai Liao, Tianyi Zhang, Yufeng Wu +6Whole-Slide ImagesPathology Foundation Models

    Department of Biomedical Engineering, National University of Singapore, Singapore 117417, Singapore

  41. PathReportEval: A Systematic Benchmark for Pathology Report Generation

    Jul 20, 2026Suryakant Singh, Sejuti Majumder, Beatrice Knudsen +2ReportDiagnostic Benchmark

    Department of Biomedical Informatics Stony Brook University Stony Brook, NY, USA · Department of Pathology University of Utah Salt Lake City, UT, USA

  42. VEHBench: A Stage-Local Diagnostic Benchmark for LLM-Assisted Vibration Energy Harvester Design

    Jul 20, 2026Depeng Su, Yuyu Luo, Guobiao HuLarge Language Models(LlmsDiagnostic Benchmark

    1Internet of Things Thrust, Information Hub 2Data Science and Analytics Thrust, Information Hub The Hong Kong University of Science and Technology (Guangzhou)

  43. Measuring and Improving Complex-Atomic Answer Consistency in Endoscopic VQA

    Jul 20, 2026Yuhao Liu, Cheng Zhao, Guanghui YueColonoscopyDiagnostic Benchmark

    School of Biomedical Engineering, Shenzhen University, Shenzhen, China

  44. Can Multimodal Large Language Models Understand OCT?

    Jul 18, 2026Baochen Fu, Wenzhi Deng, Baihao Jin +5Optical Coherence TomographyMultimodal Benchmarks

    1Shandong University · 4Shandong Provincial Qianfoshan Hospital · University of Science and Technology of China +2

  45. MedFailBench: A Clinician-Built Open-Source Benchmark for Medical AI Safety Boundary Inspection

    Jul 16, 2026Goktug OzkanArtificial Intelligence SafetyClinician Trust

    Department of Internal Medicine, Kutahya Emet Dr. Fazil Dogan State Hospital, Kutahya, Turkiye

  46. MamaBench: Benchmarking LLM Robustness in Maternal and Child Health Diagnosis through Counterfactual Clinical Perturbation

    Jul 15, 2026Thanni Adewuyi, Anuoluwa Sotome, Samuel Okoko +7Clinical Reasoning TrainingDiagnostic Benchmark

    Helpmum Africa · University of Ibadan

  47. Capabilities of Claude Fable 5 on Biomedical Challenge Problems

    Jul 12, 2026Dominic Okonkwo, Magnus Hodgson, Temitope I. David +1Biomedical TextDiagnostic Benchmark

    School of Computing, University of Georgia · Department of Chemistry, University of Illinois · Institute of Bioinformatics, University of Georgia

  48. MultiView-Bench: A Diagnostic Benchmark for World-Centric Multi-View Integration in VLMs

    Jul 9, 2026Hantao Zhang, Jinru Sui, Ed Li +2Multi-View3D Scene Understanding

    Yale University · University of Edinburgh

  49. HamQASBench: A Hamiltonian-Informed Diagnostic Benchmark for Evaluating Quantum Architecture Search

    Jul 6, 2026Jiayang Niu, Akib Karim, Yan Wang +5Diagnostic BenchmarkMoleculenet

    School of Computing Technologies RMIT University · Quantum Systems School of Computing Technologies RMIT University · Data61, CSIRO

  50. MatPhaseBench: A Semantics-Guided Benchmark for Materials Phase Diagrams Understanding

    Jul 3, 2026Hanwen Wang, Sihan Liang, Zhiwei Liu +4Materials ScienceDiagnostic Benchmark

    Computer Network Information Center, Chinese Academy of Sciences University of Chinese Academy of Sciences Beijing, China · University of Manchester Manchester, UK · Institute of Metal Research, Chinese Academy of Sciences Shenyang, China +1

  51. MedStreamBench: A Time-Aware Benchmark for Streaming and Proactive Medical Video Understanding

    Jul 2, 2026Yuan Wang, Shujian Gao, Songtao Jiang +2Medical Visual Question AnsweringDiagnostic Benchmark

    College of Computer Science and Technology, Zhejiang University, Zhejiang, China · Insititute of Trustworthy Embodied AI, Fudan University, Shanghai, China · Department of Electrical and Computer Engineering, Carnegie Mellon University, Pennsylvania, USA

  52. PHREEQC-MCQ-200: A Diagnostic Benchmark for Tool-Augmented Scientific Simulator Agents

    Jul 1, 2026Ke Zhang, Sahchit Chundur, Mohammad Javad Qomi +1MemoryagentbenchScientific Agents

    University of California, Riverside · University of California, Irvine

  53. Beyond Binary Instrument QA: Probing Instrument Grounding in Music Audio-Language Models

    Jun 30, 2026Yujun Lee, Joonhyeok Shin, Hyoeun Kim +1Music Structure AnalysisLarge Audio Language Models

    Sungkyunkwan University

  54. SHOVIR: A Benchmark for Evaluating Vision Shortcut Learning in Radiology Report Generation

    Jun 29, 2026Filippo Ruffini, Marco Salmé, Rosa Sicilia +2Radiology Report GenerationMedical Vision-Language Models

    Unit of Artificial Intelligence and Computer Systems, Department of Engineering, Universit`a Campus Bio-Medico di Roma, Rome, Italy · Department of Diagnostics and Intervention, Radiation Physics, Biomedical Engineering, Ume˚a University, Ume˚a, Sweden · UniCamillus-Saint Camillus International University of Health Sciences, Rome, Italy

  55. Video-MME-Logical: A Controlled Diagnostic Benchmark for Video Temporal-Logical Reasoning

    Jun 26, 2026Hohin Kwan, Hongyu Li, Ray Zhang +5Visual EvidenceMultimodal Large Language Models

    1HKUST · 2Colab, Beihang University · 3CUHK

  56. MedBench v5: A Dynamic, Process-Oriented, and Hallucination-Aware Benchmark for Clinical Multimodal Models

    Jun 23, 2026Jinru Ding, Chuchu Jiang, Lu Lu +12Multimodal Clinical DataDiagnostic Benchmark

    1Shanghai Artificial Intelligence Laboratory

  57. EHR-Complex: Benchmarking Medical Agents for Complex Clinical Reasoning

    Jun 22, 2026Yitong Qiao, Lei Liu, Yue Shen +4Clinical Reasoning TrainingElectronic Health Records

    1Zhejiang University · †Work done during an internship at Ant Group. · 2Ant Group

  58. WeGenBench: A Multidimensional Diagnostic Benchmark towards Text-to-Image Model Optimization

    Jun 18, 2026Qian Liang, Xiaomin Li, Ying Zhang +6Modern Text-To-Image ModelsText-To-Image

    University of Electronic Science and Technology of China · Dalian University of Technology · Weixin, Tencent

  59. Benchmarking Large Vision-Language Models on Fine-Grained Image Tasks: From Evaluation to Diagnosis

    Jun 17, 2026Hong-Tao Yu, Chen-Wei Xie, Yuxin Peng +2Fine-Grained PerceptionMultimodal Benchmarks

    School of Computer Science and Engineering, Southeast University, China · Alibaba Group · Wangxuan Institute of Computer Technology, National Key Laboratory for Multimedia Information Processing, Peking University, China +2

  60. ERQA-Plus: A Diagnostic Benchmark for Reasoning in Embodied AI

    Jun 16, 2026Hong Yang, Basura FernandoEmbodied Artificial IntelligenceDiagnostic Benchmark

    Centre for Frontier AI Research Agency for Science, Technology and Research Singapore · Centre for Frontier AI Research, Agency for Science, Technology and Research, Singapore · College of Computing and Data Science, Nanyang Technological University, Singapore