Dental-TriageBench: Benchmarking Multimodal Reasoning for Hierarchical Dental Triage

Ziyi He; Yushi Feng; Shuangyu Yang; Yinghao Zhu; Xichen Zhang; Pak Chuen Patrick Tai; Hei Yuet Lo; Songying Wu

cs.CLMar 18, 2026

Dental-TriageBench: Benchmarking Multimodal Reasoning for Hierarchical Dental Triage

Authors: Ziyi He, Yushi Feng, Shuangyu Yang, Yinghao Zhu, Xichen Zhang, Pak Chuen Patrick Tai, Hei Yuet Lo, Songying Wu, +2 more

Organizations: School of Computing and Data Science, The University of Hong Kong · Faculty of Dentistry, The University of Hong Kong · The Prince Philip Dental Hospital · The Hong Kong University of Science and Technology · Li Ka Shing Faculty of Medicine, The University of Hong Kong

Abstract

Dental triage is a safety-critical clinical routing task that requires integrating multimodal clinical information (e.g., patient complaints and radiographic evidence) to determine complete referral plans. We present Dental-TriageBench, the first expert-annotated benchmark for reasoning-driven multimodal dental triage. Built from authentic outpatient workflows, it contains 246 de-identified cases annotated with expert-authored golden reasoning trajectories, together with hierarchical triage labels. We benchmark 19 proprietary, open-source, and medical-domain MLLMs against three junior dentists serving as the human baseline, and find a substantial human--model gap, on fine-grained treatment-level triage. Further analyses show that accurate triage requires both complaint and OPG information, and that model errors concentrate on cases with multiple referral domains, where MLLMs tend to produce overly narrow referral sets and omission-heavy errors. Dental-TriageBench provides a realistic testbed for developing multimodal clinical AI systems that are more clinically grounded, coverage-aware, and safer for downstream care.

Figures & tables

Figure 1: Multimodal dental triage. Given a patient’s multimodal information (e.g., chief complaint, orthopantomogram), the model predicts a multi-label specialty referral plan that reflects the patient’s treatment needs.

Figure 2: Overview of Dental-TriageBench . Steps 1–4 illustrate the dataset construction pipeline, and Step 5 summarizes the main experimental settings and key findings.

Figure 3: Data statistics of Dental-TriageBench .

Proprietary MLLMs
Model	Fine-grained (22 labels)				Coarse-grained (8 domains)
Model	Macro-F1	Macro-Rec	Micro-F1	Exact-Match	Macro-F1	Macro-Rec	Micro-F1	Exact-Match
Claude-Sonnet-4-5-20250929	0.269	0.275	0.350	0.024	0.442	0.466	0.565	0.130
Gemini-3-Flash	0.302	0.310	0.459	0.049	0.488	0.492	0.656	0.220
Gemini-3-Pro	0.294	0.262	0.447	0.118	0.493	0.444	0.641	0.224
GPT-5.2	0.272	0.239	0.404	0.041	0.398	0.320	0.576	0.215

Table 1: Benchmark results on Dental-TriageBench across 22 fine-grained triage labels and 8 aggregated coarse-grained domains. We report Macro-F1, Macro-Recall, Micro-F1, and Exact-Match for 19 proprietary, open-source, and medical MLLMs, together with the human baseline. Within each model domain, the best results are in bold and the second-best results are underlined .

Figure 4: CC-only: only chief complaint; OPG-only: only OPG; CC+OPG: both chief complaint and OPG

Figure 5: Distribution of five clinically grounded failure dimensions across six representative MLLMs, estimated by Gemini-3-Pro over all benchmark cases.

Model	$\Delta$ F1 $>0$	$\Delta$ F1 $>0.1$	$\Delta$ F1 $>0.2$
Claude Sonnet 4.5	17.1%	13.0%	11.0%
Gemini 3 Flash	26.0%	17.5%	10.2%
GPT-5.2	35.4%	34.1%	27.6%

Table 2: Percentage of cases with positive case-level F1 gain after adding the missing modality.

Figure 6: A failure taxonomy case study on MedGemma-27B, with correct triage label on green , and missing triage label on red .

Figure 7: Comparison of case-level Micro-F1 between models and human baseline among different case complexity levels.

Figure 8: Distribution of the number of positive triage domains per case. The arrow marks the threshold for cases with at least two positive domains.

Figure 9: Trade-off between omission risk and over-referral burden. The lower-left region is preferred, indicating fewer missed necessary referrals and fewer unnecessary referrals. Bubble size denotes fine-grained Macro-F1 reported in Table 1 .

Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 10: Confusion matrices for human–judge agreement on five failure dimensions over 60 sampled model outputs.

Proprietary MLLMs
Models	Perio	Cariology	Endo	Crowns	Bridges	Denture	Implants	OMFS
Claude-Sonnet-4-5-20250929	0.598	0.340	0.446	0.368	0.130	0.571	0.259	0.821
Gemini-3-Flash	0.652	0.443	0.337	0.375	0.222	0.723	0.237	0.913
Gemini-3-Pro	0.735	0.368	0.444	0.413	0.235	0.610	0.333	0.807
GPT-5.2	0.494	0.378	0.338	0.255	0.069	0.571	0.226	0.850
Grok-4	0.561	0.452	0.354	0.250	0.125	0.577	0.120	0.695

Appendix

Table 3: Class-wise F1 scores on the 8 coarse-grained domains. Within each model domain, the best results are in bold and the second-best results are underlined .

Rater	Fine-grained (22 labels)				Coarse-grained (8 specialties)
Rater	Macro-F1	Macro-Rec	Micro-F1	Exact-Match	Macro-F1	Macro-Rec	Micro-F1	Exact-Match
Junior Dentist 1	0.394	0.440	0.511	0.049	0.567	0.647	0.696	0.203
Junior Dentist 2	0.409	0.439	0.520	0.077	0.547	0.604	0.730	0.260
Junior Dentist 3	0.402	0.413	0.546	0.122	0.571	0.586	0.754	0.354
Average	0.402	0.431	0.525	0.083	0.562	0.613	0.727	0.272

Appendix

Table 4: Individual performance of the three junior dentists on Dental-TriageBench . We report the same evaluation metrics as in the main paper.

Figure 11: Hierarchical taxonomy of the dental triage label space in our benchmark. The taxonomy is organized into two levels: 8 coarse-grained triage domains (e.g., Periodontology, Endodontics, and OMFS) and 22 fine-grained triage categories, which specify treatment-oriented referral categories within each coarse-grained triage domain.

Figure 12: A representative modality-ablation case on Gemini-3-Flash. The chief complaint alone supports Gingivitis and Exodontia , but misses the surgical complexity required for Minor O.S. . With access to the OPG, the model correctly recovers the full OMFS referral plan.

Figure 13: Additional failure case study of Gemini-3-Pro. The model misinterprets the status of tooth 36 and overestimates the severity of tooth 27 on the OPG, leading to incorrect positive predictions for Molar Endodontics and Crowns . Although it correctly identifies the impaction of tooth 48, it fails to map this finding to the appropriate OMFS referral for Exodontia . This example illustrates how OPG misinterpretation can simultaneously produce both over-triage and under-triage errors within the same case.

Explore similar work

GlobalDentBench: A Multinational Benchmark for Evaluating LLM Clinical Reasoning in Dentistry with Expert Calibration
May 23, 2026Junjie Zhao, Jingyi Liang, Zhenyang Cai +22Language Model Safety Evaluation Clinical Reasoning
Division of Applied Oral Sciences and Community Dental Care, Faculty of Dentistry, The University of Hong Kong. · Beijing Institute of Collaborative Innovation. · School of Data Science, The Chinese University of Hong Kong, Shenzhen. +13
OralMLLM-Bench: Evaluating Cognitive Capabilities of Multimodal Large Language Models in Dental Practice
May 2, 2026Rongyang Wang, Shuang Zhou, Jiashuo Wang +2Medical Imaging Cognitive Modeling
Department of Orthodontics, Beijing Stomatological Hospital, School of Stomatology, Capital Medical University, Fanjiacun Road #9, Fengtai District, Beijing 100070, China · Division of Computational Health Sciences, Department of Surgery, University of Minnesota, Minneapolis, MN, USA · Department of Computing, The Hong Kong Polytechnic University, Hong Kong SAR, China +1
Evaluating Multi-Turn Multimodal Diagnostic Reasoning on Challenging Real-World Clinical Cases
Jul 28, 2026Rui Yang, Weihao Xuan, Yi Lin +23Medical VLMs Multi-Turn Dialogue Evaluation
Center for Biomedical Data Science, Duke-NUS Medical School, Singapore, Singapore · Duke-NUS AI + Medical Sciences Initiative, Duke-NUS Medical School, Singapore, Singapore · Department of Population Health Sciences, Weill Cornell Medicine, New York, NY, USA +24

May 23, 2026J/K move · Enter open · S save

GlobalDentBench: A Multinational Benchmark for Evaluating LLM Clinical Reasoning in Dentistry with Expert Calibration

Junjie Zhao, Jingyi Liang, Zhenyang Cai, Jiaming Zhang+21

Division of Applied Oral Sciences and Community Dental Care, Faculty of Dentistry, The University of Hong Kong. · Beijing Institute of Collaborative Innovation. · School of Data Science, The Chinese University of Hong Kong, Shenzhen.+13

Language Model Safety Evaluation Clinical Reasoning Clinical Language Model Evaluation

Open paper AskPDF arXiv

While large language models (LLMs) hold transformative potential for medicine, their reasoning robustness and safety in real-world clinical scenarios remain critically underexplored, particularly in dentistry. Here we introduce GlobalDentBench, the first multinational dental benchmark, featuring a taxonomy that encompasses 14 dental specialties across 88 countries and regions spanning six continents. The benchmark comprises 8,978 expert-validated questions across three formats (multiple-choice, short-answer, and case-based questions) and assesses three progressive reasoning levels: knowledge recall (L1), routine reasoning (L2), and individualized reasoning (L3). To ensure data quality, the automated construction framework was calibrated by six senior dentists, achieving expert agreement rates of 99.98% for multiple-choice and short-answer questions and 96.78% for the more complex case-based questions. Evaluation of 12 frontier LLMs on GlobalDentBench revealed a sharp, stepwise performance degradation with increasing reasoning complexity. Specifically, accuracy plummeted from 81.34% on multiple-choice to 64.53% on short-answer and 22.34% on case-based questions, while declining markedly from 74.01% at L1 to 55.64% at L2 and 35.71% at L3. More critically, risk analysis of real-world dental cases demonstrated an alarming overall unsafe rate of 31.01% in LLM-generated clinical recommendations, with 4.51% posing risks of irreversible patient harm and risks particularly pronounced in specialties such as orthodontics. These findings expose fundamental limitations in the medical reasoning and safety of current LLMs. Consequently, GlobalDentBench provides a scalable foundation for trustworthy clinical AI evaluation, underscoring the urgent need for rigorous validation before the safe deployment of these models in healthcare.

Abstract

Explore similar work

GlobalDentBench: A Multinational Benchmark for Evaluating LLM Clinical Reasoning in Dentistry with Expert Calibration

OralMLLM-Bench: Evaluating Cognitive Capabilities of Multimodal Large Language Models in Dental Practice

Evaluating Multi-Turn Multimodal Diagnostic Reasoning on Challenging Real-World Clinical Cases