cs.CVNov 17, 2026

OmniTongue: A Comprehensive MLLM Benchmark for Automated Tongue Diagnosis in Traditional Chinese Medicine

Authors: Liuyi Xu, Yun Guo, Shusheng Li, Xuewen Zhang, Yining Qian, An-Yang Lu

Organizations: College of Information Science and Engineering, Northeastern University, Shenyang 110819, China · College of Computer Science and Engineering, Northeastern University, Shenyang 110189, China

Abstract

Multimodal Large Language Models (MLLMs) have excelled in Western medical imaging, yet their potential in Traditional Chinese Medicine (TCM) tongue diagnosis remains systematically unevaluated. Tongue diagnosis requires sophisticated global and local perception, serving as a rigorous touchstone for assessing the visual understanding depth of MLLMs. However, existing datasets are limited by single dimensions, coarse granularity, and ambiguous labels, hindering the evaluation of authentic clinical capabilities. To bridge this gap, we introduce OmniTongue, the first comprehensive, fine-grained Visual Question Answering benchmark for TCM tongue diagnosis. This benchmark integrates 1,351 expert-validated images and 21,616 instruction pairs. Its core design features 16 clinical dimensions for multidimensional evaluation, region-specific annotations for pathological localization, and 66 fine-grained labels for precise diagnostic representation. Extensive evaluations of existing MLLMs reveal significant performance gaps, demonstrating that medical-specific models lack domain advantages, while universally poor Macro-F1 scores expose a reliance on statistical shortcuts rather than genuine understanding of tongue pathology. These findings not only highlight the current limitations of MLLMs in interpreting regional medical features but also underscore OmniTongue's significance for developing robust automated tongue diagnosis models. The code and dataset will be made publicly available in a future release.

Figures & tables

Explore similar work

CardsList
  1. MMIR-TCM: Memory-Integrated Multimodal Inference and Retrieval for TCM Clinical Decision Support

    Jul 2, 2026Lihui Luo, Joongwon Chae, Ziyan Chen +13Multimodal RAGMedical Diagnosis

  2. What Makes Deep Learning Work for Traditional Chinese Medicine Tongue Diagnosis? A Comprehensive Ablation Study

    Jul 30, 2026Longxia Gao, Linan Wang, Yuhe Han +3Medical DiagnosisMedical Image Classification

  3. LingLan: An Advancing Traditional Chinese Medicine Diagnosis LLM with Multimodal Data

    Sep 22, 2026Zheng Chen, Zhicheng Du, Haoxuan Li +2Medical DiagnosisTraditional Chinese Medicine