Organizations: School of Computer Science & Technology, Beijing Jiaotong University, Beijing, 100044, China · Tianjin Tasly Digital Chinese Medicine Technology Co., Ltd., Tianjin, 300410, China · Tasly Biopharmaceuticals Co., Ltd., Tianjin, 300410, China · Institute of Liver Diseases, Hubei Key Laboratory of the Theory and Application Research of Liver and Kidney in Traditional Chinese Medicine, Hubei Provincial Hospital of Traditional Chinese Medicine, Wuhan, 430061, China · Affiliated Hospital of Hubei University of Chinese Medicine, Wuhan, 430061, China · Da’an Health Technology (Beijing) Co., Ltd., Beijing, 100020, China · Institute of Chinese Medicine Epidemic Disease, Beijing University of Chinese Medicine, Beijing, 102488, China · Department of Gastroenterology, Guang’anmen Hospital, China Academy of Chinese Medical Sciences, Beijing, 100053, China · Cardiovascular Diseases Center, National Clinical Research Center for Chinese Medicine Cardiology, Xiyuan Hospital, China Academy of Chinese Medical Sciences, Beijing, 100091, China · The First Affiliated Hospital, Henan University of Chinese Medicine, Zhengzhou, 450000, China · China Academy of Chinese Medical Sciences, Beijing, 100700, China
Large language models have advanced medical reasoning, but clinically grounded reasoning in traditional Chinese medicine (TCM) remains challenging because prescription decisions require integrating domain knowledge, syndrome differentiation, treatment principles, herbal prescription composition, and dosage determination. We develop a TCM language model family comprising models with 4 billion, 8 billion, and 14 billion parameters. The artificial intelligence contribution is a staged clinical alignment framework integrating large scale continued pretraining on TCM and clinical records, complementary reasoning supervision, and prescription-centered reinforcement learning (RL). We further formulate herbal prescription generation as a structured RL problem with prescription-level feedback, enabling direct optimization of clinical decision outputs beyond knowledge reproduction and generic instruction following. The engineering application is TCM clinical decision support, particularly herbal prescription recommendation. We evaluate the models on a public benchmark containing 25,620 expert-verified items across 13 TCM knowledge and clinical reasoning tasks and on a private retrospective dataset of 4,348 spleen and stomach disease prescription cases. The 14 billion parameter reinforcement aligned model achieves a prescription generation F1 of 37.0 on the public benchmark, compared with 32.3 for DeepSeek-R1 with 671 billion total parameters, and a herb-level F1 of 52.8 on the clinical dataset with improved dosage agreement. Reinforcement post-training consistently improves prescription recommendation across all scales, while physician-source analysis shows that expert heterogeneity affects reward stability. These findings demonstrate a scalable pathway from domain grounding and reasoning supervision to clinically oriented prescription decision support.
Figures & tables
Figure 1: Overview of the Lingdan-V2 development and evaluation workflow. Qwen3 backbones at 4B, 8B and 14B scales were trained using the same CPT, SFT and GRPO pipeline. Evaluation was performed using LingLan, LingLan-Hard and a private clinical prescription recommendation test set. MAE, mean absolute error.
Data category
Tokens (B)
Share (%)
Domain
Longitudinal patient care trajectories corpus
10.42
62.72
TCM/EMR
Clinical diagnosis and physician-assistance data
0.50
2.98
TCM/EMR
Entity extraction data
0.44
2.66
TCM
Lingdan V1 data
0.32
1.90
TCM
Classical texts with translations
0.29
1.75
TCM
Knowledge graph and patent medicine question answering
0.18
1.11
TCM
Table 1: Continued pretraining corpus used for Lingdan-V2 model development after final consolidation. Token counts are reported in billions.
Training component
Samples
Tokens (M)
Direct-answer instruction data
6,879,359
1,238.49
Reasoning-format SFT data
200,529
216.24
Instruction distillation with reasoning traces
46,221
70.33
Instruction data without reasoning traces
75,741
26.98
TCM multi-turn dialogue data
18,468
8.45
Physician-synthesized cases
262,987
155.18
Table 2: Final composition of the consolidated SFT corpus. Categories are mutually exclusive at the final training-file level; token counts are rounded to two decimals.
Figure 2: Clinical data processing and task construction. Heterogeneous outpatient and inpatient records are mapped to a common schema, de-identified, and standardized for medical orders, medications and diagnoses. Longitudinal patient trajectories support CPT, whereas task conversion produces SFT data for clinical decision support, diagnosis completion and clinical-note summarization. PII, personally identifiable information.
Figure 3: Overall Score comparison on LingLan. Panel A shows Overall Scores on the full benchmark and LingLan-Hard subset for Lingdan-14B-R1 and external models, together with the performance drop on the harder subset. Panel B illustrates Overall Score as a function of model scale, showing the relative position of Lingdan-14B-R1 among models with different parameter sizes. Performance drops are expressed in percentage points (pp).
Figure 4: Prescription recommendation on LingLan. Wide light-green bars show prescription-recommendation F1 on the full benchmark, and narrow dark-green bars show F1 on LingLan-Hard.
Dom.
Subtask
Metric
DeepSeek-R1 [-1pt] 671B
DeepSeek-V3.1 [-1pt] 671B
GPT-5 [-1pt] Undisclosed
Qwen3 [-1pt] 235B
Qwen3 [-1pt] 32B
Baichuan-M2 [-1pt] 32B
Qwen3-Next [-1pt] 80B-A3B
GPT-OSS [-1pt] 120B
Lingdan-14B [-1pt] R1
TLE
Comprehensive
Accuracy
95.0 / 78.3
95.5 / 81.0
87.6 / 52.3
95.0 / 77.8
91.2 / 62.3
89.8 / 58.3
93.8 / 73.0
58.1 / 14.5
91.7 / 67.0
FTK
Single-choice
Accuracy
86.1 / 46.5
86.1 / 45.3
85.4 / 44.5
85.0 / 45.3
86.1 / 44.3
83.4 / 35.8
82.1 / 33.3
70.4 / 28.2
88.2 / 54.8
FTK
Multiple-choice
Accuracy
64.8 / 31.5
62.4 / 28.0
53.9 / 19.3
61.3 / 25.0
56.3 / 23.3
48.6 / 14.0
52.6 / 14.8
35.3 / 9.8
62.2 / 30.3
FTK
Multiple-choice
Precision
92.5 / 82.4
91.9 / 80.5
90.6 / 75.3
93.0 / 80.9
90.3 / 77.5
90.5 / 75.4
89.4 / 71.6
81.2 / 63.5
89.7 / 77.9
FTK
Multiple-choice
Recall
92.1 / 79.3
92.0 / 80.5
87.2 / 69.8
89.1 / 73.7
90.9 / 78.0
84.5 / 66.1
84.7 / 65.2
83.2 / 65.6
94.9 / 86.1
FTK
Multiple-choice
F1
91.0 / 77.8
90.6 / 77.5
87.3 / 69.3
89.5 / 73.7
89.1 / 74.8
85.3 / 66.4
84.7 / 64.2
80.0 / 61.3
91.1 / 79.4
Table 3: Complete LingLan task-level results for large and strong external baseline models and the final Lingdan-14B-R1 model. Cells report full-set / Hard-subset scores; bold marks the best value within this table and split, with lower values better for MAE. The final row reports Overall Score, as defined in Methods. Abbreviations: TLE, TCM Licensing Examination; FTK, Fundamental TCM Knowledge; CPMK, Chinese Patent Medicine Knowledge; IE, Information Extraction; DTR, Diagnostic and Therapeutic Reasoning; DTR-F1, DTR with synonym-tolerant character-level matching; DR, Decision Recognition.
Dom.
Subtask
Metric
Lingdan-V1 [-1pt] 13B
TCMChat [-1pt] 7B
Baichuan2 [-1pt] 13B
BianCang [-1pt] 7B
BianCang [-1pt] 14B
ShizhenGPT [-1pt] 7B
ShizhenGPT [-1pt] 32B
Lingdan-R1 [-1pt] 14B
TLE
Comprehensive
Accuracy
44.3 / 20.8
31.6 / 19.5
50.1 / 21.8
88.9 / 70.0
91.2 / 73.5
83.6 / 54.5
90.7 / 67.0
91.7 / 67.0
FTK
Single-choice
Accuracy
48.4 / 26.3
11.1 / 7.5
60.6 / 28.5
78.2 / 43.3
82.6 / 49.3
79.9 / 41.3
83.5 / 47.8
88.2 / 54.8
FTK
Multiple-choice
F1
52.4 / 49.0
39.9 / 32.4
75.9 / 67.7
76.2 / 68.0
55.8 / 50.3
81.6 / 66.9
86.7 / 72.7
91.1 / 79.4
FTK
Cloze
F1
8.5 / 4.5
13.9 / 6.5
33.2 / 13.3
35.4 / 15.0
38.7 / 14.6
40.6 / 17.7
49.8 / 20.9
60.2 / 23.9
CPMK
Single-choice
Accuracy
28.3 / 16.3
19.2 / 18.5
41.6 / 24.8
58.4 / 31.8
62.5 / 35.5
56.5 / 20.3
68.7 / 27.3
78.1 / 38.3
CPMK
Multiple-choice
F1
52.3 / 51.2
33.8 / 29.7
75.2 / 65.4
77.1 / 68.3
36.3 / 27.8
78.0 / 63.1
82.7 / 67.8
87.7 / 76.6
Table 4: Primary LingLan results for TCM-oriented language models and related Chinese-language baselines. Cells report full-set / Hard-subset scores. The table includes the 16 metrics used to calculate Overall Score; bold marks the best value within each split.
Figure 5: Performance changes from the SFT to R1 checkpoints across Lingdan-V2 model scales. Each cell compares the corresponding SFT and R1 checkpoints for one evaluation metric and benchmark split; endpoint labels show the two scores, and Δ denotes R1 minus SFT.
Model
F1
Precision
Recall
Dose cosine
Dose MAE
Lingdan-14B-R1
52.8
51.8
55.4
50.6
4.369
Lingdan-8B-R1
50.7
55.0
48.3
49.5
6.006
Lingdan-4B-R1
52.7
52.5
54.1
49.5
5.062
Lingdan-14B-SFT
35.1
52.0
27.1
34.8
5.840
Lingdan-8B-SFT
34.2
52.7
26.0
34.2
5.812
Lingdan-4B-SFT
33.7
51.6
25.7
34.1
5.787
Table 5: Prescription recommendation performance on the private spleen-stomach disease test set. Herb-level metrics and dose cosine similarity are reported on a 0–100 scale; lower dose MAE indicates better dosage estimation.
Figure 6: GRPO reward trajectories under different physician-source configurations. The curves compare prescription training pools containing cases from 1, 5, 10, and 20 physicians over the first 3,000 global optimization steps. Faint curves show raw rewards, and solid curves show the corresponding moving averages.
Appendix figures & tables5 assets
Supplementary material from the paper’s appendix.
Appendix
Dataset
Category
TCMChat Dai et al. (2024)
TCM dialogue
Huatuo-26M Li et al. (2023)
Medical instruction data
CMB Wang et al. (2024)
Chinese medical knowledge benchmark
CMExam Liu et al. (2023)
TCM examination dataset
MTCMB Kong et al. (2025)
TCM multi-task dataset
cMedQA2 Zhang et al. (2018)
Medical question answering
Appendix
Table A.1: Public datasets used for Lingdan-V2 training.
Dom.
Subtask
Metric
Qwen3 [-1pt] 32B
Qwen3 [-1pt] 30B-A3B
Qwen3 [-1pt] 14B
Qwen3 [-1pt] 8B
Qwen3 [-1pt] 4B
GPT-5-mini [-1pt] Undisclosed
GPT-OSS [-1pt] 20B
Lingdan-4B [-1pt] R1
Lingdan-8B [-1pt] R1
Lingdan-14B [-1pt] R1
TLE
Comprehensive
Accuracy
91.2 / 62.3
88.8 / 55.3
87.6 / 50.0
84.6 / 40.5
76.6 / 30.3
76.3 / 24.5
48.3 / 13.5
81.0 / 40.8
86.2 / 50.2
91.7 / 67.0
FTK
Single-choice
Accuracy
86.1 / 44.3
82.6 / 32.0
83.0 / 34.5
80.8 / 33.8
76.2 / 28.5
77.0 / 28.7
65.4 / 29.3
81.5 / 40.3
83.9 / 46.0
88.2 / 54.8
FTK
Multiple-choice
Accuracy
56.3 / 23.3
52.0 / 17.0
49.5 / 13.8
47.8 / 13.0
42.4 / 12.3
46.2 / 10.8
24.8 / 6.0
51.6 / 23.0
55.6 / 24.5
62.2 / 30.3
FTK
Multiple-choice
Precision
90.3 / 77.5
89.8 / 75.2
90.6 / 75.2
88.4 / 71.8
86.1 / 69.0
86.1 / 67.7
76.1 / 59.5
86.2 / 73.2
90.1 / 77.5
89.7 / 77.9
FTK
Multiple-choice
Recall
90.9 / 78.0
88.3 / 73.5
85.7 / 69.4
86.8 / 70.2
84.9 / 67.6
86.8 / 69.0
74.9 / 58.1
93.6 / 85.3
89.5 / 78.8
94.9 / 86.1
FTK
Multiple-choice
F1
89.1 / 74.8
87.4 / 71.0
86.3 / 68.7
85.8 / 67.6
83.5 / 64.8
84.8 / 65.3
72.9 / 55.0
88.4 / 76.3
88.2 / 74.9
91.1 / 79.4
Appendix
Table C.1: Complete LingLan task-level results for smaller external models and the Lingdan-V2 R1 family at 4B, 8B and 14B. Qwen3-32B is repeated as a scale-boundary anchor. Cells report full-set / Hard-subset scores; bold marks the best value within this table and split, with lower values better for MAE. The final row reports Overall Score, as defined in Methods.
Figure D.1: Information extraction on LingLan. Scores are the equal-weight mean of clinical-record and classical-text extraction F1. Wide light-blue bars show full-benchmark scores, and narrow dark-blue bars show LingLan-Hard scores.
Figure E.1: Prescription recommendation on the private spleen-stomach disease test set. From left to right, the plots show herb-level F1, dose cosine similarity, and dose MAE. Dark-blue bars represent Lingdan-R1 models, light-blue hatched bars represent Lingdan-SFT models, and light-grey bars represent Qwen3 reference models. Higher F1 and cosine similarity indicate better agreement; lower MAE indicates lower absolute dosage error.
Figure F.1: GRPO prescription reward trajectories for four Lingdan-4B development runs using the 10-physician source pool under different hyperparameter settings. Faint curves show raw rewards, and solid curves show moving averages calculated over a fixed 100-step window.
Large language models (LLMs) can generate clinical narratives that are insufficiently grounded in patient-specific evidence. In traditional Chinese medicine (TCM), errors can propagate from etiology and pathogenesis through syndrome diagnosis and treatment principles to prescription generation. We developed TCMClinicalReason-Bench using 2,000 multicenter electronic health record cases to distinguish case-grounded responses from fluent but unsupported diagnostic and therapeutic conclusions. Five general-purpose and two TCM-specific LLMs were evaluated in zero-shot settings. An evidence-constrained rubric assessed seven diagnostic and therapeutic components and three cross-block relations, allowing case-supported alternatives. Qwen3.7-Plus with TCM retrieval served as the automated judge, alongside parallel blinded ratings by five senior TCM clinicians on a 600-case subset. Structural completeness was nearly saturated (99.3-100.0%), but normalized content scores ranged from 40.7% to 54.1%. The five general-purpose models averaged 50.0%, versus 41.3% for the two smaller TCM-specific models. Cross-block logic consistency ranged from 60.8% to 66.8% and correlated moderately with content across cases (Pearson's r = 0.515-0.656). Deficits were greatest in prescription generation, prescription analysis, and symptom-guided modification. In judge stress testing on 100 independent cases, perturbation detection rates across the three relations were 57%, 56%, and 31%, with contradictions detected more reliably than omissions. Separating component quality from cross-block consistency localizes failures missed by endpoint and completeness metrics and identifies where clinician oversight remains necessary.
Jirui Dai, Chenkai Zhang, Yan Jia +4
the Department of Computer Science, Johns Hopkins University, Baltimore, USA · the School of Pharmacy, Nanjing University of Chinese Medicine, Nanjing, China · the School of Information Engineering, Huzhou University, Huzhou, China +3
Applying large language models to Traditional Chinese Medicine (TCM) prescription generation reveals three clinically critical gaps: models produce end-to-end mappings without auditable reasoning following the li-fa-fang-yao paradigm (SR Gap), treat each encounter in isolation without follow-up adjustment via sui zheng jia jian (LA Gap), and fail to enforce absolute contraindication rules such as Shi Ba Fan (SC Gap). We propose a progressive four-stage framework (SFT → PG-CoT → Dynamic → K-RL) that addresses each gap: PG-CoT constrains CoT distillation under the li-fa-fang-yao paradigm to produce auditable diagnostic chains, Dynamic SFT models patient trajectories with explicit transition reasoning, and K-RL encodes deterministic pharmacological rules as rule-based DPO preference signals. Across 12 fine-tuned models and 6 zero-shot baselines, our framework substantially improves prescription quality over zero-shot baselines---with a 7B model (Mistral-7B) surpassing zero-shot GPT-5 on all three TCM evaluation metrics.
Zheng Chen, ZhiCheng Du, Haoxuan Li +1
Tsinghua University · Guangdong Provincial Laboratory of Traditional Chinese Medicine Hengqin
Reliable evaluation of large language models in traditional Chinese medicine remains limited because existing benchmarks mainly focus on examination-style question answering or isolated generative tasks. We developed LingLanMedBench, a unified benchmark comprising 25,620 expert-verified items across 13 subtasks and five domains, covering knowledge, language understanding, information extraction, and diagnostic-therapeutic decision-making. LingLanMedBench standardizes task formats and evaluation metrics and further incorporates synonym-tolerant clinical-label matching, empirically derived hard subsets, and decision-recognition reformulations of open-ended clinical tasks. Fourteen open-source and proprietary models were evaluated under a standardized zero-shot protocol. Performance varied substantially across domains, model families, task formulations, and difficulty levels. The leading overall score decreased from 51.1 on the full benchmark to 31.9 on the hard subsets. Prescription performance also differed markedly between open-ended generation, with F1 scores of 32.7 and 20.3, and decision recognition, with accuracies of 90.6 and 56.8 on the full and hard subsets, respectively. These findings reveal a substantial gap between knowledge-oriented performance and clinically faithful reasoning. LingLanMedBench provides a standardized and reproducible framework for characterizing current capabilities, identifying systematic limitations, and tracking meaningful progress in large language models for traditional Chinese medicine.
Rui Hua, Yu Wei, Zixin Shu +17
Department of Computer Science and Technology, Beijing Jiaotong University, Beijing, China · Tianjin Tasly Digital Chinese Medicine Technology Co., Ltd., Tianjin, China · Tasly Biopharmaceuticals Co., Ltd., Tianjin, China +11