TCMScience: An Autonomous Scientist for Traditional Chinese Medicine

Yanlan Kang; Ruiqi Liu; Xinpei Gu; Tzu-Yi Lei; Shuai Xu; Xukun Zhang; William Cheng-Chung Chu

cs.LGOct 17, 2026

TCMScience: An Autonomous Scientist for Traditional Chinese Medicine

Authors: Yanlan Kang, Ruiqi Liu, Xinpei Gu, Tzu-Yi Lei, Shuai Xu, Xukun Zhang, William Cheng-Chung Chu

Organizations: Institute of Medical Philosophy & Future AI (IMPF-AI) · Shanghai Medical College, Fudan University · Guangdong Provincial Key Laboratory of New Drug Screening and Guangdong–Hong Kong–Macao Joint Laboratory for New Drug Screening, School of Pharmaceutical Sciences, Southern Medical University · Department of Computer Science, Tunghai University, Taichung, Taiwan ·

Abstract

Traditional Chinese medicine (TCM) research connects classical texts, medicinal entities, molecular assays, computational predictions, and clinical studies. An autonomous research agent must preserve the inferential boundaries between these evidence types while coordinating analysis across heterogeneous resources. We introduce TCMScience, a scientific knowledge-discovery agent that separates language-model proposals from execution and release authority. Its architecture combines a trusted kernel, a scientific compiler, and claim-support contracts: plans are compiled into typed operations, execution is bounded by explicit permissions, and candidate findings are checked against source provenance, study design, and claim scope before release. The full capability registry contains 117 public-source connectors with 444 typed operations across 29 domains and 150 native tools across 12 domains; the TCM data hub catalogs 136 databases, 76 dataset specifications, and 84,294 formula-source records. These resources support model-independent workflows through browser and local execution interfaces. In a network-pharmacology analysis of Gegen Qinlian Tang, replacing the whole-proteome background with the assayed-protein background changed the number of significant pathways from 66 to zero; incorporating inactive screening records identified nine pathways, including drug-metabolism signals retained as mechanistic hypotheses. A governance ablation over 121 constructed erroneous outputs spanning 26 classes released 121 errors without governance, 57 with the domain layer, seven with the kernel, and zero with both layers. All 10 compliant controls were retained. These results establish complementary control of known failure modes and show how explicit evidence contracts change the conclusions an agent is permitted to report.

Figures & tables

Figure 1: TCMScience framework. The governance layer constrains the trusted kernel; the capability plane is reached through governed gateways. Solid arrows show execution and dashed arrows show constraints. Counts use the full-registry convention in Table 2 . Governance evaluation refers to the constructed regression corpus.

Figure 2: Detailed execution architecture. Proposals are compiled and validated before the broker can act; evidence, claim, and output checks precede release. The governance panel separates locked skills from candidate compositions. Capability-plane counts are registry inventories; source cards identify the snapshot subsystem, and external entries retain their own readiness state. The runtime admits capabilities under the active policy and dependencies.

Claim kind	Relevant evidence	Required boundary
Textual attribution	Located classical passage or source record	Preserve wording, edition, locator, and attribution; do not infer a measured treatment effect.
Traditional use	Documented historical or practice source	Retain the traditional indication, formula identity, preparation, and source context.
Mechanistic hypothesis	Docking, target prediction, network analysis, or exploratory preclinical evidence	Label the conclusion as a hypothesis and preserve computational or experimental conditions.
Experimental mechanism	Relevant in-vitro or animal experiment	Match biological system, intervention, concentration, outcome, and causal design; do not transfer directly to patients.
Human association or effect	Appropriately designed human study	Match population, intervention, comparator, endpoint, and study design; retain uncertainty and bias assessment.
Safety finding	Explicitly measured and reported safety outcome	Name the event, observation window, and population; an unmeasured outcome remains unknown.

Table 1: Representative claim-support obligations. Permissions are conditional on source verification and matching scope; evidence design alone is not sufficient for release.

Registry object	Count	Counting unit
Public-source connectors	117	Registered source connectors in the general capability plane.
Typed operations	444 / 29 domains	Declared source operations with typed inputs and outputs.
Native analytical tools	150 / 12 domains	Registered computational tool implementations.
TCM database catalog	136	Source resources with access and provenance metadata.
TCM dataset specifications	76	Dataset acquisition or integration specifications.
Formula-source records	84,294	Source-retaining formula entries, including variants.

Table 2: Unified full-registry inventory. Counts describe registered resources and operations. Runtime admission is evaluated separately for each requested capability.

Resource	Data role	Preserved interpretation
NPASS, CMAUP, LOTUS	Natural-product occurrence, composition, and available activity records	Separate medicinal occurrence, constituent identity, and assay-derived activity.
BindingDB, PubChem BioAssay	Binding measurements or screening outcomes	Retain assay IDs, units, qualifiers, target identity, active/inactive status, and assay conditions.
UniProt reference mapping	Reviewed human protein identity	Map to unique primary accessions; retain unresolved or excluded mappings in quality-control counts.
STRING	Protein association network	Preserve association confidence; an association edge does not establish direct binding or causal direction.
Reactome	Protein-to-pathway membership	Retain pathway version, annotation provenance, and the declared enrichment background.
Open Targets	Target–disease associations	Keep genetic, literature, and other evidence channels distinct, with channel-specific thresholds.

Table 3: Source roles and preserved relation semantics in the network-pharmacology workflow.

Representative Chinese request	Executable obligation and English interpretation
\zh 区分葛根芩连汤成分的实测靶点与预测靶点。	Separate measured targets from predicted targets; preserve source records, study design, and relation type.
\zh 请确认“ \zh 姜” \zh 对应的中药实体。	Resolve “jiang” to a canonical medicinal entity; request disambiguation when fresh and dried ginger remain possible.
\zh 解释通路富集的背景集与阴性记录。	Identify the enrichment background, retain inactive assay outcomes, and state the multiple-testing procedure.
\zh 用英文总结，但保留原始名称和出处。	Translate the explanation while preserving original entity names, source locators, units, and validated claim scope.

Table 4: Bilingual requests mapped to language-independent research obligations.

Figure 3: The 16-stage governed lifecycle. Numbered stages and arrows indicate execution order. The release gate is the boundary between candidate output and a released artifact; validated state is committed only after that boundary. Failed obligations produce a recorded refusal at the relevant stage. Revisions re-enter plan validation without expanding authority.

Analysis specification	Result	Permitted interpretation
Potency hits; whole annotation background	12,155 background proteins; 66/1,684 pathways significant	Background-sensitive enrichment requiring assay-coverage control.
Potency hits; assayed-protein background	391 background proteins; 0/959 pathways significant	No pathway passes the prespecified procedure; no positive mechanism claim released.
Active/inactive screening; $\geq 20$ constituents per protein	472 proteins; 51,304 tests; 9/468 pathways significant	Screening-derived hypotheses, including drug-metabolism signals.
T2DM genetic-association channel	Overlap 9; fold 0.97; $p=0.59$	No detected enrichment under this disease-evidence definition.
T2DM literature co-mention channel	Overlap 57; fold 5.26; $p=5.2\times 10^{-26}$	Literature-defined overlap with potential shared-source and study-selection dependence.

Table 5: Case-study contrasts. Pathway counts use their stated test families. Disease-overlap rows use the whole Reactome background and Open Targets 26.09, with channel score

\geq 0.5

Configuration	Errors released	95% CI (%)	False refusals
No governance	121/121 (100.00%)	97.00–100.00	0/10
Domain layer only	57/121 (47.11%)	37.97–56.39	0/10
Kernel only	7/121 (5.79%)	2.36–11.56	0/10
Full stack	0/121 (0.00%)	0.00–3.00	0/10

Table 6: Layer-wise ablation on the same 121 mutants and ten compliant controls. Intervals use the two-sided exact binomial method. Counts are existing regression results; intervals are recomputed uniformly for this report.

Gate removed	Additional releases	Failure exposed
Output validation	42	Wording–claim drift, altered direction or dose, and unsupported citations or quotations.
Evidence-ingest verification	10	Records modified after signing.
Domain claim contract	6	Duplicate evidence, medicinal identity, exposure transfer, and an unmeasured outcome treated as absent.
Snapshot-path release check	1	A claimed direction unsupported by the cited relation path.
Claim-kind licensing	0	Its tested errors were also rejected by remaining gates.

Table 7: Leave-one-gate-out attribution relative to the full stack. All remaining gates stay enabled. Counts are additional erroneous releases among the same 121 mutants and must not be summed as disjoint error classes.

Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

Symbol	Meaning and domain
$\mathbb{I}[\cdot]$	Indicator: one when the stated condition holds, zero otherwise.
$o$ ; $A_{\rm eff}$	Requested operation; intersection of the host, project, run, and skill action sets.
$c$ , $r$ , $\pi$	Candidate claim, execution run, and governing policy.
$e$ ; $C(c)$	Evidence record; finite set of all records cited for $c$ , with supporting, contextual, or opposing roles.
$\mathcal{S}_{\pi}(c)$ ; $S$	Family of admissible candidate supporting sets; a nonempty member containing only records offered as support.
$V(e)$	Boolean verification of record identity, integrity, and source binding.

Appendix

Table A1: Notation for execution, claim support, and statistical calculations.

Channel / background	$N$	$K$	$n_{H}$	$k$	$F$	$p_{\rm HG}$
Genetic / Reactome	12,155	478	237	9	0.97	0.59
Literature / Reactome	12,155	556	237	57	5.26	$5.2\times 10^{-26}$
Genetic / assayed	391	14	237	9	1.06	0.50

Appendix

Table C1: Arithmetic reconstruction of the reported disease-overlap comparisons.

p_{\rm HG}

is an unadjusted upper-tail probability.

Configuration / event	Count	Two-sided 95% interval
No governance / erroneous release	121/121	97.00–100.00
Domain layer / erroneous release	57/121	37.97–56.39
Kernel / erroneous release	7/121	2.36–11.56
Full stack / erroneous release	0/121	0.00–3.00
Any configuration / false refusal	0/10	0.00–30.85

Appendix

Table D1: Exact interval calculations from the existing regression counts. All endpoints are percentages.

AI Agents for Scientific Discovery

AI agents for scientific discovery use language models, tools, and experiments to generate, test, and refine scientific knowledge.

See all 270 papers

TCMQA: A 38K-Question Traditional Chinese Medicine Benchmark with a Licensed-Practitioner Reference
Sep 26, 2026Tzu-Heng Huang, Jet Lin, Eric LinTraditional Chinese Medicine Medical QA
University of Wisconsin-Madison · University of California, Merced · TechTCM
Lingdan-V2: Large language models for clinically grounded reasoning in traditional Chinese medicine
Oct 10, 2026Rui Hua, Yu Wei, Ziri Jin +15Traditional Chinese Medicine Medical LLMs
School of Computer Science & Technology, Beijing Jiaotong University, Beijing, 100044, China · Tianjin Tasly Digital Chinese Medicine Technology Co., Ltd., Tianjin, 300410, China · Tasly Biopharmaceuticals Co., Ltd., Tianjin, 300410, China +8
Syndrome, Synergy, and Safety: Structured Reasoning and Knowledge-Driven Alignment for TCM Prescription Generation
Sep 22, 2026Zheng Chen, ZhiCheng Du, Haoxuan Li +1LLM Safety Alignment CoT Reasoning
Tsinghua University · Guangdong Provincial Laboratory of Traditional Chinese Medicine Hengqin

Sep 26, 2026J/K move · Enter open · S save

TCMQA: A 38K-Question Traditional Chinese Medicine Benchmark with a Licensed-Practitioner Reference

Tzu-Heng Huang, Jet Lin, Eric Lin

University of Wisconsin-Madison · University of California, Merced · TechTCM

Traditional Chinese Medicine Medical QA Medical LLMs LLM Evaluation

Open paper AskPDF arXiv

Medical benchmarks for language models are built almost entirely on Western biomedicine. Traditional Chinese Medicine (TCM) is a separate system, with its own diagnostic framework and its own literature, and it remains largely unmeasured. The few TCM evaluations that exist are small, narrow, and rarely paired with a human reference. We present TCMQA, an open benchmark of 38,279 questions from Chinese TCM licensing examinations, paired with 15,151 responses from 101 licensed practitioners. We evaluate 29 instruction-tuned models from 9 families, spanning 0.27B to 14.8B parameters. Accuracy ranges over 59 points, and no model approaches saturation. Pretraining data predicts TCM ability far better than scale: a 12B Western-pretrained model reaches 39.6%, while a Chinese-pretrained model an eighth its size reaches 60.8%. Nine models exceed the practitioner majority vote of 64.9%, the best by 21.8 points, and all nine come from that same Chinese-pretrained family. Yet difficulty does not transfer between models and practitioners: accuracy is flat across practitioner-rated difficulty, item-level agreement is near zero for all 29 models, and on

8.4\%

of items the practitioners are correct where the leading model is wrong. We release the corpus, the practitioner responses, the harness, and per-item model outputs at https://huggingface.co/datasets/TechTCM/TCMQA.

Abstract

AI Agents for Scientific Discovery

Explore similar work

TCMQA: A 38K-Question Traditional Chinese Medicine Benchmark with a Licensed-Practitioner Reference

Lingdan-V2: Large language models for clinically grounded reasoning in traditional Chinese medicine

Syndrome, Synergy, and Safety: Structured Reasoning and Knowledge-Driven Alignment for TCM Prescription Generation