The recently unprecedented advancements in Large Language Models (LLMs) have
propelled the medical community by establishing advanced medical-domain models.
However, due to the limited collection of medical datasets, there are only a
few comprehensive benchmarks available to gauge progress in this area. In this
paper, we introduce a new medical question-answering (QA) dataset that contains
massive manual instruction for solving Traditional Chinese Medicine examination
tasks, called TCMD. Specifically, our TCMD collects massive questions across
diverse domains with their annotated medical subjects and thus supports us in
comprehensively assessing the capability of LLMs in the TCM domain. Extensive
evaluation of various general LLMs and medical-domain-specific LLMs is
conducted. Moreover, we also analyze the robustness of current LLMs in solving
TCM QA tasks by introducing randomness. The inconsistency of the experimental
results also reveals the shortcomings of current LLMs in solving QA tasks. We
also expect that our dataset can further facilitate the development of LLMs in
the TCM area.

通过引入新的医学问答数据集 TCMD，本文评估了不同领域通用语言模型和医学领域特定语言模型在中医领域的能力，并分析了它们在中医问答任务中的鲁棒性及其存在的不足。希望该数据集进一步促进中医领域通用语言模型的发展。