Large Language Models (LLMs) have demonstrated remarkable success in diverse
natural language processing (NLP) tasks in general domains. However, LLMs
sometimes generate responses with the hallucination about medical facts due to
limited domain knowledge. Such shortcomings pose potential risks in the
utilization of LLMs within medical contexts. To address this challenge, we
propose knowledge-tuning, which leverages structured medical knowledge bases
for the LLMs to grasp domain knowledge efficiently and facilitate reliable
response generation. We also release cMedKnowQA, a Chinese medical knowledge
question-answering dataset constructed from medical knowledge bases to assess
the medical knowledge proficiency of LLMs. Experimental results show that the
LLMs which are knowledge-tuned with cMedKnowQA, can exhibit higher levels of
accuracy in response generation compared with vanilla instruction-tuning and
offer a new reliable way for the domain adaptation of LLMs.

为了解决大型语言模型（LLM）在医学领域中知识局限导致产生虚假医学事实的困境，本研究提出了知识调整的方法，利用结构化的医学知识库帮助 LLMs 有效掌握领域知识，从而实现可靠的回答生成。通过引入基于医学知识库构建的中文医学知识问答数据集 cMedKnowQA，实验结果表明，经过 cMedKnowQA 知识调整的 LLMs 在回答生成上表现出更高的准确性，为 LLMs 的领域适应提供了可靠的新途径。

使用结构化医学知识库对大型语言模型进行知识调整，以提高中文可靠性的响应生成

Knowledge-tuning Large Language Models with Structured Medical Knowledge  Bases for Reliable Response Generation in Chinese

Automatic radiology report summarization is a crucial clinical task, whose
key challenge is to maintain factual accuracy between produced summaries and
ground truth radiology findings. Existing research adopts reinforcement
learning to directly optimize factual consistency metrics such as CheXBert or
RadGraph score. However, their decoding method using greedy search or beam
search considers no factual consistency when picking the optimal candidate,
leading to limited factual consistency improvement. To address it, we propose a
novel second-stage summarizing approach FactReranker, the first attempt that
learns to choose the best summary from all candidates based on their estimated
factual consistency score. We propose to extract medical facts of the input
medical report, its gold summary, and candidate summaries based on the RadGraph
schema and design the fact-guided reranker to efficiently incorporate the
extracted medical facts for selecting the optimal summary. We decompose the
fact-guided reranker into the factual knowledge graph generation and the
factual scorer, which allows the reranker to model the mapping between the
medical facts of the input text and its gold summary, thus can select the
optimal summary even the gold summary can't be observed during inference. We
also present a fact-based ranking metric (RadMRR) for measuring the ability of
the reranker on selecting factual consistent candidates. Experimental results
on two benchmark datasets demonstrate the superiority of our method in
generating summaries with higher factual consistency scores when compared with
existing methods.

该研究旨在通过使用 FactReranker 技术，提高医学报告自动摘要的真实性，并通过对医学事实的提取和排名，实现对候选人摘要的最佳选择。