Language Models (LMs) have performed well on biomedical natural language
processing applications. In this study, we conducted some experiments to use
prompt methods to extract knowledge from LMs as new knowledge Bases (LMs as
KBs). However, prompting can only be used as a low bound for knowledge
extraction, and perform particularly poorly on biomedical domain KBs. In order
to make LMs as KBs more in line with the actual application scenarios of the
biomedical domain, we specifically add EHR notes as context to the prompt to
improve the low bound in the biomedical domain. We design and validate a series
of experiments for our Dynamic-Context-BioLAMA task. Our experiments show that
the knowledge possessed by those language models can distinguish the correct
knowledge from the noise knowledge in the EHR notes, and such distinguishing
ability can also be used as a new metric to evaluate the amount of knowledge
possessed by the model.

通过在 EHR 笔记上下文中添加提示，我们设计了并验证了一系列实验，用于实现动态上下文下的生物医学语言模型内置知识库的任务。我们的实验表明，这些语言模型所具有的知识可以从 EHR 笔记的噪声知识中区分出正确的知识，这种区分能力也可以作为一个新的度量来评估模型所具有的知识量。