Traditional representations like Bag of words are high dimensional, sparse
and ignore the order as well as syntactic and semantic information. Distributed
vector representations or embeddings map variable length text to dense fixed
length vectors as well as capture the prior knowledge which can transferred to
downstream tasks. Even though embedding has become de facto standard for
representations in deep learning based NLP tasks in both general and clinical
domains, there is no survey paper which presents a detailed review of
embeddings in Clinical Natural Language Processing. In this survey paper, we
discuss various medical corpora and their characteristics, medical codes and
present a brief overview as well as comparison of popular embeddings models. We
classify clinical embeddings into nine types and discuss each embedding type in
detail. We discuss various evaluation methods followed by possible solutions to
various challenges in clinical embeddings. Finally, we conclude with some of
the future directions which will advance the research in clinical embeddings.

本文是一篇关于临床自然语言处理中嵌入表示法的全面调查研究，讨论了各种医学语料库及其特征、医学编码、流行嵌入模型的概述和比较，并将临床嵌入表示划分为九类，并详细讨论每种嵌入类型，探讨临床嵌入表示法中的各种挑战及可能的解决方案和未来研究方向。

临床自然语言处理中嵌入技术综述

SECNLP: A Survey of Embeddings in Clinical Natural Language Processing

BACKGROUND: The amount of biomedical literature is rapidly growing and it is
becoming increasingly difficult to keep manually curated knowledge bases and
ontologies up-to-date. In this study we applied the word2vec deep learning
toolkit to medical corpora to test its potential for identifying relationships
from unstructured text. We evaluated the efficiency of word2vec in identifying
properties of pharmaceuticals based on mid-sized, unstructured medical text
corpora available on the web. Properties included relationships to diseases
('may treat') or physiological processes ('has physiological effect'). We
compared the relationships identified by word2vec with manually curated
information from the National Drug File - Reference Terminology (NDF-RT)
ontology as a gold standard. RESULTS: Our results revealed a maximum accuracy
of 49.28% which suggests a limited ability of word2vec to capture linguistic
regularities on the collected medical corpora compared with other published
results. We were able to document the influence of different parameter settings
on result accuracy and found and unexpected trade-off between ranking quality
and accuracy. Pre-processing corpora to reduce syntactic variability proved to
be a good strategy for increasing the utility of the trained vector models.
CONCLUSIONS: Word2vec is a very efficient implementation for computing vector
representations and for its ability to identify relationships in textual data
without any prior domain knowledge. We found that the ranking and retrieved
results generated by word2vec were not of sufficient quality for automatic
population of knowledge bases and ontologies, but could serve as a starting
point for further manual curation.

本研究采用 Word2Vec 深度学习工具对医学语料库进行了测试，以验证其在未结构化文本中识别关系的潜力，结果表明 Word2Vec 的排名生成和检索结果质量不足以用于知识库自动填充，但可作为手动筛选的起点。