Contextualised word embeddings generated from Neural Language Models (NLMs), such as BERT, represent a word with a vector that considers the semantics of the target word as well its context. On the other hand, static word embeddings such as GloVe represent words by relatively low-dimensional, memory- and compute-efficient vectors but are not sensitive to the different senses of the word. We propose Context Derived Embeddings of Senses (CDES), a method that extracts sense related information from contextualised embeddings and injects it into static embeddings to create sense-specific static embeddings. Experimental results on multiple benchmarks for word sense disambiguation and sense discrimination tasks show that CDES can accurately learn sense-specific static embeddings reporting comparable performance to the current state-of-the-art sense embeddings.

本文介绍了一种名为Context Derived Embeddings of Senses (CDES)的方法，它从上下文嵌入中提取感官相关信息，并将其注入到静态嵌入中，以创建特定于感官的静态嵌入。实验结果表明，CDES能够准确地学习感官特定的静态嵌入，具有与当前最先进的感官嵌入相当的性能。

使用上下文化单词嵌入作为代理学习具有特定意义的静态嵌入