Methods for learning word sense embeddings represent a single word with
multiple sense-specific vectors. These methods should not only produce
interpretable sense embeddings, but should also learn how to select which sense
to use in a given context. We propose an unsupervised model that learns sense
embeddings using a modified Gumbel softmax function, which allows for
differentiable discrete sense selection. Our model produces sense embeddings
that are competitive (and sometimes state of the art) on multiple similarity
based downstream evaluations. However, performance on these downstream
evaluations tasks does not correlate with interpretability of sense embeddings,
as we discover through an interpretability comparison with competing
multi-sense embeddings. While many previous approaches perform well on
downstream evaluations, they do not produce interpretable embeddings and learn
duplicated sense groups; our method achieves the best of both worlds.

该论文提出一种使用修改过的 Gumbel softmax 函数的无监督模型来学习词义嵌入，该模型可以进行可区分的离散词义选择，并通过多个基于相似性的下游评估实现了有竞争力（有时是最先进的）的词义嵌入。虽然该模型在下游评估任务上表现出色，但与竞争多词义嵌入的可解释性比较中发现，评估任务的性能与词义嵌入的可解释性不相关。该方法实现了最优选，既能够实现良好的下游任务性能，又能够产生可解释的嵌入且减少同义分组的学习。

使用缩放 Gumbel Softmax 诱导和嵌入感知

Inducing and Embedding Senses with Scaled Gumbel Softmax

We present a simple yet effective approach for learning word sense
embeddings. In contrast to existing techniques, which either directly learn
sense representations from corpora or rely on sense inventories from lexical
resources, our approach can induce a sense inventory from existing word
embeddings via clustering of ego-networks of related words. An integrated WSD
mechanism enables labeling of words in context with learned sense vectors,
which gives rise to downstream applications. Experiments show that the
performance of our method is comparable to state-of-the-art unsupervised WSD
systems.

我们开发了一种简单而有效的方法来学习单词意义嵌入。通过聚类相关单词的自我网络，我们的方法可以从现有的单词嵌入中引出一种意义库，并通过学习的意义向量标记上下文中的单词，从而产生了下游应用。实验表明，我们的方法的性能与最先进的无监督 WSD 系统相当。