Continuous word representation (aka word embedding) is a basic building block
in many neural network-based models used in natural language processing tasks.
Although it is widely accepted that words with similar semantics should be
close to each other in the embedding space, we find that word embeddings
learned in several tasks are biased towards word frequency: the embeddings of
high-frequency and low-frequency words lie in different subregions of the
embedding space, and the embedding of a rare word and a popular word can be far
from each other even if they are semantically similar. This makes learned word
embeddings ineffective, especially for rare words, and consequently limits the
performance of these neural network models. In this paper, we develop a neat,
simple yet effective way to learn \emph{FRequency-AGnostic word Embedding}
(FRAGE) using adversarial training. We conducted comprehensive studies on ten
datasets across four natural language processing tasks, including word
similarity, language modeling, machine translation and text classification.
Results show that with FRAGE, we achieve higher performance than the baselines
in all tasks.

本文介绍了一种通过对抗性训练学习频率不可知词嵌入（FRAGE）的方法，可使得语义相似的高频和低频词语在同一词嵌入空间中，实验表明此方法在十种数据集上均优于基准。