Retrofitting techniques, which inject external resources into word
representations, have compensated the weakness of distributed representations
in semantic and relational knowledge between words. Implicitly retrofitting
word vectors by expansional technique outperforms retrofitting in word
similarity tasks with word vector generalization. In this paper, we propose
unsupervised extrofitting: expansional retrofitting (extrofitting) without
external semantic lexicons. We also propose deep extrofitting: in-depth
stacking of extrofitting and further combinations of extrofitting with
retrofitting. When experimenting with GloVe, we show that our methods
outperform the previous methods on most of word similarity tasks while
requiring only synonyms as an external resource. Lastly, we show the effect of
word vector enrichment on text classification task, as a downstream task.

本文提出了一种新的 Word embeddings 改进技术：extrofitting，它可以在不使用外部语义词典的情况下提高 word similarity 任务的表现，并且使用 deep extrofitting 可进一步提升表现。同时，我们证明了这种改进方法可以提高文本分类任务的效果。

词向量扩展改造

Expansional Retrofitting for Word Vector Enrichment

We propose post-processing method for enriching not only word representation
but also its vector space using semantic lexicons, which we call extrofitting.
The method consists of 3 steps as follows: (i) Expanding 1 or more dimension(s)
on all the word vectors, filling with their representative value. (ii)
Transferring semantic knowledge by averaging each representative values of
synonyms and filling them in the expanded dimension(s). These two steps make
representations of the synonyms close together. (iii) Projecting the vector
space using Linear Discriminant Analysis, which eliminates the expanded
dimension(s) with semantic knowledge. When experimenting with GloVe, we find
that our method outperforms Faruqui's retrofitting on some of word similarity
task. We also report further analysis on our method in respect to word vector
dimensions, vocabulary size as well as other well-known pretrained word vectors
(e.g., Word2Vec, Fasttext).

本文提出了一种名为 extrofitting 的后处理方法，使用语义词典丰富单词表示和其向量空间，包括向所有单词向量上扩展 1 个或多个维度，填充代表值并将语义知识传输到这些扩展的维度。此外，采用线性判别分析进行向量空间映射，得到最终的词向量表示。实验证明，该方法在一些单词相似度任务上胜过 Faruqui 的后调方法，并对其维度、词汇表大小等进行了进一步的分析。