Word vector specialisation (also known as retrofitting) is a portable,
light-weight approach to fine-tuning arbitrary distributional word vector
spaces by injecting external knowledge from rich lexical resources such as
WordNet. By design, these post-processing methods only update the vectors of
words occurring in external lexicons, leaving the representations of all unseen
words intact. In this paper, we show that constraint-driven vector space
specialisation can be extended to unseen words. We propose a novel
post-specialisation method that: a) preserves the useful linguistic knowledge
for seen words; while b) propagating this external signal to unseen words in
order to improve their vector representations as well. Our post-specialisation
approach explicits a non-linear specialisation function in the form of a deep
neural network by learning to predict specialised vectors from their original
distributional counterparts. The learned function is then used to specialise
vectors of unseen words. This approach, applicable to any post-processing
model, yields considerable gains over the initial specialisation models both in
intrinsic word similarity tasks, and in two downstream tasks: dialogue state
tracking and lexical text simplification. The positive effects persist across
three languages, demonstrating the importance of specialising the full
vocabulary of distributional word vector spaces.

该论文提出了一种基于神经网络的后置特化方法，能够更新分布式词向量空间中所有单词并且在内在单词上保留有用的语言知识，该方法对各种后处理模型都有显著的改进，跨越了三种语言的应用。