We provide the first extensive evaluation of how using different types of context to learn skip-gram word embeddings affects performance on a wide range of intrinsic and extrinsic NLP tasks. Our results suggest that while intrinsic tasks tend to exhibit a clear preference to particular types of contexts and higher dimensionality, more careful tuning is required for finding the optimal settings for most of the extrinsic tasks that we considered. Furthermore, for these extrinsic tasks, we find that once the benefit from increasing the embedding dimensionality is mostly exhausted, simple concatenation of word embeddings, learned with different context types, can yield further performance gains. As an additional contribution, we propose a new variant of the skip-gram model that learns word embeddings from weighted contexts of substitute words.

本文通过对不同类型的上下文学习skip-gram词嵌入对广泛的内在和外在自然语言处理任务性能的首次全面评估，发现内在任务在特定类型的上下文和更高的维度方面更好，而对于大多数外在任务，需要更仔细的调整来找到最佳设置；另外，对于这些外在任务，一旦提高嵌入维度的好处大多用尽，使用不同上下文类型学习的简单词嵌入的串联，可以产生进一步的性能提升；此外，本文还提出了一种新的skip-gram模型变体，从替换词的加权上下文中学习单词嵌入。

学习词向量中上下文类型和维度的作用