Vector-based word representations help countless Natural Language Processing
(NLP) tasks capture both semantic and syntactic regularities of the language.
In this paper, we present the characteristics of existing word embedding
approaches and analyze them with regards to many classification tasks. We
categorize the methods into two main groups - Traditional approaches mostly use
matrix factorization to produce word representations, and they are not able to
capture the semantic and syntactic regularities of the language very well.
Neural-Network based approaches, on the other hand, can capture sophisticated
regularities of the language and preserve the word relationships in the
generated word representations. We report experimental results on multiple
classification tasks and highlight the scenarios where one approach performs
better than the rest.

本文通过对现有单词嵌入方法的特点和分类任务的分析，将单词嵌入方法划分为传统方法和基于神经网络的方法，揭示基于神经网络的单词表示方法相比于传统方法更能捕捉语言的语义和句法规律。实验验证了不同方法的性能差异。

现有词嵌入方法的全面实证评估

A Comprehensive Empirical Evaluation of Existing Word Embedding Approaches

There has recently been much interest in extending vector-based word
representations to multiple languages, such that words can be compared across
languages. In this paper, we shift the focus from words to documents and
introduce a method for embedding documents written in any language into a
single, language-independent vector space. For training, our approach leverages
a multilingual corpus where the same concept is covered in multiple languages
(but not necessarily via exact translations), such as Wikipedia. Our method,
Cr5 (Crosslingual reduced-rank ridge regression), starts by training a
ridge-regression-based classifier that uses language-specific bag-of-word
features in order to predict the concept that a given document is about. We
show that, when constraining the learned weight matrix to be of low rank, it
can be factored to obtain the desired mappings from language-specific
bags-of-words to language-independent embeddings. As opposed to most prior
methods, which use pretrained monolingual word vectors, postprocess them to
make them crosslingual, and finally average word vectors to obtain document
vectors, Cr5 is trained end-to-end and is thus natively crosslingual as well as
document-level. Moreover, since our algorithm uses the singular value
decomposition as its core operation, it is highly scalable. Experiments show
that our method achieves state-of-the-art performance on a crosslingual
document retrieval task. Finally, although not trained for embedding sentences
and words, it also achieves competitive performance on crosslingual sentence
and word retrieval tasks.

本文介绍了一种新的跨语言文件嵌入方法 Cr5，它使用跨语言降维回归来在多语言文本语料库中将文档嵌入到单一、不依赖于语言的向量空间中。本方法使用奇异值分解作为核心操作，可扩展性极强，在跨语言文档检索任务中实现了最先进的性能。

基于降秩岭回归的跨语言文档嵌入

Crosslingual Document Embedding as Reduced-Rank Ridge Regression

The zero-shot paradigm exploits vector-based word representations extracted
from text corpora with unsupervised methods to learn general mapping functions
from other feature spaces onto word space, where the words associated to the
nearest neighbours of the mapped vectors are used as their linguistic labels.
We show that the neighbourhoods of the mapped elements are strongly polluted by
hubs, vectors that tend to be near a high proportion of items, pushing their
correct labels down the neighbour list. After illustrating the problem
empirically, we propose a simple method to correct it by taking the proximity
distribution of potential neighbours across many mapped vectors into account.
We show that this correction leads to consistent improvements in realistic
zero-shot experiments in the cross-lingual, image labeling and image retrieval
domains.

该研究提出了一种简单的方法来解决 zero-shot 任务中邻居向量被强权向心性卡住的问题，即通过考虑多个向量中潜在邻居的相似度分布来校正错误结果。结果表明，这种校正方法在跨语言、图像标记和图像检索等领域的 zero-shot 任务中均获得了稳定的改善。