A common evaluation practice in the vector space models (VSMs) literature is
to measure the models' ability to predict human judgments about lexical
semantic relations between word pairs. Most existing evaluation sets, however,
consist of scores collected for English word pairs only, ignoring the potential
impact of the judgment language in which word pairs are presented on the human
scores. In this paper we translate two prominent evaluation sets, wordsim353
(association) and SimLex999 (similarity), from English to Italian, German and
Russian and collect scores for each dataset from crowdworkers fluent in its
language. Our analysis reveals that human judgments are strongly impacted by
the judgment language. Moreover, we show that the predictions of monolingual
VSMs do not necessarily best correlate with human judgments made with the
language used for model training, suggesting that models and humans are
affected differently by the language they use when making semantic judgments.
Finally, we show that in a large number of setups, multilingual VSM combination
results in improved correlations with human judgments, suggesting that
multilingualism may partially compensate for the judgment language effect on
human judgments.

本研究旨在通过将两个英文评估集（Wordsim353 和 SimLex999）翻译成意大利语、德语和俄语，并从熟悉该语言的人群工作者那里收集每个数据集的得分，以评估向量空间模型（VSM）在预测人类在其母语下对词汇语义关系的判断方面的能力，结果发现人类的判断结果强烈受到评判语言的影响，而多语言 VSM 的组合可以部分补偿评判语言对人类判断的影响。

分离的不同语言：基于判断语言信息的向量空间建模

Separated by an Un-common Language: Towards Judgment Language Informed  Vector Space Modeling

Recent research has taken advantage of Wikipedia's multilingualism as a
resource for cross-language information retrieval and machine translation, as
well as proposed techniques for enriching its cross-language structure. The
availability of documents in multiple languages also opens up new opportunities
for querying structured Wikipedia content, and in particular, to enable answers
that straddle different languages. As a step towards supporting such queries,
in this paper, we propose a method for identifying mappings between attributes
from infoboxes that come from pages in different languages. Our approach finds
mappings in a completely automated fashion. Because it does not require
training data, it is scalable: not only can it be used to find mappings between
many language pairs, but it is also effective for languages that are
under-represented and lack sufficient training samples. Another important
benefit of our approach is that it does not depend on syntactic similarity
between attribute names, and thus, it can be applied to language pairs that
have distinct morphologies. We have performed an extensive experimental
evaluation using a corpus consisting of pages in Portuguese, Vietnamese, and
English. The results show that not only does our approach obtain high precision
and recall, but it also outperforms state-of-the-art techniques. We also
present a case study which demonstrates that the multilingual mappings we
derive lead to substantial improvements in answer quality and coverage for
structured queries over Wikipedia content.

本文提出了一个方法，用于在不同语言的 Infoboxes 之间识别属性映射，该方法具有可扩展性和适用性广泛性，它能够通过 Wikipedia 的多语言资源和跨语言结构支持 Wikipedia 内容的结构化查询，并提高了查询结果的质量和覆盖范围。