Cross-lingual text representations have gained popularity lately and act as
the backbone of many tasks such as unsupervised machine translation and
cross-lingual information retrieval, to name a few. However, evaluation of such
representations is difficult in the domains beyond standard benchmarks due to
the necessity of obtaining domain-specific parallel language data across
different pairs of languages. In this paper, we propose an automatic metric for
evaluating the quality of cross-lingual textual representations using images as
a proxy in a paired image-text evaluation dataset. Experimentally,
Backretrieval is shown to highly correlate with ground truth metrics on
annotated datasets, and our analysis shows statistically significant
improvements over baselines. Our experiments conclude with a case study on a
recipe dataset without parallel cross-lingual data. We illustrate how to judge
cross-lingual embedding quality with Backretrieval, and validate the outcome
with a small human study.

本文提出了一种使用图像作为代理的自动度量标准，评估跨语言文本表示的质量，实验表明，Backretrieval 能够高度相关地与标注数据的正确度量相关，是评估跨语言文本表示质量的一种有效方式。