Making decent multi-lingual sentence representations is critical to achieve
high performances in cross-lingual downstream tasks. In this work, we propose a
novel method to align multi-lingual embeddings based on the similarity of
sentences measured by a pre-trained mono-lingual embedding model. Given
translation sentence pairs, we train a multi-lingual model in a way that the
similarity between cross-lingual embeddings follows the similarity of sentences
measured at the mono-lingual teacher model. Our method can be considered as
contrastive learning with soft labels defined as the similarity between
sentences. Our experimental results on five languages show that our contrastive
loss with soft labels far outperforms conventional contrastive loss with hard
labels in various benchmarks for bitext mining tasks and STS tasks. In
addition, our method outperforms existing multi-lingual embeddings including
LaBSE, for Tatoeba dataset. The code is available at
this https URL

提出新的方法来对齐多语种嵌入，基于单语种嵌入模型中句子的相似性测量；实验证明我们的方法在多语种数据集上表现出色，优于现有的多语种嵌入方法和对比损失方法。