Massively multilingual sentence representation models, e.g., LASER,
SBERT-distill, and LaBSE, help significantly improve cross-lingual downstream
tasks. However, multiple training procedures, the use of a large amount of
data, or inefficient model architectures result in heavy computation to train a
new model according to our preferred languages and domains. To resolve this
issue, we introduce efficient and effective massively multilingual sentence
representation learning (EMS), using cross-lingual sentence reconstruction
(XTR) and sentence-level contrastive learning as training objectives. Compared
with related studies, the proposed model can be efficiently trained using
significantly fewer parallel sentences and GPU computation resources without
depending on large-scale pre-trained models. Empirical results show that the
proposed model significantly yields better or comparable results with regard to
bi-text mining, zero-shot cross-lingual genre classification, and sentiment
classification. Ablative analyses demonstrate the effectiveness of each
component of the proposed model. We release the codes for model training and
the EMS pre-trained model, which supports 62 languages
(this https URL).

通过跨语言重建和句子级对比学习，提出了一种有效的高效多语句子表示学习算法 EMS，不依赖于大规模预训练模型，能显著减少并行句子和 GPU 计算资源的使用，支持 62 种语言，在双语挖掘、零样本跨语言流派分类和情感分类方面，该模型显著获得更好或相当的效果。