In this paper, we elaborate upon recipes for building multilingual
representation models that are not only competitive with existing
state-of-the-art models but are also more parameter efficient, thereby
promoting better adoption in resource-constrained scenarios and practical
applications. We show that going beyond English-centric bitexts, coupled with a
novel sampling strategy aimed at reducing under-utilization of training data,
substantially boosts performance across model sizes for both Electra and MLM
pre-training objectives. We introduce XY-LENT: X-Y bitext enhanced Language
ENcodings using Transformers which not only achieves state-of-the-art
performance over 5 cross-lingual tasks within all model size bands, is also
competitive across bands. Our XY-LENT XL variant outperforms XLM-RXXL and
exhibits competitive performance with mT5 XXL while being 5x and 6x smaller
respectively. We then show that our proposed method helps ameliorate the curse
of multilinguality, with the XY-LENT XL achieving 99.3% GLUE performance and
98.5% SQuAD 2.0 performance compared to a SoTA English only model in the same
size band. We then analyze our models performance on extremely low resource
languages and posit that scaling alone may not be sufficient for improving the
performance in this scenario

本文介绍了一种构建多语言表示模型的方法，该方法不仅可以与现有的最先进的模型竞争，而且还更加参数高效，从而促进了资源受限情况和实际应用的更好采用。

更好的多语言表示学习：超越以英语为中心的双语语料

Beyond English-Centric Bitexts for Better Multilingual Language Representation Learning

Mining high-quality bitexts for low-resource languages is challenging. This
paper shows that sentence representation of language models fine-tuned with
multiple negatives ranking loss, a contrastive objective, helps retrieve clean
bitexts. Experiments show that parallel data mined from our approach
substantially outperform the previous state-of-the-art method on low resource
languages Khmer and Pashto.

本文介绍了一种针对低资源语言挖掘高质量双语数据的方法，即使用多负对比损失进行语言模型微调，可以获得更干净的双语数据，实验证明，在高棉语和普什图语等低资源语言数据挖掘方面，该方法明显优于以往的最先进方法。

对比学习在低资源语言的双语文本挖掘中的应用

Bitext Mining for Low-Resource Languages via Contrastive Learning

Simultaneous translation is vastly different from full-sentence translation,
in the sense that it starts translation before the source sentence ends, with
only a few words delay. However, due to the lack of large-scale, high-quality
simultaneous translation datasets, most such systems are still trained on
conventional full-sentence bitexts. This is far from ideal for the simultaneous
scenario due to the abundance of unnecessary long-distance reorderings in those
bitexts. We propose a novel method that rewrites the target side of existing
full-sentence corpora into simultaneous-style translation. Experiments on
Zh->En and Ja->En simultaneous translation show substantial improvements (up to
+2.7 BLEU) with the addition of these generated pseudo-references.

提出了一种将现有的全句子语料库改写为同时翻译风格的新方法，用生成的伪参考文本增强机器翻译系统，在中英文和日英文同时翻译任务中获得高达 2.7 BLEU 的显著改善。

利用伪参考文本并减少重新排序来提高同声传译质量

Improving Simultaneous Translation by Incorporating Pseudo-References  with Fewer Reorderings

Twenty-five years ago, morphologically aligned Hebrew-Finnish and
Greek-Finnish bitexts (texts accompanied by a translation) were constructed
manually in order to create an analytical concordance (Luoto et al., 1997) for
a Finnish Bible translation. The creators of the bitexts recently secured the
publisher's permission to release its fine-grained alignment, but the alignment
was still dependent on proprietary, third-party resources such as a copyrighted
text edition and proprietary morphological analyses of the source texts. In
this paper, we describe a nontrivial editorial process starting from the
creation of the original one-purpose database and ending with its
reconstruction using only freely available text editions and annotations. This
process produced an openly available dataset that contains (i) the source texts
and their translations, (ii) the morphological analyses, (iii) the
cross-lingual morpheme alignments.

本文介绍了一个包含源文本、翻译、词汇分析和跨语言形态素对齐的数据集，它是通过使用免费的文本版本和标注来重建一个人工构建的位对位语料库的。