The field of cross-lingual sentence embeddings has recently experienced
significant advancements, but research concerning low-resource languages has
lagged due to the scarcity of parallel corpora. This paper shows that
cross-lingual word representation in low-resource languages is notably
under-aligned with that in high-resource languages in current models. To
address this, we introduce a novel framework that explicitly aligns words
between English and eight low-resource languages, utilizing off-the-shelf word
alignment models. This framework incorporates three primary training
objectives: aligned word prediction and word translation ranking, along with
the widely used translation ranking. We evaluate our approach through
experiments on the bitext retrieval task, which demonstrate substantial
improvements on sentence embeddings in low-resource languages. In addition, the
competitive performance of the proposed model across a broader range of tasks
in high-resource languages underscores its practicality.

跨语言句子嵌入领域最近取得了很大的进展，但是由于平行语料的稀缺性，对于低资源语言的研究相对滞后。本文表明当前模型中低资源语言的跨语言词表示与高资源语言的对齐程度明显不足。为了解决这个问题，我们引入了一种新的框架，通过使用现成的词对齐模型，显式地对英语和八种低资源语言之间的单词进行对齐。该框架包含三个主要的训练目标：对齐的单词预测、单词翻译排序，以及广泛使用的翻译排序。我们通过在双语检索任务上进行实验证明了我们的方法，该方法在低资源语言的句子嵌入上取得了显著的改进。此外，所提出模型在高资源语言上更广泛任务的竞争性表现凸显了其实用性。

利用词对齐改进低资源语言的跨语言句子嵌入

Enhancing Cross-lingual Sentence Embedding for Low-resource Languages  with Word Alignment

West Greenlandic, known by native speakers as Kalaallisut, is an extremely
low-resource polysynthetic language spoken by around 56,000 people in
Greenland. Here, we attempt to finetune a pretrained Kalaallisut-to-English
neural machine translation (NMT) system using web-crawled pseudoparallel
sentences from around 30 multilingual websites. We compile a corpus of over
93,000 Kalaallisut sentences and over 140,000 Danish sentences, then use
cross-lingual sentence embeddings and approximate nearest-neighbors search in
an attempt to mine near-translations from these corpora. Finally, we translate
the Danish sentence to English to obtain a synthetic Kalaallisut-English
aligned corpus. Although the resulting dataset is too small and noisy to
improve the pretrained MT model, we believe that with additional resources, we
could construct a better pseudoparallel corpus and achieve more promising
results on MT. We also note other possible uses of the monolingual Kalaallisut
data and discuss directions for future work. We make the code and data for our
experiments publicly available.

本文介绍了如何利用多语言网站数据，构建 Kalaallisut 语言和英语之间的机器翻译模型并生成合成语料库，同时也讨论了如何利用单语 Kalaallisut 数据以及未来的研究方向和可能性。

使用网络抓取的数据微调 Kalaallisut - 英语机器翻译系统

Finetuning a Kalaallisut-English machine translation system using web-crawled data

Document alignment aims to identify pairs of documents in two distinct
languages that are of comparable content or translations of each other. Such
aligned data can be used for a variety of NLP tasks from training cross-lingual
representations to mining parallel data for machine translation. In this paper
we develop an unsupervised scoring function that leverages cross-lingual
sentence embeddings to compute the semantic distance between documents in
different languages. These semantic distances are then used to guide a document
alignment algorithm to properly pair cross-lingual web documents across a
variety of low, mid, and high-resource language pairs. Recognizing that our
proposed scoring function and other state of the art methods are
computationally intractable for long web documents, we utilize a more tractable
greedy algorithm that performs comparably. We experimentally demonstrate that
our distance metric performs better alignment than current baselines
outperforming them by 7% on high-resource language pairs, 15% on mid-resource
language pairs, and 22% on low-resource language pairs.

本文提出一种基于跨语言句嵌入的无监督打分函数，用于计算不同语言中文档之间的语义距离，从而指导文档对齐算法以适当地匹配跨语言 Web 文档，并在不同语言对中显著提高对齐效果。