The field of cross-lingual sentence embeddings has recently experienced
significant advancements, but research concerning low-resource languages has
lagged due to the scarcity of parallel corpora. This paper shows that
cross-lingual word representation in low-resource languages is notably
under-aligned with that in high-resource languages in current models. To
address this, we introduce a novel framework that explicitly aligns words
between English and eight low-resource languages, utilizing off-the-shelf word
alignment models. This framework incorporates three primary training
objectives: aligned word prediction and word translation ranking, along with
the widely used translation ranking. We evaluate our approach through
experiments on the bitext retrieval task, which demonstrate substantial
improvements on sentence embeddings in low-resource languages. In addition, the
competitive performance of the proposed model across a broader range of tasks
in high-resource languages underscores its practicality.

跨语言句子嵌入领域最近取得了很大的进展，但是由于平行语料的稀缺性，对于低资源语言的研究相对滞后。本文表明当前模型中低资源语言的跨语言词表示与高资源语言的对齐程度明显不足。为了解决这个问题，我们引入了一种新的框架，通过使用现成的词对齐模型，显式地对英语和八种低资源语言之间的单词进行对齐。该框架包含三个主要的训练目标：对齐的单词预测、单词翻译排序，以及广泛使用的翻译排序。我们通过在双语检索任务上进行实验证明了我们的方法，该方法在低资源语言的句子嵌入上取得了显著的改进。此外，所提出模型在高资源语言上更广泛任务的竞争性表现凸显了其实用性。

利用词对齐改进低资源语言的跨语言句子嵌入

Enhancing Cross-lingual Sentence Embedding for Low-resource Languages  with Word Alignment

An ongoing challenge in current natural language processing is how its major
advancements tend to disproportionately favor resource-rich languages, leaving
a significant number of under-resourced languages behind. Due to the lack of
resources required to train and evaluate models, most modern language
technologies are either nonexistent or unreliable to process endangered, local,
and non-standardized languages. Optical character recognition (OCR) is often
used to convert endangered language documents into machine-readable data.
However, such OCR output is typically noisy, and most word alignment models are
not built to work under such noisy conditions. In this work, we study the
existing word-level alignment models under noisy settings and aim to make them
more robust to noisy data. Our noise simulation and structural biasing method,
tested on multiple language pairs, manages to reduce the alignment error rate
on a state-of-the-art neural-based alignment model up to 59.6%.

本研究旨在研究现有的词级联模型在嘈杂环境下的对齐问题，并通过噪声模拟和结构偏置的方法提高模型的稳健性，从而显著降低基于神经网络的对齐模型的对齐误差率。