The training paradigm for machine translation has gradually shifted, from
learning neural machine translation (NMT) models with extensive parallel
corpora to instruction finetuning on pretrained multilingual large language
models (LLMs) with high-quality translation pairs. In this paper, we focus on
boosting the many-to-many multilingual translation performance of LLMs with an
emphasis on zero-shot translation directions. We demonstrate that prompt
strategies adopted during instruction finetuning are crucial to zero-shot
translation performance and introduce a cross-lingual consistency
regularization, XConST, to bridge the representation gap among different
languages and improve zero-shot translation performance. XConST is not a new
method, but a version of CrossConST (Gao et al., 2023a) adapted for
multilingual finetuning on LLMs with translation instructions. Experimental
results on ALMA (Xu et al., 2023) and LLaMA-2 (Touvron et al., 2023) show that
our approach consistently improves translation performance. Our implementations
are available at this https URL

本文研究了机器翻译的训练模型，从学习具有广泛平行语料的神经机器翻译模型逐渐转变为在预训练的多语言大型语言模型上进行指导微调，并重点关注增强多对多多语言翻译性能和零 - shot 翻译方向的。我们证明了在指导微调过程中采用的提示策略对零 - shot 翻译性能至关重要，并引入了跨语言一致性正则化（CrossConST）的版本，即 XConST，以弥合不同语言之间的表示差距并提高零 - shot 翻译性能。在 ALMA 和 LLaMA-2 的实验结果表明，我们的方法持续改善了翻译性能。我们的实现代码可以在此 https URL 上找到。

提升多对多多语言机器翻译的方法与大型语言模型

Towards Boosting Many-to-Many Multilingual Machine Translation with  Large Language Models

Multilingual sentence representations are the foundation for similarity-based
bitext mining, which is crucial for scaling multilingual neural machine
translation (NMT) system to more languages. In this paper, we introduce MuSR: a
one-for-all Multilingual Sentence Representation model that supports more than
220 languages. Leveraging billions of English-centric parallel corpora, we
train a multilingual Transformer encoder, coupled with an auxiliary Transformer
decoder, by adopting a multilingual NMT framework with CrossConST, a
cross-lingual consistency regularization technique proposed in Gao et al.
(2023). Experimental results on multilingual similarity search and bitext
mining tasks show the effectiveness of our approach. Specifically, MuSR
achieves superior performance over LASER3 (Heffernan et al., 2022) which
consists of 148 independent multilingual sentence encoders.

MuSR 是一种支持 220 多种语言的全能多语种句子表示模型，通过使用数十亿个英文为中心的平行语料库，采用使用跨语言一致性规则化技术的多语言 NMT 框架，以训练多语言 Transformer 编码器，并采用辅助 Transformer 解码器，实现了多语言相似性搜索和双文本挖掘任务，表现优于由 148 个独立的多语种句子编码器组成的 LASER3。

跨语言一致性正则化学习多语句子表示

Learning Multilingual Sentence Representations with Cross-lingual  Consistency Regularization

The multilingual neural machine translation (NMT) model has a promising
capability of zero-shot translation, where it could directly translate between
language pairs unseen during training. For good transfer performance from
supervised directions to zero-shot directions, the multilingual NMT model is
expected to learn universal representations across different languages. This
paper introduces a cross-lingual consistency regularization, CrossConST, to
bridge the representation gap among different languages and boost zero-shot
translation performance. The theoretical analysis shows that CrossConST
implicitly maximizes the probability distribution for zero-shot translation,
and the experimental results on both low-resource and high-resource benchmarks
show that CrossConST consistently improves the translation performance. The
experimental analysis also proves that CrossConST could close the sentence
representation gap and better align the representation space. Given the
universality and simplicity of CrossConST, we believe it can serve as a strong
baseline for future multilingual NMT research.

本文介绍了一种跨语言一致性正则化方法 CrossConST，用于在多语言神经机器翻译模型中实现零 - shot 翻译，以实现不同语言之间的普适表示，并提高翻译性能。实验结果证明 CrossConST 能够提高模型的翻译性能并降低句子表示间的差距，为未来的多语言 NMT 研究奠定了基础。