Large pre-trained language models have brought remarkable progress in NLP.
Pre-training and Fine-tuning have given state-of-art performance across tasks
in text processing. Data Augmentation techniques have also helped build
state-of-art models on low or zero resource tasks. Many works in the past have
attempted at learning a single massively-multilingual machine translation model
for zero-shot translation. Although those translation models are producing
correct translations, the main challenge is those models are producing the
wrong languages for zero-shot translation. This work and its results indicate
that prompt conditioned large models do not suffer from off-target language
errors i.e. errors arising due to translation to wrong languages. We
empirically demonstrate the effectiveness of self-supervised pre-training and
data augmentation for zero-shot multi-lingual machine translation.

本研究表明，在零样本机器翻译中，基于输入提示的大型模型不会出现离题语言错误，实验证明自监督预训练和数据增强对于零样本多语言机器翻译的效果显著。

MALM：混合增强语言建模实现零 - shot 机器翻译

MALM: Mixing Augmented Language Modeling for Zero-Shot Machine Translation

Our book "The Reality of Multi-Lingual Machine Translation" discusses the
benefits and perils of using more than two languages in machine translation
systems. While focused on the particular task of sequence-to-sequence
processing and multi-task learning, the book targets somewhat beyond the area
of natural language processing. Machine translation is for us a prime example
of deep learning applications where human skills and learning capabilities are
taken as a benchmark that many try to match and surpass. We document that some
of the gains observed in multi-lingual translation may result from simpler
effects than the assumed cross-lingual transfer of knowledge.
In the first, rather general part, the book will lead you through the
motivation for multi-linguality, the versatility of deep neural networks
especially in sequence-to-sequence tasks to complications of this learning. We
conclude the general part with warnings against too optimistic and unjustified
explanations of the gains that neural networks demonstrate.
In the second part, we fully delve into multi-lingual models, with a
particularly careful examination of transfer learning as one of the more
straightforward approaches utilizing additional languages. The recent
multi-lingual techniques, including massive models, are surveyed and practical
aspects of deploying systems for many languages are discussed. The conclusion
highlights the open problem of machine understanding and reminds of two ethical
aspects of building large-scale models: the inclusivity of research and its
ecological trace.

本书通过对多语言机器翻译的探讨阐述了自然语言处理、深度学习及多语言模型和迁移学习等相关问题，并提醒人们在构建大规模模型时需要考虑研究的包容性和生态痕迹。