Neural Machine Translation (NMT) has become a significant technology in
natural language processing through extensive research and development.
However, the deficiency of high-quality bilingual language pair data still
poses a major challenge to improving NMT performance. Recent studies are
exploring the use of contextual information from pre-trained language model
(PLM) to address this problem. Yet, the issue of incompatibility between PLM
and NMT model remains unresolved. This study proposes a PLM-integrated NMT
(PiNMT) model to overcome the identified problems. The PiNMT model consists of
three critical components, PLM Multi Layer Converter, Embedding Fusion, and
Cosine Alignment, each playing a vital role in providing effective PLM
information to NMT. Furthermore, two training strategies, Separate Learning
Rates and Dual Step Training, are also introduced in this paper. By
implementing the proposed PiNMT model and training strategy, we achieved
state-of-the-art performance on the IWSLT'14 En$\leftrightarrow$De dataset.
This study's outcomes are noteworthy as they demonstrate a novel approach for
efficiently integrating PLM with NMT to overcome incompatibility and enhance
performance.

本研究提出了一种整合预训练语言模型和神经机器翻译的方法（PiNMT），通过引入 PLM 多层转换器、嵌入融合和余弦对齐等关键组件以及分离学习率和双步训练等训练策略，实现了 PLM 与 NMT 的有效整合，提升了性能，并在 IWSLT'14 数据集上取得了最新的研究成果。