Neural Machine Translation (NMT) is an open vocabulary problem. As a result,
dealing with the words not occurring during training (a.k.a. out-of-vocabulary
(OOV) words) have long been a fundamental challenge for NMT systems. The
predominant method to tackle this problem is Byte Pair Encoding (BPE) which
splits words, including OOV words, into sub-word segments. BPE has achieved
impressive results for a wide range of translation tasks in terms of automatic
evaluation metrics. While it is often assumed that by using BPE, NMT systems
are capable of handling OOV words, the effectiveness of BPE in translating OOV
words has not been explicitly measured. In this paper, we study to what extent
BPE is successful in translating OOV words at the word-level. We analyze the
translation quality of OOV words based on word type, number of segments,
cross-attention weights, and the frequency of segment n-grams in the training
data. Our experiments show that while careful BPE settings seem to be fairly
useful in translating OOV words across datasets, a considerable percentage of
OOV words are translated incorrectly. Furthermore, we highlight the slightly
higher effectiveness of BPE in translating OOV words for special cases, such as
named-entities and when the languages involved are linguistically close to each
other.

本文针对 NMT 中 OOV 问题进行研究，发现尽管 BPE 可在一定程度上处理 OOV，但仍有相当一部分单词翻译错误，特别是涉及语言间距离较远的情况。