Machine translation has seen rapid progress with the advent of
Transformer-based models. These models have no explicit linguistic structure
built into them, yet they may still implicitly learn structured relationships
by attending to relevant tokens. We hypothesize that this structural learning
could be made more robust by explicitly endowing Transformers with a structural
bias, and we investigate two methods for building in such a bias. One method,
the TP-Transformer, augments the traditional Transformer architecture to
include an additional component to represent structure. The second method
imbues structure at the data level by segmenting the data with morphological
tokenization. We test these methods on translating from English into
morphologically rich languages, Turkish and Inuktitut, and consider both
automatic metrics and human evaluations. We find that each of these two
approaches allows the network to achieve better performance, but this
improvement is dependent on the size of the dataset. In sum, structural
encoding methods make Transformers more sample-efficient, enabling them to
perform better from smaller amounts of data.

本论文旨在探讨如何通过加入结构偏置来提高 Transformer 的翻译能力，并且通过 TP-Transformer 和形态结构的分割两种方式来实现。实验表明，这两种方法都能使 Transformer 在翻译土耳其语和伊努伊特语等形态丰富的语言时表现更好，但要考虑数据集的大小。总的来说，结构编码方法可以使 Transformer 更加高效，在少量数据的情况下表现更好。