Recent works have revealed that Transformers are implicitly learning the
syntactic information in its lower layers from data, albeit is highly dependent
on the quality and scale of the training data. However, learning syntactic
information from data is not necessary if we can leverage an external syntactic
parser, which provides better parsing quality with well-defined syntactic
structures. This could potentially improve Transformer's performance and sample
efficiency. In this work, we propose a syntax-guided localized self-attention
for Transformer that allows directly incorporating grammar structures from an
external constituency parser. It prohibits the attention mechanism to
overweight the grammatically distant tokens over close ones. Experimental
results show that our model could consistently improve translation performance
on a variety of machine translation datasets, ranging from small to large
dataset sizes, and with different source languages.

本文提出了一种语法指导的本地化自注意力 Transformer，该方法允许直接从外部成分句法分析器中加入语法结构，禁止注意机制通过加权距离较远但语法正确的令牌而不是接近的令牌。实验结果显示，我们的模型可以在从小到大不同规模的机器翻译数据集上持续地提高翻译性能。