Although self-attention networks (SANs) have advanced the state-of-the-art on
various NLP tasks, one criticism of SANs is their ability of encoding positions
of input words (Shaw et al., 2018). In this work, we propose to augment SANs
with structural position representations to model the latent structure of the
input sentence, which is complementary to the standard sequential positional
representations. Specifically, we use dependency tree to represent the
grammatical structure of a sentence, and propose two strategies to encode the
positional relationships among words in the dependency tree. Experimental
results on NIST Chinese-to-English and WMT14 English-to-German translation
tasks show that the proposed approach consistently boosts performance over both
the absolute and relative sequential position representations.

本文提出将依赖树结构位置表示与传统的序列位置编码相结合，以更好地模拟输入句子的潜在结构，在中文到英文和英文到德语的翻译任务中，分别针对绝对和相对位置编码方式，实验证明提出的方法能够稳定提高自我注意力网络在各种自然语言处理任务上的表现。