Self-attention networks (SANs) have drawn increasing interest due to their
high parallelization in computation and flexibility in modeling dependencies.
SANs can be further enhanced with multi-head attention by allowing the model to
attend to information from different representation subspaces. In this work, we
propose novel convolutional self-attention networks, which offer SANs the
abilities to 1) strengthen dependencies among neighboring elements, and 2)
model the interaction between features extracted by multiple attention heads.
Experimental results of machine translation on different language pairs and
model settings show that our approach outperforms both the strong Transformer
baseline and other existing models on enhancing the locality of SANs. Comparing
with prior studies, the proposed model is parameter free in terms of
introducing no more parameters.

本篇论文介绍了一种新颖的卷积自注意力网络，通过引入多头注意力机制，加强了邻近元素之间的依赖关系，并能够对各个注意力头提取的特征之间的交互进行建模，用于机器翻译任务中能有效提高自注意力网络的本地性，实验证明该方法优于常用的 Transformer 模型和其他已有的模型，并且没有更多的额外参数。