Depthwise separable convolutions reduce the number of parameters and
computation used in convolutional operations while increasing representational
efficiency. They have been shown to be successful in image classification
models, both in obtaining better models than previously possible for a given
parameter count (the Xception architecture) and considerably reducing the
number of parameters required to perform at a given level (the MobileNets
family of architectures). Recently, convolutional sequence-to-sequence networks
have been applied to machine translation tasks with good results. In this work,
we study how depthwise separable convolutions can be applied to neural machine
translation. We introduce a new architecture inspired by Xception and ByteNet,
called SliceNet, which enables a significant reduction of the parameter count
and amount of computation needed to obtain results like ByteNet, and, with a
similar parameter count, achieves new state-of-the-art results. In addition to
showing that depthwise separable convolutions perform well for machine
translation, we investigate the architectural changes that they enable: we
observe that thanks to depthwise separability, we can increase the length of
convolution windows, removing the need for filter dilation. We also introduce a
new "super-separable" convolution operation that further reduces the number of
parameters and computational cost for obtaining state-of-the-art results.

本研究探讨如何将深度可分离卷积应用于神经机器翻译任务。我们提出了一种新的基于 Xception 和 ByteNet 的架构 SliceNet，并发现深度可分离卷积能够在减少参数数量和计算量的同时获得与 ByteNet 相似的性能，同时展示了深度可分离卷积对于神经机器翻译任务的性能表现以及其带来的架构变化。