Transformer-based deep neural networks have achieved great success in various
sequence applications due to their powerful ability to model long-range
dependency. The key module of Transformer is self-attention (SA) which extracts
features from the entire sequence regardless of the distance between positions.
Although SA helps Transformer performs particularly well on long-range tasks,
SA requires quadratic computation and memory complexity with the input sequence
length. Recently, attention map reuse, which groups multiple SA layers to share
one attention map, has been proposed and achieved significant speedup for
speech recognition models. In this paper, we provide a comprehensive study on
attention map reuse focusing on its ability to accelerate inference. We compare
the method with other SA compression techniques and conduct a breakdown
analysis of its advantages for a long sequence. We demonstrate the
effectiveness of attention map reuse by measuring the latency on both CPU and
GPU platforms.

本文研究了基于 Transformer 的自注意力（SA）提取序列特征方案，在注意力地图重用方面做了全面的研究，并证明了其在加速推理方面具有显著的优势。实验结果表明，注意力地图重用方法在 CPU 和 GPU 平台上可以减少推理延迟。