Transformers have achieved promising results on a variety of tasks. However,
the quadratic complexity in self-attention computation has limited the
applications, especially in low-resource settings and mobile or edge devices.
Existing works have proposed to exploit hand-crafted attention patterns to
reduce computation complexity. However, such hand-crafted patterns are
data-agnostic and may not be optimal. Hence, it is likely that relevant keys or
values are being reduced, while less important ones are still preserved. Based
on this key insight, we propose a novel deformable audio Transformer for audio
recognition, named DATAR, where a deformable attention equipping with a pyramid
transformer backbone is constructed and learnable. Such an architecture has
been proven effective in prediction tasks,~\textit{e.g.}, event classification.
Moreover, we identify that the deformable attention map computation may
over-simplify the input feature, which can be further enhanced. Hence, we
introduce a learnable input adaptor to alleviate this issue, and DATAR achieves
state-of-the-art performance.

基于变形注意力和金字塔 Transformer 骨干，提出了一种用于音频识别的新型变形音频 Transformer，名为 DATAR，解决了自注意力计算中的二次复杂度问题，适用于低资源环境和移动 / 边缘设备，并通过引入学习输入适配器对变形注意力图计算进行增强，实现了最先进的性能。