We design a family of image classification architectures that optimize the
trade-off between accuracy and efficiency in a high-speed regime. Our work
exploits recent findings in attention-based architectures, which are
competitive on highly parallel processing hardware. We revisit principles from
the extensive literature on convolutional neural networks to apply them to
transformers, in particular activation maps with decreasing resolutions. We
also introduce the attention bias, a new way to integrate positional
information in vision transformers. As a result, we propose LeVIT: a hybrid
neural network for fast inference image classification. We consider different
measures of efficiency on different hardware platforms, so as to best reflect a
wide range of application scenarios. Our extensive experiments empirically
validate our technical choices and show they are suitable to most
architectures. Overall, LeViT significantly outperforms existing convnets and
vision transformers with respect to the speed/accuracy tradeoff. For example,
at 80% ImageNet top-1 accuracy, LeViT is 5 times faster than EfficientNet on
CPU. We release the code at this https URL

本文介绍了 LeViT：一个用于快速推理图像分类的混合神经网络，它利用了最近在具有高度并行处理硬件竞争力的注意力架构中的发现，重新审视了卷积神经网络文献中的原则以将它们应用于 transformers，特别是带有分辨率降低的激活映射，还引入了一种新的方式来将位置信息集成到 vision transformers 中的注意偏差。