We present BoTNet, a conceptually simple yet powerful backbone architecture that incorporates self-attention for multiple computer vision tasks including image classification, object detection and instance segmentation. By just replacing the spatial convolutions with global self-attention in the final three bottleneck blocks of a ResNet and no other changes, our approach improves upon the baselines significantly on instance segmentation and object detection while also reducing the parameters, with minimal overhead in latency. Through the design of BoTNet, we also point out how ResNet bottleneck blocks with self-attention can be viewed as Transformer blocks. Without any bells and whistles, BoTNet achieves 44.4% Mask AP and 49.7% Box AP on the COCO Instance Segmentation benchmark using the Mask R-CNN framework; surpassing the previous best published single model and single scale results of ResNeSt evaluated on the COCO validation set. Finally, we present a simple adaptation of the BoTNet design for image classification, resulting in models that achieve a strong performance of 84.7% top-1 accuracy on the ImageNet benchmark while being up to 2.33x faster in compute time than the popular EfficientNet models on TPU-v3 hardware. We hope our simple and effective approach will serve as a strong baseline for future research in self-attention models for vision.

BoTNet是一种基于自注意力机制的视觉模型，可以应用于图像分类、目标检测和实例分割，通过在ResNet的最后三个bottleneck块中使用自注意力代替空间卷积，能够大幅提高实例分割和目标检测的性能，同时减少参数量和计算开销。BoTNet的设计表明ResNet bottleneck块中的自注意力可以视为Transformer块。在COCO Instance Segmentation基准测试中，BoTNet能够取得44.4%的Mask AP和49.7%的Box AP，超过了之前最佳单模型和单尺度结果ResNeSt，同时在ImageNet基准测试中，BoTNet的简单改进能够取得84.7%的top-1准确率，比EfficientNet在TPU-v3上计算速度快1.64倍。这一简单而有效的方法有望成为未来自注意力视觉模型研究的强有力基准。

基于瓶颈结构的Transformer用于视觉识别