Vision transformers have been successfully applied to image recognition tasks
due to their ability to capture long-range dependencies within an image.
However, there are still gaps in both performance and computational cost
between transformers and existing convolutional neural networks (CNNs). In this
paper, we aim to address this issue and develop a network that can outperform
not only the canonical transformers, but also the high-performance
convolutional models. We propose a new transformer based hybrid network by
taking advantage of transformers to capture long-range dependencies, and of
CNNs to model local features. Furthermore, we scale it to obtain a family of
models, called CMTs, obtaining much better accuracy and efficiency than
previous convolution and transformer based models. In particular, our CMT-S
achieves 83.5% top-1 accuracy on ImageNet, while being 14x and 2x smaller on
FLOPs than the existing DeiT and EfficientNet, respectively. The proposed CMT-S
also generalizes well on CIFAR10 (99.2%), CIFAR100 (91.7%), Flowers (98.7%),
and other challenging vision datasets such as COCO (44.3% mAP), with
considerably less computational cost.

本文提出了一种基于 Transformer 和 CNN 的新型混合神经网络（CMTs），通过捕捉图像中的长程依赖和建模本地特征，实现了比现有的 DeiT 和 EfficientNet 更高的精度和更小的计算成本。