We present Mobile-Former, a parallel design of MobileNet and transformer with
a two-way bridge in between. This structure leverages the advantages of
MobileNet at local processing and transformer at global interaction. And the
bridge enables bidirectional fusion of local and global features. Different
from recent works on vision transformer, the transformer in Mobile-Former
contains very few tokens (e.g. 6 or fewer tokens) that are randomly initialized
to learn global priors, resulting in low computational cost. Combining with the
proposed light-weight cross attention to model the bridge, Mobile-Former is not
only computationally efficient, but also has more representation power. It
outperforms MobileNetV3 at low FLOP regime from 25M to 500M FLOPs on ImageNet
classification. For instance, Mobile-Former achieves 77.9\% top-1 accuracy at
294M FLOPs, gaining 1.3\% over MobileNetV3 but saving 17\% of computations.
When transferring to object detection, Mobile-Former outperforms MobileNetV3 by
8.6 AP in RetinaNet framework. Furthermore, we build an efficient end-to-end
detector by replacing backbone, encoder and decoder in DETR with Mobile-Former,
which outperforms DETR by 1.1 AP but saves 52\% of computational cost and 36\%
of parameters.

Mobile-Former 是一种结合 MobileNet 和 Transformer 的二元桥设计，其具有较低的计算成本和更强的表示能力，可以用于图像分类和对象检测，并在低 FLOP 区间内胜过 MobileNetV3 以及传统目标检测框架 DETR