This paper aims at addressing the problem of substantial performance
degradation at extremely low computational cost (e.g. 5M FLOPs on ImageNet
classification). We found that two factors, sparse connectivity and dynamic
activation function, are effective to improve the accuracy. The former avoids
the significant reduction of network width, while the latter mitigates the
detriment of reduction in network depth. Technically, we propose
micro-factorized convolution, which factorizes a convolution matrix into low
rank matrices, to integrate sparse connectivity into convolution. We also
present a new dynamic activation function, named Dynamic Shift Max, to improve
the non-linearity via maxing out multiple dynamic fusions between an input
feature map and its circular channel shift. Building upon these two new
operators, we arrive at a family of networks, named MicroNet, that achieves
significant performance gains over the state of the art in the low FLOP regime.
For instance, under the constraint of 12M FLOPs, MicroNet achieves 59.4\% top-1
accuracy on ImageNet classification, outperforming MobileNetV3 by 9.6\%. Source
code is at
\href{https://github.com/liyunsheng13/micronet}{this https URL}.

本论文提出了两种新的方法 —— 微型分解卷积和动态移位最大化激活函数，并将其应用在一种新型网络 MicroNet 中，使得在低计算复杂度条件下，ImageNet 分类任务 top-1 精度能够比 MobileNetV3 高出 9.6 个百分点。

MicroNet：极低 FLOPs 下提升图像识别

MicroNet: Improving Image Recognition with Extremely Low FLOPs

In this paper, we present MicroNet, which is an efficient convolutional
neural network using extremely low computational cost (e.g. 6 MFLOPs on
ImageNet classification). Such a low cost network is highly desired on edge
devices, yet usually suffers from a significant performance degradation. We
handle the extremely low FLOPs based upon two design principles: (a) avoiding
the reduction of network width by lowering the node connectivity, and (b)
compensating for the reduction of network depth by introducing more complex
non-linearity per layer. Firstly, we propose Micro-Factorized convolution to
factorize both pointwise and depthwise convolutions into low rank matrices for
a good tradeoff between the number of channels and input/output connectivity.
Secondly, we propose a new activation function, named Dynamic Shift-Max, to
improve the non-linearity via maxing out multiple dynamic fusions between an
input feature map and its circular channel shift. The fusions are dynamic as
their parameters are adapted to the input. Building upon Micro-Factorized
convolution and dynamic Shift-Max, a family of MicroNets achieve a significant
performance gain over the state-of-the-art in the low FLOP regime. For
instance, MicroNet-M1 achieves 61.1% top-1 accuracy on ImageNet classification
with 12 MFLOPs, outperforming MobileNetV3 by 11.3%.

本文介绍了 MicroNet，一种高效的卷积神经网络，利用极低的计算成本（例如，在 ImageNet 分类中为 6 MFLOPs）。这种低成本网络在边缘设备上非常受欢迎，但通常会遭受显著的性能下降。我们提出了 Micro-Factorized 卷积和 Dynamic Shift-Max 激活函数两个设计原则，为构建性能显著优于现有最佳法在低 FLOP 区间的 MicroNets 打下基础，例如，MicroNet-M1 在 ImageNet 分类中以 12 MFLOPs 为 6.1％，超过 MobileNetV3 的 11.3％。