The inverted residual block is dominating architecture design for mobile
networks recently. It changes the classic residual bottleneck by introducing
two design rules: learning inverted residuals and using linear bottlenecks. In
this paper, we rethink the necessity of such design changes and find it may
bring risks of information loss and gradient confusion. We thus propose to flip
the structure and present a novel bottleneck design, called the sandglass
block, that performs identity mapping and spatial transformation at higher
dimensions and thus alleviates information loss and gradient confusion
effectively. Extensive experiments demonstrate that, different from the common
belief, such bottleneck structure is more beneficial than the inverted ones for
mobile networks. In ImageNet classification, by simply replacing the inverted
residual block with our sandglass block without increasing parameters and
computation, the classification accuracy can be improved by more than 1.7% over
MobileNetV2. On Pascal VOC 2007 test set, we observe that there is also 0.9%
mAP improvement in object detection. We further verify the effectiveness of the
sandglass block by adding it into the search space of neural architecture
search method DARTS. With 25% parameter reduction, the classification accuracy
is improved by 0.13% over previous DARTS models. Code can be found at:
this https URL

本文介绍了一种新的 bottleneck design，叫做 sandglass block，它通过在更高的维度上执行 identity mapping 和空间变换来减轻信息丢失和梯度混淆的风险，并证明了它对于移动网络的性能更有益。