Binary Neural Networks (BNNs), known to be one among the effectively compact
network architectures, have achieved great outcomes in the visual tasks.
Designing efficient binary architectures is not trivial due to the binary
nature of the network. In this paper, we propose a use of evolutionary search
to facilitate the construction and training scheme when binarizing MobileNet, a
compact network with separable depth-wise convolution. Inspired by one-shot
architecture search frameworks, we manipulate the idea of group convolution to
design efficient 1-Bit Convolutional Neural Networks (CNNs), assuming an
approximately optimal trade-off between computational cost and model accuracy.
Our objective is to come up with a tiny yet efficient binary neural
architecture by exploring the best candidates of the group convolution while
optimizing the model performance in terms of complexity and latency. The
approach is threefold. First, we train strong baseline binary networks with a
wide range of random group combinations at each convolutional layer. This
set-up gives the binary neural networks a capability of preserving essential
information through layers. Second, to find a good set of hyperparameters for
group convolutions we make use of the evolutionary search which leverages the
exploration of efficient 1-bit models. Lastly, these binary models are trained
from scratch in a usual manner to achieve the final binary model. Various
experiments on ImageNet are conducted to show that following our construction
guideline, the final model achieves 60.09% Top-1 accuracy and outperforms the
state-of-the-art CI-BCNN with the same computational cost.

本研究提出了使用进化搜索来促进 MobileNet 的二值化设计和训练方案，通过操作组卷积的想法来设计高效的 1 位卷积神经网络，从而探索最佳的候选组卷积，以优化模型性能，并在 ImageNet 上进行了各种实验，展示了我们的构建指南，最终模型在相同计算成本下超越了现有技术。

基于进化搜索的二值化 MobileNet

Binarizing MobileNet via Evolution-based Searching

In this paper, we study 1-bit convolutional neural networks (CNNs), of which
both the weights and activations are binary. While efficient, the lacking of
representational capability and the training difficulty impede 1-bit CNNs from
performing as well as real-valued networks. We propose Bi-Real net with a novel
training algorithm to tackle these two challenges. To enhance the
representational capability, we propagate the real-valued activations generated
by each 1-bit convolution via a parameter-free shortcut. To address the
training difficulty, we propose a training algorithm using a tighter
approximation to the derivative of the sign function, a magnitude-aware
gradient for weight updating, a better initialization method, and a two-step
scheme for training a deep network. Experiments on ImageNet show that an
18-layer Bi-Real net with the proposed training algorithm achieves 56.4% top-1
classification accuracy, which is 10% higher than the state-of-the-arts (e.g.,
XNOR-Net) with greater memory saving and lower computational cost. Bi-Real net
is also the first to scale up 1-bit CNNs to an ultra-deep network with 152
layers, and achieves 64.5% top-1 accuracy on ImageNet. A 50-layer Bi-Real net
shows comparable performance to a real-valued network on the depth estimation
task with only a 0.3% accuracy gap.

本文研究了一比特卷积神经网络，通过使用 Bi-Real net 和新颖的训练算法来提高其表现能力和训练困难性，并在 ImageNet 上获得了 56.4％的 top-1 分类准确性，比现有技术提高了 10％以上。