In this paper, we study 1-bit convolutional neural networks (CNNs), of which
both the weights and activations are binary. While efficient, the lacking of
representational capability and the training difficulty impede 1-bit CNNs from
performing as well as real-valued networks. We propose Bi-Real net with a novel
training algorithm to tackle these two challenges. To enhance the
representational capability, we propagate the real-valued activations generated
by each 1-bit convolution via a parameter-free shortcut. To address the
training difficulty, we propose a training algorithm using a tighter
approximation to the derivative of the sign function, a magnitude-aware
gradient for weight updating, a better initialization method, and a two-step
scheme for training a deep network. Experiments on ImageNet show that an
18-layer Bi-Real net with the proposed training algorithm achieves 56.4% top-1
classification accuracy, which is 10% higher than the state-of-the-arts (e.g.,
XNOR-Net) with greater memory saving and lower computational cost. Bi-Real net
is also the first to scale up 1-bit CNNs to an ultra-deep network with 152
layers, and achieves 64.5% top-1 accuracy on ImageNet. A 50-layer Bi-Real net
shows comparable performance to a real-valued network on the depth estimation
task with only a 0.3% accuracy gap.

本文研究了一比特卷积神经网络，通过使用 Bi-Real net 和新颖的训练算法来提高其表现能力和训练困难性，并在 ImageNet 上获得了 56.4％的 top-1 分类准确性，比现有技术提高了 10％以上。