This paper studies the Binary Neural Networks (BNNs) in which weights and
activations are both binarized into 1-bit values, thus greatly reducing the
memory usage and computational complexity. Since the modern deep neural
networks are of sophisticated design with complex architecture for the accuracy
reason, the diversity on distributions of weights and activations is very high.
Therefore, the conventional sign function cannot be well used for effectively
binarizing full-precision values in BNNs. To this end, we present a simple yet
effective approach called AdaBin to adaptively obtain the optimal binary sets
$\{b_1, b_2\}$ ($b_1, b_2\in \mathbb{R}$) of weights and activations for each
layer instead of a fixed set (\textit{i.e.}, $\{-1, +1\}$). In this way, the
proposed method can better fit different distributions and increase the
representation ability of binarized features. In practice, we use the center
position and distance of 1-bit values to define a new binary quantization
function. For the weights, we propose an equalization method to align the
symmetrical center of binary distribution to real-valued distribution, and
minimize the Kullback-Leibler divergence of them. Meanwhile, we introduce a
gradient-based optimization method to get these two parameters for activations,
which are jointly trained in an end-to-end manner. Experimental results on
benchmark models and datasets demonstrate that the proposed AdaBin is able to
achieve state-of-the-art performance. For instance, we obtain a 66.4% Top-1
accuracy on the ImageNet using ResNet-18 architecture, and a 69.4 mAP on PASCAL
VOC using SSD300. The PyTorch code is available at
\url{this https URL}
and the MindSpore code is available at
\url{this https URL}.

本文提出了一种名为 AdaBin 的自适应二值化方法，它通过使用符号函数难以准确地将全精度值二值化的事实，使得我们获得了一组自适应的最优二进制集合，进一步提高了二值特征的表示能力。实验结果表明，AdaBin 在大量基准模型和数据集上达到了 state-of-the-art 的性能。