Quantized neural networks employ reduced precision representations for both
weights and activations. This quantization process significantly reduces the
memory requirements and computational complexity of the network. Binary Neural
Networks (BNNs) are the extreme quantization case, representing values with
just one bit. Since the sign function is typically used to map real values to
binary values, smooth approximations are introduced to mimic the gradients
during error backpropagation. Thus, the mismatch between the forward and
backward models corrupts the direction of the gradient, causing training
inconsistency problems and performance degradation. In contrast to current BNN
approaches, we propose to employ a binary periodic (BiPer) function during
binarization. Specifically, we use a square wave for the forward pass to obtain
the binary values and employ the trigonometric sine function with the same
period of the square wave as a differentiable surrogate during the backward
pass. We demonstrate that this approach can control the quantization error by
using the frequency of the periodic function and improves network performance.
Extensive experiments validate the effectiveness of BiPer in benchmark datasets
and network architectures, with improvements of up to 1% and 0.69% with respect
to state-of-the-art methods in the classification task over CIFAR-10 and
ImageNet, respectively. Our code is publicly available at
this https URL

通过使用二进制周期函数，将二进制神经网络的前向和反向模型一致化，可以控制量化误差并提高网络性能。在 CIFAR-10 和 ImageNet 数据集上的分类任务中，相比于现有方法，我们的方法能获得高达 1% 和 0.69% 的改进效果。