Binary neural networks (BNNs) represent original full-precision weights and
activations into 1-bit with sign function. Since the gradient of the
conventional sign function is almost zero everywhere which cannot be used for
back-propagation, several attempts have been proposed to alleviate the
optimization difficulty by using approximate gradient. However, those
approximations corrupt the main direction of factual gradient. To this end, we
propose to estimate the gradient of sign function in the Fourier frequency
domain using the combination of sine functions for training BNNs, namely
frequency domain approximation (FDA). The proposed approach does not affect the
low-frequency information of the original sign function which occupies most of
the overall energy, and high-frequency coefficients will be ignored to avoid
the huge computational overhead. In addition, we embed a noise adaptation
module into the training phase to compensate the approximation error. The
experiments on several benchmark datasets and neural architectures illustrate
that the binary network learned using our method achieves the state-of-the-art
accuracy. Code will be available at
\textit{this https URL}.

本文提出了一种使用正弦函数组合估计傅里叶频率域中的符号函数梯度的方法，同时嵌入噪声自适应模块来弥补近似误差，结果表明该方法训练的二进制神经网络可达到最先进的精度水平。