Low bit-width weights and activations are an effective way of combating the
increasing need for both memory and compute power of Deep Neural Networks. In
this work, we present a probabilistic training method for Neural Network with
both binary weights and activations, called BLRNet. By embracing stochasticity
during training, we circumvent the need to approximate the gradient of
non-differentiable functions such as sign(), while still obtaining a fully
Binary Neural Network at test time. Moreover, it allows for anytime ensemble
predictions for improved performance and uncertainty estimates by sampling from
the weight distribution. Since all operations in a layer of the BLRNet operate
on random variables, we introduce stochastic versions of Batch Normalization
and max pooling, which transfer well to a deterministic network at test time.
We evaluate the BLRNet on multiple standardized benchmarks.

本文提出使用二进制权重和激活的神经网络的概率训练方法 - BLRNet，通过随机性避免了对 sign () 等不可微函数的梯度的逼近，并仍在测试时获得完全的二进制神经网络。 此外，它允许通过从权重分布中进行抽样来进行任时集成预测以提高性能和不确定性估计。我们在多个标准基准上评估了 BLRNet。