In computer vision and machine learning, a crucial challenge is to lower the
computation and memory demands for neural network inference. A commonplace
solution to address this challenge is through the use of binarization. By
binarizing the network weights and activations, one can significantly reduce
computational complexity by substituting the computationally expensive floating
operations with faster bitwise operations. This leads to a more efficient
neural network inference that can be deployed on low-resource devices. In this
work, we extend previous approaches that trained networks with discrete weights
using the local reparameterization trick to also allow for discrete
activations. The original approach optimized a distribution over the discrete
weights and uses the central limit theorem to approximate the pre-activation
with a continuous Gaussian distribution. Here we show that the probabilistic
modeling can also allow effective training of networks with discrete activation
as well. This further reduces runtime and memory footprint at inference time
with state-of-the-art results for networks with binary activations.

研究二值化神经网络的运算和内存需求，提出通过训练网络的离散权重和激活，结合概率建模，实现更高效的离散激活网络推断，以降低运行时间和内存占用。