We introduce an algorithm where the individual bits representing the weights
of a neural network are learned. This method allows training weights with
integer values on arbitrary bit-depths and naturally uncovers sparse networks,
without additional constraints or regularization techniques. We show better
results than the standard training technique with fully connected networks and
similar performance as compared to standard training for convolutional and
residual networks. By training bits in a selective manner we found that the
biggest contribution to achieving high accuracy is given by the first three
most significant bits, while the rest provide an intrinsic regularization. As a
consequence more than 90\% of a network can be used to store arbitrary codes
without affecting its accuracy. These codes may be random noise, binary files
or even the weights of previously trained networks.

介绍了一种学习表示神经网络权重的个别位的算法，允许对任意位深度的整数值进行训练，无需额外的约束或正则化技术，自然地发现了稀疏网络。证明了在选择性地训练位的情况下，实现高准确度的最大贡献来自前三位最重要的位，而其余位则提供内在的正则化。因此，可以使用超过 90％的网络存储任意代码，例如随机噪声，二进制文件或甚至先前训练的网络的权重。