Recent results show that deep neural networks achieve excellent performance
even when, during training, weights are quantized and projected to a binary
representation. Here, we show that this is just the tip of the iceberg: these
same networks, during testing, also exhibit a remarkable robustness to
distortions beyond quantization, including additive and multiplicative noise,
and a class of non-linear projections where binarization is just a special
case. To quantify this robustness, we show that one such network achieves 11%
test error on CIFAR-10 even with 0.68 effective bits per weight. Furthermore,
we find that a common training heuristic--namely, projecting quantized weights
during backpropagation--can be altered (or even removed) and networks still
achieve a base level of robustness during testing. Specifically, training with
weight projections other than quantization also works, as does simply clipping
the weights, both of which have never been reported before. We confirm our
results for CIFAR-10 and ImageNet datasets. Finally, drawing from these ideas,
we propose a stochastic projection rule that leads to a new state of the art
network with 7.64% test error on CIFAR-10 using no data augmentation.

本文研究发现深度神经网络在训练时即使将权重量化并投影为二进制表示也能取得优秀的性能，而在测试阶段，这些网络对于除量化以外的扭曲，包括加性和乘性噪声以及一类非线性投影具有显著的鲁棒性。此外，我们还发现基本水平的鲁棒性可以通过调整或删除一个通用的训练启发式（即在反向传播过程中投影量化权重）来实现，包括使用其他类型的权重投影和简单地剪裁权重。最后，我们提出了一种随机投影规则，实现了无数据增强条件下 CIFAR-10 中 7.64% 的测试错误率的最新记录。