Binary neural networks are the extreme case of network quantization, which
has long been thought of as a potential edge machine learning solution.
However, the significant accuracy gap to the full-precision counterparts
restricts their creative potential for mobile applications. In this work, we
revisit the potential of binary neural networks and focus on a compelling but
unanswered problem: how can a binary neural network achieve the crucial
accuracy level (e.g., 80%) on ILSVRC-2012 ImageNet? We achieve this goal by
enhancing the optimization process from three complementary perspectives: (1)
We design a novel binary architecture BNext based on a comprehensive study of
binary architectures and their optimization process. (2) We propose a novel
knowledge-distillation technique to alleviate the counter-intuitive overfitting
problem observed when attempting to train extremely accurate binary models. (3)
We analyze the data augmentation pipeline for binary networks and modernize it
with up-to-date techniques from full-precision models. The evaluation results
on ImageNet show that BNext, for the first time, pushes the binary model
accuracy boundary to 80.57% and significantly outperforms all the existing
binary networks. Code and trained models are available at:
this https URL.

通过三个互补的角度：创新设计一个基于全方位研究后的二进制结构 BNext、提出一种新的知识蒸馏技术以缓解训练极精确二进制模型时出现的过拟合问题、并使用来自全精度模型的最新技术现代化二进制网络的数据增强管线，我们让二进制神经网络在 ILSVRC-2012 ImageNet 上达到了至关重要的准确度水平 (例如 80.57%)，并且明显优于所有现有的二进制网络。