Backbone architectures of most binary networks are well-known floating point
(FP) architectures such as the ResNet family. Questioning that the
architectures designed for FP networks might not be the best for binary
networks, we propose to search architectures for binary networks (BNAS) by
defining a new search space for binary architectures and a novel search
objective. Specifically, based on the cell based search method, we define the
new search space of binary layer types, design a new cell template, and
rediscover the utility of and propose to use the Zeroise layer instead of using
it as a placeholder. The novel search objective diversifies early search to
learn better performing binary architectures. We show that our method searches
architectures with stable training curves despite the quantization error
inherent in binary networks. Quantitative analyses demonstrate that our
searched architectures outperform the architectures used in state-of-the-art
binary networks and outperform or perform on par with state-of-the-art binary
networks that employ various techniques other than architectural changes. In
addition, we further propose improvements to the training scheme of our
searched architectures. With the new training scheme for our searched
architectures, we achieve the state-of-the-art performance by binary networks
by outperforming all previous methods by non-trivial margins.

该论文提出了一种基于单元的搜索方法，通过为二进制网络定义新的搜索空间和新的搜索目标，重新定义二进制层类型，设计一种新的单元模板，提出使用 Zeroise 层来代替占位符，并进一步提出了改进我们搜索到的体系结构的训练方案。经过定量分析，实验结果表明，与采用其他技术的现有二进制网络相比，使用该方法搜索的体系结构性能更好且训练曲线更加稳定，通过新的训练方案进一步提升了性能并在二进制网络中实现了领先的表现。

BNAS v2: 二进制网络结构学习与实证优化

BNAS v2: Learning Architectures for Binary Networks with Empirical  Improvements

Backbone architectures of most binary networks are well-known floating point
architectures such as the ResNet family. Questioning that the architectures
designed for floating point networks would not be the best for binary networks,
we propose to search architectures for binary networks (BNAS) by defining a new
search space for binary architectures and a novel search objective.
Specifically, based on the cell based search method, we define the new search
space of binary layer types, design a new cell template, and rediscover the
utility of and propose to use the Zeroise layer instead of using it as a
placeholder. The novel search objective diversifies early search to learn
better performing binary architectures. We show that our proposed method
searches architectures with stable training curves despite the quantization
error inherent in binary networks. Quantitative analyses demonstrate that our
searched architectures outperform the architectures used in state-of-the-art
binary networks and outperform or perform on par with state-of-the-art binary
networks that employ various techniques other than architectural changes.

本研究提出了一种针对二进制网络的新型搜索体系结构，并定义了二进制层类型的新搜索空间和新的单元模板。使用零化层的新搜索目标使得搜索优良性能的二进制体系结构变得更加多样化，并在量化误差固有的二进制网络中实现稳定的训练曲线。本研究的搜索结果表明其超越了现有二进制网络的架构，并且在采用各种其他技术而非架构改变的状态下，性能明显提高。