In the past years, significant improvements in the field of neural
architecture search(NAS) have been made. However, it is still challenging to
search for efficient networks due to the gap between the searched constraint
and real inference time exists. To search for a high-performance network with
low inference time, several previous works set a computational complexity
constraint for the search algorithm. However, many factors affect the speed of
inference(e.g., FLOPs, MACs). The correlation between a single indicator and
the latency is not strong. Currently, some re-parameterization(Rep) techniques
are proposed to convert multi-branch to single-path architecture which is
inference-friendly. Nevertheless, multi-branch architectures are still
human-defined and inefficient. In this work, we propose a new search space that
is suitable for structural re-parameterization techniques. RepNAS, a one-stage
NAS approach, is present to efficiently search the optimal diverse branch
block(ODBB) for each layer under the branch number constraint. Our experimental
results show the searched ODBB can easily surpass the manual diverse branch
block(DBB) with efficient training.

本文提出一种新的搜索空间并使用 RepNAS 方法在分支数量约束下为每个层搜索最佳的多元分支块 (ODBB) 以实现高性能网络的搜索。实验结果表明，搜索到的 ODBB 可以轻松超越人工设计的多样分支块 (DBB)，并具有有效的训练。

RepNAS：寻找高效的重新参数化模块

RepNAS: Searching for Efficient Re-parameterizing Blocks

We propose a universal building block of Convolutional Neural Network
(ConvNet) to improve the performance without any inference-time costs. The
block is named Diverse Branch Block (DBB), which enhances the representational
capacity of a single convolution by combining diverse branches of different
scales and complexities to enrich the feature space, including sequences of
convolutions, multi-scale convolutions, and average pooling. After training, a
DBB can be equivalently converted into a single conv layer for deployment.
Unlike the advancements of novel ConvNet architectures, DBB complicates the
training-time microstructure while maintaining the macro architecture, so that
it can be used as a drop-in replacement for regular conv layers of any
architecture. In this way, the model can be trained to reach a higher level of
performance and then transformed into the original inference-time structure for
inference. DBB improves ConvNets on image classification (up to 1.9% higher
top-1 accuracy on ImageNet), object detection and semantic segmentation. The
PyTorch code and models are released at
this https URL

本文提出了一种名为 Diverse Branch Block 的通用卷积神经网络模块，通过结合不同尺度和复杂度的多样分支来丰富特征空间，可提高模型性能，在图像分类、目标检测和语义分割方面取得了优异结果。