Differentiable Neural Architecture Search (DNAS) has demonstrated great
success in designing state-of-the-art, efficient neural networks. However,
DARTS-based DNAS's search space is small when compared to other search
methods', since all candidate network layers must be explicitly instantiated in
memory. To address this bottleneck, we propose a memory and computationally
efficient DNAS variant: DMaskingNAS. This algorithm expands the search space by
up to $10^{14}\times$ over conventional DNAS, supporting searches over spatial
and channel dimensions that are otherwise prohibitively expensive: input
resolution and number of filters. We propose a masking mechanism for feature
map reuse, so that memory and computational costs stay nearly constant as the
search space expands. Furthermore, we employ effective shape propagation to
maximize per-FLOP or per-parameter accuracy. The searched FBNetV2s yield
state-of-the-art performance when compared with all previous architectures.
With up to 421$\times$ less search cost, DMaskingNAS finds models with 0.9%
higher accuracy, 15% fewer FLOPs than MobileNetV3-Small; and with similar
accuracy but 20% fewer FLOPs than Efficient-B0. Furthermore, our FBNetV2
outperforms MobileNetV3 by 2.6% in accuracy, with equivalent model size.
FBNetV2 models are open-sourced at
this https URL

DMaskingNAS 是一种内存和计算效率高的可微神经网络搜索算法，在搜索空间方面比传统算法扩展了 10^14 倍，并利用了特征图重用的掩膜机制以及有效的形状传播，以达到与现有架构相比最优的性能表现。