We propose a neural architecture search (NAS) algorithm, Petridish, to
iteratively add shortcut connections to existing network layers. The added
shortcut connections effectively perform gradient boosting on the augmented
layers. The proposed algorithm is motivated by the feature selection algorithm
forward stage-wise linear regression, since we consider NAS as a generalization
of feature selection for regression, where NAS selects shortcuts among layers
instead of selecting features. In order to reduce the number of trials of
possible connection combinations, we train jointly all possible connections at
each stage of growth while leveraging feature selection techniques to choose a
subset of them. We experimentally show this process to be an efficient forward
architecture search algorithm that can find competitive models using few GPU
days in both the search space of repeatable network modules (cell-search) and
the space of general networks (macro-search). Petridish is particularly
well-suited for warm-starting from existing models crucial for
lifelong-learning scenarios.

本文提出了一个神经架构搜索算法 Petridish，用于随着现有网络层的迭代添加快捷连接。其利用加入的快捷连接在增强的层上有效地执行梯度增强。文中所提出的算法受到特征选择算法前向逐步线性回归的启发，因为我们将 NAS 视为回归的特征选择的泛化，即 NAS 在层之间选择快捷方式而不是选择特征。为了减少可能的连接组合的试验次数，我们共同训练每个增长阶段的所有可能连接，同时利用特征选择技术选择其中的子集。我们在研究中发现，Petridish 是一种有效率的前向架构搜索算法，在重复网络模块（cell-search）和通用网络（macro-search）的搜索空间中，使用了很少的 GPU 天数可以找到竞争模型。 特别适用于从现有模型启动的生命周期学习场景。