Neural Architecture Search (NAS) can automatically design architectures for
deep neural networks (DNNs) and has become one of the hottest research topics
in the current machine learning community. However, NAS is often
computationally expensive because a large number of DNNs require to be trained
for obtaining performance during the search process. Performance predictors can
greatly alleviate the prohibitive cost of NAS by directly predicting the
performance of DNNs. However, building satisfactory performance predictors
highly depends on enough trained DNN architectures, which are difficult to
obtain in most scenarios. To solve this critical issue, we propose an effective
DNN architecture augmentation method named GIAug in this paper. Specifically,
we first propose a mechanism based on graph isomorphism, which has the merit of
efficiently generating a factorial of $\boldsymbol n$ (i.e., $\boldsymbol n!$)
diverse annotated architectures upon a single architecture having $\boldsymbol
n$ nodes. In addition, we also design a generic method to encode the
architectures into the form suitable to most prediction models. As a result,
GIAug can be flexibly utilized by various existing performance predictors-based
NAS algorithms. We perform extensive experiments on CIFAR-10 and ImageNet
benchmark datasets on small-, medium- and large-scale search space. The
experiments show that GIAug can significantly enhance the performance of most
state-of-the-art peer predictors. In addition, GIAug can save three magnitude
order of computation cost at most on ImageNet yet with similar performance when
compared with state-of-the-art NAS algorithms.

本文提出了一种名为 GIAug 的有效的 DNN 架构扩增方法，它可以通过基于图同构的机制，非常高效地生成单一架构上多元化的注释架构，并将其编码成适合于大多数预测模型的形式，从而非常灵活地利用各种现有的基于性能预测的 NAS 算法，并且在 CIFAR-10 和 ImageNet 基准数据集上进行的广泛实验表明，GIAug 可以显著提高大多数最先进的同行预测器的性能，并且在与现有的最先进 NAS 算法相比时，可以在 ImageNet 上节省三个数量级的计算成本，同时保持类似的性能。