Neural Architecture Search (NAS) methods have shown to output networks that
largely outperform human-designed networks. However, conventional NAS methods
have mostly tackled the single dataset scenario, incuring in a large
computational cost as the procedure has to be run from scratch for every new
dataset. In this work, we focus on predictor-based algorithms and propose a
simple and efficient way of improving their prediction performance when dealing
with data distribution shifts. We exploit the Kronecker-product on the randomly
wired search-space and create a small NAS benchmark composed of networks
trained over four different datasets. To improve the generalization abilities,
we propose GRASP-GCN, a ranking Graph Convolutional Network that takes as
additional input the shape of the layers of the neural networks. GRASP-GCN is
trained with the not-at-convergence accuracies, and improves the
state-of-the-art of 3.3 % for Cifar-10 and increasing moreover the
generalization abilities under data distribution shift.

通过使用 Kronecker 积在随机连接搜索空间上，我们提出了一种简单高效的方法来改善基于预测器的算法在处理数据分布变化时的预测性能，我们创建了一个包含四个不同数据集上训练的网络的小型 NAS 基准测试，进一步提高了 GRASP-GCN 的泛化能力并改进了 Cifar-10 的准确度达到 3.3%。

GRASP-GCN：在分布偏移下进行神经架构搜索的图形形状优化

GRASP-GCN: Graph-Shape Prioritization for Neural Architecture Search  under Distribution Shifts

Predictor-based algorithms have achieved remarkable performance in the Neural
Architecture Search (NAS) tasks. However, these methods suffer from high
computation costs, as training the performance predictor usually requires
training and evaluating hundreds of architectures from scratch. Previous works
along this line mainly focus on reducing the number of architectures required
to fit the predictor. In this work, we tackle this challenge from a different
perspective - improve search efficiency by cutting down the computation budget
of architecture training. We propose NOn-uniform Successive Halving (NOSH), a
hierarchical scheduling algorithm that terminates the training of
underperforming architectures early to avoid wasting budget. To effectively
leverage the non-uniform supervision signals produced by NOSH, we formulate
predictor-based architecture search as learning to rank with pairwise
comparisons. The resulting method - RANK-NOSH, reduces the search budget by ~5x
while achieving competitive or even better performance than previous
state-of-the-art predictor-based methods on various spaces and datasets.

本文提出了一种利用 NOn-uniform Successive Halving 策略的 Predictor-based algorithms，通过提高搜索效率来减少架构训练的计算预算，并使用学习排序的方法来利用 NOn-uniform Successive Halving 的非均匀监督信号，在各个空间和数据集上实现了与前沿方法相当或更好的性能。