Predictor-based Neural Architecture Search (NAS) employs an architecture
performance predictor to improve the sample efficiency. However,
predictor-based NAS suffers from the severe ``cold-start'' problem, since a
large amount of architecture-performance data is required to get a working
predictor. In this paper, we focus on exploiting information in
cheaper-to-obtain performance estimations (i.e., low-fidelity information) to
mitigate the large data requirements of predictor training. Despite the
intuitiveness of this idea, we observe that using inappropriate low-fidelity
information even damages the prediction ability and different search spaces
have different preferences for low-fidelity information types. To solve the
problem and better fuse beneficial information provided by different types of
low-fidelity information, we propose a novel dynamic ensemble predictor
framework that comprises two steps. In the first step, we train different
sub-predictors on different types of available low-fidelity information to
extract beneficial knowledge as low-fidelity experts. In the second step, we
learn a gating network to dynamically output a set of weighting coefficients
conditioned on each input neural architecture, which will be used to combine
the predictions of different low-fidelity experts in a weighted sum. The
overall predictor is optimized on a small set of actual
architecture-performance data to fuse the knowledge from different low-fidelity
experts to make the final prediction. We conduct extensive experiments across
five search spaces with different architecture encoders under various
experimental settings. Our method can easily be incorporated into existing
predictor-based NAS frameworks to discover better architectures.

本研究提出了一个新颖的动态集成预测器框架，包括两个步骤，旨在解决基于预测器的神经架构搜索面临的 “冷启动” 问题，它可以从多种低保真信息中提取有益的知识，通过加权求和的方式融合低给定的架构性能预测信息来训练整个预测器。