Predictor-based methods have substantially enhanced Neural Architecture
Search (NAS) optimization. The efficacy of these predictors is largely
influenced by the method of encoding neural network architectures. While
traditional encodings used an adjacency matrix describing the graph structure
of a neural network, novel encodings embrace a variety of approaches from
unsupervised pretraining of latent representations to vectors of zero-cost
proxies. In this paper, we categorize and investigate neural encodings from
three main types: structural, learned, and score-based. Furthermore, we extend
these encodings and introduce \textit{unified encodings}, that extend NAS
predictors to multiple search spaces. Our analysis draws from experiments
conducted on over 1.5 million neural network architectures on NAS spaces such
as NASBench-101 (NB101), NB201, NB301, Network Design Spaces (NDS), and
TransNASBench-101. Building on our study, we present our predictor
\textbf{FLAN}: \textbf{Fl}ow \textbf{A}ttention for \textbf{N}AS. FLAN
integrates critical insights on predictor design, transfer learning, and
\textit{unified encodings} to enable more than an order of magnitude cost
reduction for training NAS accuracy predictors. Our implementation and
encodings for all neural networks are open-sourced at
\href{https://github.com/abdelfattah-lab/flan_nas}{this https URL}.

利用预测器进行神经结构搜索（NAS）优化的方法大大提高了其有效性，本文对神经网络架构的编码方法进行了分类和研究，包括结构编码、学习编码和基于分数的编码，并引入了统一编码方法，进一步扩展了 NAS 预测器的应用范围。此研究基于在 NASBench-101（NB101）、NB201、NB301、Network Design Spaces（NDS）和 TransNASBench-101 等 NAS 领域超过 150 万个神经网络架构进行的实验证明，我们提出的预测器 FLAN（流动关注）通过融合预测器设计、迁移学习和统一编码，在训练 NAS 准确性预测器时能大幅度降低成本。所有神经网络的实现和编码方式均已在 https://github.com/abdelfattah-lab/flan_nas 开源。