Neural architecture search (NAS) finds high performing networks for a given
task. Yet the results of NAS are fairly prosaic; they did not e.g. create a
shift from convolutional structures to transformers. This is not least because
the search spaces in NAS often aren't diverse enough to include such
transformations a priori. Instead, for NAS to provide greater potential for
fundamental design shifts, we need a novel expressive search space design which
is built from more fundamental operations. To this end, we introduce einspace,
a search space based on a parameterised probabilistic context-free grammar. Our
space is versatile, supporting architectures of various sizes and complexities,
while also containing diverse network operations which allow it to model
convolutions, attention components and more. It contains many existing
competitive architectures, and provides flexibility for discovering new ones.
Using this search space, we perform experiments to find novel architectures as
well as improvements on existing ones on the diverse Unseen NAS datasets. We
show that competitive architectures can be obtained by searching from scratch,
and we consistently find large improvements when initialising the search with
strong baselines. We believe that this work is an important advancement towards
a transformative NAS paradigm where search space expressivity and strategic
search initialisation play key roles.

基于参数化概率上下文无关文法构建的 einspace 搜索空间通过在多种大小和复杂性的架构中引入包含卷积、注意力组件等多样网络操作的可变性，为寻找新的架构和改进现有架构提供了灵活性，同时加强了策略性搜索初始化，从而使神经网络结构搜索的表达能力及其初始搜索具有重要作用。

einspace: 从基本操作中搜索神经架构

einspace: Searching for Neural Architectures from Fundamental Operations

We explore efficient neural architecture search methods and show that a
simple yet powerful evolutionary algorithm can discover new architectures with
excellent performance. Our approach combines a novel hierarchical genetic
representation scheme that imitates the modularized design pattern commonly
adopted by human experts, and an expressive search space that supports complex
topologies. Our algorithm efficiently discovers architectures that outperform a
large number of manually designed models for image classification, obtaining
top-1 error of 3.6% on CIFAR-10 and 20.3% when transferred to ImageNet, which
is competitive with the best existing neural architecture search approaches. We
also present results using random search, achieving 0.3% less top-1 accuracy on
CIFAR-10 and 0.1% less on ImageNet whilst reducing the search time from 36
hours down to 1 hour.

本文介绍了一种基于层次遗传表征和复杂拓扑的新颖神经体系结构搜索方法，该方法能够高效地发现优于许多手动设计模型的分类器，同时该方法通过随机搜索得到了更高的准确度并将搜索时间缩短至 1 小时。