Recent breakthroughs of Neural Architecture Search (NAS) extend the field's
research scope towards a broader range of vision tasks and more diversified
search spaces. While existing NAS methods mostly design architectures on a
single task, algorithms that look beyond single-task search are surging to
pursue a more efficient and universal solution across various tasks. Many of
them leverage transfer learning and seek to preserve, reuse, and refine network
design knowledge to achieve higher efficiency in future tasks. However, the
enormous computational cost and experiment complexity of cross-task NAS are
imposing barriers for valuable research in this direction. Existing NAS
benchmarks all focus on one type of vision task, i.e., classification. In this
work, we propose TransNAS-Bench-101, a benchmark dataset containing network
performance across seven tasks, covering classification, regression,
pixel-level prediction, and self-supervised tasks. This diversity provides
opportunities to transfer NAS methods among tasks and allows for more complex
transfer schemes to evolve. We explore two fundamentally different types of
search space: cell-level search space and macro-level search space. With 7,352
backbones evaluated on seven tasks, 51,464 trained models with detailed
training information are provided. With TransNAS-Bench-101, we hope to
encourage the advent of exceptional NAS algorithms that raise cross-task search
efficiency and generalizability to the next level. Our dataset file will be
available at Mindspore, VEGA.

本文提出了 TransNAS-Bench-101，这是一个包含七个视觉任务的基准数据集，旨在鼓励研究跨任务搜索效率和通用性的顶尖神经结构搜索算法，同时提供两种不同类型的搜索空间。

TransNAS-Bench-101: 改进跨任务神经架构搜索的可转移性和泛化性

TransNAS-Bench-101: Improving Transferability and Generalizability of  Cross-Task Neural Architecture Search

Differentiable architecture search (DAS) has made great progress in searching
for high-performance architectures with reduced computational cost. However,
DAS-based methods mainly focus on searching for a repeatable cell structure,
which is then stacked sequentially in multiple stages to form the networks.
This configuration significantly reduces the search space, and ignores the
importance of connections between the cells. To overcome this limitation, in
this paper, we propose a Hierarchical Differentiable Architecture Search
(H-DAS) that performs architecture search both at the cell level and at the
stage level. Specifically, the cell-level search space is relaxed so that the
networks can learn stage-specific cell structures. For the stage-level search,
we systematically study the architectures of stages, including the number of
cells in each stage and the connections between the cells. Based on insightful
observations, we design several search rules and losses, and mange to search
for better stage-level architectures. Such hierarchical search space greatly
improves the performance of the networks without introducing expensive search
cost. Extensive experiments on CIFAR10 and ImageNet demonstrate the
effectiveness of the proposed H-DAS. Moreover, the searched stage-level
architectures can be combined with the cell structures searched by existing DAS
methods to further boost the performance. Code is available at:
this https URL

该研究提出了层级可微架构搜索 (Hierarchical Differentiable Architecture Search, H-DAS) 方法，旨在在单元层面和阶段层面上进行架构搜索，通过放宽单元层面的搜索空间，设计阶段层面的搜索规则和损失，有效提升了网络性能，实验证明该方法的有效性。