One-shot Neural Architecture Search (NAS) has been widely used to discover
architectures due to its efficiency. However, previous studies reveal that
one-shot performance estimations of architectures might not be well correlated
with their performances in stand-alone training because of the excessive
sharing of operation parameters (i.e., large sharing extent) between
architectures. Thus, recent methods construct even more over-parameterized
supernets to reduce the sharing extent. But these improved methods introduce a
large number of extra parameters and thus cause an undesirable trade-off
between the training costs and the ranking quality. To alleviate the above
issues, we propose to apply Curriculum Learning On Sharing Extent (CLOSE) to
train the supernet both efficiently and effectively. Specifically, we train the
supernet with a large sharing extent (an easier curriculum) at the beginning
and gradually decrease the sharing extent of the supernet (a harder
curriculum). To support this training strategy, we design a novel supernet
(CLOSENet) that decouples the parameters from operations to realize a flexible
sharing scheme and adjustable sharing extent. Extensive experiments demonstrate
that CLOSE can obtain a better ranking quality across different computational
budget constraints than other one-shot supernets, and is able to discover
superior architectures when combined with various search strategies. Code is
available at this https URL

本研究提出一种名为 CLOSE 的课程学习方法，其使用名为 CLOSENet 的新型 Supernet 来有效地训练神经架构搜索，并在结合各种搜索策略时发现具有优越性能的架构。

CLOSE: 基于共享程度的课程学习方法，致力于改进一次性神经网络搜索技术

CLOSE: Curriculum Learning On the Sharing Extent Towards Better One-shot  NAS

Neural architecture search (NAS) has emerged as a promising avenue for
automatically designing task-specific neural networks. Existing NAS approaches
require one complete search for each deployment specification of hardware or
objective. This is a computationally impractical endeavor given the potentially
large number of application scenarios. In this paper, we propose Neural
Architecture Transfer (NAT) to overcome this limitation. NAT is designed to
efficiently generate task-specific custom models that are competitive under
multiple conflicting objectives. To realize this goal we learn task-specific
supernets from which specialized subnets can be sampled without any additional
training. The key to our approach is an integrated online transfer learning and
many-objective evolutionary search procedure. A pre-trained supernet is
iteratively adapted while simultaneously searching for task-specific subnets.
We demonstrate the efficacy of NAT on 11 benchmark image classification tasks
ranging from large-scale multi-class to small-scale fine-grained datasets. In
all cases, including ImageNet, NATNets improve upon the state-of-the-art under
mobile settings ($\leq$ 600M Multiply-Adds). Surprisingly, small-scale
fine-grained datasets benefit the most from NAT. At the same time, the
architecture search and transfer is orders of magnitude more efficient than
existing NAS methods. Overall, the experimental evaluation indicates that,
across diverse image classification tasks and computational objectives, NAT is
an appreciably more effective alternative to conventional transfer learning of
fine-tuning weights of an existing network architecture learned on standard
datasets. Code is available at
this https URL

本文介绍了一种基于神经架构搜索（NAS）的新方法，称为神经架构传递（NAT），该方法可将已训练的超网络集成到在线迁移学习和多目标进化搜索流程中，以快速生成适用于多个应用场景的任务特定定制模型，并在 11 项基准图像分类任务中证明了其有效性。