Neural architecture search (NAS) has emerged as a promising avenue for
automatically designing task-specific neural networks. Existing NAS approaches
require one complete search for each deployment specification of hardware or
objective. This is a computationally impractical endeavor given the potentially
large number of application scenarios. In this paper, we propose Neural
Architecture Transfer (NAT) to overcome this limitation. NAT is designed to
efficiently generate task-specific custom models that are competitive under
multiple conflicting objectives. To realize this goal we learn task-specific
supernets from which specialized subnets can be sampled without any additional
training. The key to our approach is an integrated online transfer learning and
many-objective evolutionary search procedure. A pre-trained supernet is
iteratively adapted while simultaneously searching for task-specific subnets.
We demonstrate the efficacy of NAT on 11 benchmark image classification tasks
ranging from large-scale multi-class to small-scale fine-grained datasets. In
all cases, including ImageNet, NATNets improve upon the state-of-the-art under
mobile settings ($\leq$ 600M Multiply-Adds). Surprisingly, small-scale
fine-grained datasets benefit the most from NAT. At the same time, the
architecture search and transfer is orders of magnitude more efficient than
existing NAS methods. Overall, the experimental evaluation indicates that,
across diverse image classification tasks and computational objectives, NAT is
an appreciably more effective alternative to conventional transfer learning of
fine-tuning weights of an existing network architecture learned on standard
datasets. Code is available at
this https URL

本文介绍了一种基于神经架构搜索（NAS）的新方法，称为神经架构传递（NAT），该方法可将已训练的超网络集成到在线迁移学习和多目标进化搜索流程中，以快速生成适用于多个应用场景的任务特定定制模型，并在 11 项基准图像分类任务中证明了其有效性。