Recent advances show that Neural Architectural Search (NAS) method is able to
find state-of-the-art image classification deep architectures. In this paper,
we consider the one-shot NAS problem for resource constrained applications.
This problem is of great interest because it is critical to choose different
architectures according to task complexity when the resource is constrained.
Previous techniques are either too slow for one-shot learning or does not take
the resource constraint into consideration. In this paper, we propose the
resource constrained differentiable architecture search (RC-DARTS) method to
learn architectures that are significantly smaller and faster while achieving
comparable accuracy. Specifically, we propose to formulate the RC-DARTS task as
a constrained optimization problem by adding the resource constraint. An
iterative projection method is proposed to solve the given constrained
optimization problem. We also propose a multi-level search strategy to enable
layers at different depths to adaptively learn different types of neural
architectures. Through extensive experiments on the Cifar10 and ImageNet
datasets, we show that the RC-DARTS method learns lightweight neural
architectures which have smaller model size and lower computational complexity
while achieving comparable or better performances than the state-of-the-art
methods.

本文提出了资源限制下的可微架构搜索方法 (RC-DARTS)，以较小的模型尺寸和计算复杂度实现与最先进方法相当的性能，并通过对 Cifar10 和 ImageNet 数据集的实验进行了验证。

RC-DARTS: 资源受限的可微分架构搜索

RC-DARTS: Resource Constrained Differentiable Architecture Search

Recent breakthroughs in Neural Architectural Search (NAS) have achieved
state-of-the-art performance in many tasks such as image classification and
language understanding. However, most existing works only optimize for model
accuracy and largely ignore other important factors imposed by the underlying
hardware and devices, such as latency and energy, when making inference. In
this paper, we first introduce the problem of NAS and provide a survey on
recent works. Then we deep dive into two recent advancements on extending NAS
into multiple-objective frameworks: MONAS and DPP-Net. Both MONAS and DPP-Net
are capable of optimizing accuracy and other objectives imposed by devices,
searching for neural architectures that can be best deployed on a wide spectrum
of devices: from embedded systems and mobile devices to workstations.
Experimental results are poised to show that architectures found by MONAS and
DPP-Net achieves Pareto optimality w.r.t the given objectives for various
devices.

本文介绍了神经架构搜索的问题，并调研了最近的研究成果。然后，我们深入探讨了两项对 NAS 进行多目标框架扩展的最新进展：MONAS 和 DPP-Net。MONAS 和 DPP-Net 都能够优化精度和设备带入的其他目标，寻找可以在广泛设备上最好部署的神经架构。实验结果显示，MONAS 和 DPP-Net 找到的架构在各种设备上均达到了 Pareto 最优性。

面向 Pareto - 最优设备感知神经网络架构的搜索

Searching Toward Pareto-Optimal Device-Aware Neural Architectures

Recent breakthroughs in Neural Architectural Search (NAS) have achieved
state-of-the-art performances in applications such as image classification and
language modeling. However, these techniques typically ignore device-related
objectives such as inference time, memory usage, and power consumption.
Optimizing neural architecture for device-related objectives is immensely
crucial for deploying deep networks on portable devices with limited computing
resources. We propose DPP-Net: Device-aware Progressive Search for
Pareto-optimal Neural Architectures, optimizing for both device-related (e.g.,
inference time and memory usage) and device-agnostic (e.g., accuracy and model
size) objectives. DPP-Net employs a compact search space inspired by current
state-of-the-art mobile CNNs, and further improves search efficiency by
adopting progressive search (Liu et al. 2017). Experimental results on CIFAR-10
are poised to demonstrate the effectiveness of Pareto-optimal networks found by
DPP-Net, for three different devices: (1) a workstation with Titan X GPU, (2)
NVIDIA Jetson TX1 embedded system, and (3) mobile phone with ARM Cortex-A53.
Compared to CondenseNet and NASNet (Mobile), DPP-Net achieves better
performances: higher accuracy and shorter inference time on various devices.
Additional experimental results show that models found by DPP-Net also achieve
considerably-good performance on ImageNet as well.

提出了一种名为 DPP-Net 的优化神经体系结构架构的方法，能够同时考虑设备相关和设备不相关目标，通过实验结果证明其在各种不同设备上比其他相似方法更有效。