In this paper, we propose a Customizable Architecture Search (CAS) approach
to automatically generate a network architecture for semantic image
segmentation. The generated network consists of a sequence of stacked
computation cells. A computation cell is represented as a directed acyclic
graph, in which each node is a hidden representation (i.e., feature map) and
each edge is associated with an operation (e.g., convolution and pooling),
which transforms data to a new layer. During the training, the CAS algorithm
explores the search space for an optimized computation cell to build a network.
The cells of the same type share one architecture but with different weights.
In real applications, however, an optimization may need to be conducted under
some constraints such as GPU time and model size. To this end, a cost
corresponding to the constraint will be assigned to each operation. When an
operation is selected during the search, its associated cost will be added to
the objective. As a result, our CAS is able to search an optimized architecture
with customized constraints. The approach has been thoroughly evaluated on
Cityscapes and CamVid datasets, and demonstrates superior performance over
several state-of-the-art techniques. More remarkably, our CAS achieves 72.3%
mIoU on the Cityscapes dataset with speed of 108 FPS on an Nvidia TitanXp GPU.

本文提出了一种可定制的架构搜索方法，以实现自动生成用于语义图像分割的网络架构，操作被赋予成本，并通过探索搜索空间来构建最优网络架构以满足自定义约束，该方法在 Cityscapes 和 CamVid 数据集上表现优秀，并在 Nvidia TitanXp GPU 上达到了 108 FPS 和 72.3％的性能表现。

面向语义分割的可定制化架构搜索

Customizable Architecture Search for Semantic Segmentation

Neural Machine Translation (NMT) has shown remarkable progress over the past
few years with production systems now being deployed to end-users. One major
drawback of current architectures is that they are expensive to train,
typically requiring days to weeks of GPU time to converge. This makes
exhaustive hyperparameter search, as is commonly done with other neural network
architectures, prohibitively expensive. In this work, we present the first
large-scale analysis of NMT architecture hyperparameters. We report empirical
results and variance numbers for several hundred experimental runs,
corresponding to over 250,000 GPU hours on the standard WMT English to German
translation task. Our experiments lead to novel insights and practical advice
for building and extending NMT architectures. As part of this contribution, we
release an open-source NMT framework that enables researchers to easily
experiment with novel techniques and reproduce state of the art results.

本研究对神经机器翻译（NMT）架构的超参数进行了首次大规模分析，并报告了标准 WMT 英德翻译任务上数百个实验结果和方差数，相当于超过 25 万 GPU 小时。通过这些实验，得出了新的见解和建议，建立了一个开源 NMT 框架，使研究人员可以轻松地实验新技术并重现最先进的结果。