Most existing neural architecture search (NAS) algorithms are dedicated to
and evaluated by the downstream tasks, e.g., image classification in computer
vision. However, extensive experiments have shown that, prominent neural
architectures, such as ResNet in computer vision and LSTM in natural language
processing, are generally good at extracting patterns from the input data and
perform well on different downstream tasks. In this paper, we attempt to answer
two fundamental questions related to NAS. (1) Is it necessary to use the
performance of specific downstream tasks to evaluate and search for good neural
architectures? (2) Can we perform NAS effectively and efficiently while being
agnostic to the downstream tasks? To answer these questions, we propose a novel
and generic NAS framework, termed Generic NAS (GenNAS). GenNAS does not use
task-specific labels but instead adopts regression on a set of manually
designed synthetic signal bases for architecture evaluation. Such a
self-supervised regression task can effectively evaluate the intrinsic power of
an architecture to capture and transform the input signal patterns, and allow
more sufficient usage of training samples. Extensive experiments across 13 CNN
search spaces and one NLP space demonstrate the remarkable efficiency of GenNAS
using regression, in terms of both evaluating the neural architectures
(quantified by the ranking correlation Spearman's rho between the approximated
performances and the downstream task performances) and the convergence speed
for training (within a few seconds).

本文提出了一种名为 “Generic NAS” 的通用 NAS 框架，该框架采用自监督回归任务来评估架构的内在能力，并且在 13 个 CNN 搜索空间和一个 NLP 空间上进行了广泛的实验，展示了 GenNAS 的显着高效性（通过 Spearman 的 rho 度量的排名相关性和训练的收敛速度都得到了提升），并回答了两个与 NAS 相关的基本问题。

通过回归实现通用神经体系结构搜索

Generic Neural Architecture Search via Regression

In this paper, we propose a semiparametric approach, named nonparanormal
skeptic, for efficiently and robustly estimating high dimensional undirected
graphical models. To achieve modeling flexibility, we consider Gaussian Copula
graphical models (or the nonparanormal) as proposed by Liu et al. (2009). To
achieve estimation robustness, we exploit nonparametric rank-based correlation
coefficient estimators, including Spearman's rho and Kendall's tau. In high
dimensional settings, we prove that the nonparanormal skeptic achieves the
optimal parametric rate of convergence in both graph and parameter estimation.
This celebrating result suggests that the Gaussian copula graphical models can
be used as a safe replacement of the popular Gaussian graphical models, even
when the data are truly Gaussian. Besides theoretical analysis, we also conduct
thorough numerical simulations to compare different estimators for their graph
recovery performance under both ideal and noisy settings. The proposed methods
are then applied on a large-scale genomic dataset to illustrate their empirical
usefulness. The R language software package huge implementing the proposed
methods is available on the Comprehensive R Archive Network: this http URL
r-project.org/.

本文提出了一种名为 “非参数鸟瞰者” 的半参数方法，用于高效且鲁棒地估计高维无向图模型；该方法采用了 高斯 Copula 图模型 以及 Spearman's rho 和 Kendall's tau 这些非参数秩相关系数估计器，以实现建模的灵活性和估计的鲁棒性。