Neural architecture search (NAS) automates the design of deep neural
networks. One of the main challenges in searching complex and non-continuous
architectures is to compare the similarity of networks that the conventional
Euclidean metric may fail to capture. Optimal transport (OT) is resilient to
such complex structure by considering the minimal cost for transporting a
network into another. However, the OT is generally not negative definite which
may limit its ability to build the positive-definite kernels required in many
kernel-dependent frameworks. Building upon tree-Wasserstein (TW), which is a
negative definite variant of OT, we develop a novel discrepancy for neural
architectures, and demonstrate it within a Gaussian process surrogate model for
the sequential NAS settings. Furthermore, we derive a novel parallel NAS, using
quality k-determinantal point process on the GP posterior, to select diverse
and high-performing architectures from a discrete set of candidates.
Empirically, we demonstrate that our TW-based approaches outperform other
baselines in both sequential and parallel NAS.

本研究使用基于 Tree-Wasserstein 的新的差异来研究神经结构，并在高斯过程替代模型中演示了它，最后通过使用基于 GP 后验的质量 k - 行列式点过程导出了新的并行 NAS，证明了 TW-based 方法在顺序和并行 NAS 中优于其他基线。