Weight sharing is a fundamental concept in neural architecture search (NAS),
enabling gradient-based methods to explore cell-based architecture spaces
significantly faster than traditional blackbox approaches. In parallel, weight
\emph{entanglement} has emerged as a technique for intricate parameter sharing
among architectures within macro-level search spaces. %However, the macro
structure of such spaces poses compatibility challenges for gradient-based NAS
methods. %As a result, blackbox optimization methods have been commonly
employed, particularly in conjunction with supernet training, to maintain
search efficiency. %Due to the inherent differences in the structure of these
search spaces, these Since weight-entanglement poses compatibility challenges
for gradient-based NAS methods, these two paradigms have largely developed
independently in parallel sub-communities. This paper aims to bridge the gap
between these sub-communities by proposing a novel scheme to adapt
gradient-based methods for weight-entangled spaces. This enables us to conduct
an in-depth comparative assessment and analysis of the performance of
gradient-based NAS in weight-entangled search spaces. Our findings reveal that
this integration of weight-entanglement and gradient-based NAS brings forth the
various benefits of gradient-based methods (enhanced performance, improved
supernet training properties and superior any-time performance), while
preserving the memory efficiency of weight-entangled spaces. The code for our
work is openly accessible
\href{this https URL}{here}

本文旨在通过提出一种新的方案，将基于梯度的方法适应于权重相互关联的空间，从而弥合这两个子社区之间的差距，并深入比较评估梯度优化方法在权重相互关联的搜索空间中的性能，结果发现这种权重相互关联和基于梯度的 NAS 的整合带来了增强性能、改善超网训练性质以及出色的任意时间性能等各种优势，同时保持了权重相互关联空间的内存效率。