Neural Architecture Search (NAS) has become a popular method for discovering
effective model architectures, especially for target hardware. As such, NAS
methods that find optimal architectures under constraints are essential. In our
paper, we propose LayerNAS to address the challenge of multi-objective NAS by
transforming it into a combinatorial optimization problem, which effectively
constrains the search complexity to be polynomial.
For a model architecture with $L$ layers, we perform layerwise-search for
each layer, selecting from a set of search options $\mathbb{S}$. LayerNAS
groups model candidates based on one objective, such as model size or latency,
and searches for the optimal model based on another objective, thereby
splitting the cost and reward elements of the search. This approach limits the
search complexity to $ O(H \cdot |\mathbb{S}| \cdot L) $, where $H$ is a
constant set in LayerNAS.
Our experiments show that LayerNAS is able to consistently discover superior
models across a variety of search spaces in comparison to strong baselines,
including search spaces derived from NATS-Bench, MobileNetV2 and MobileNetV3.

LayerNAS 是一种利用组合优化来解决多目标神经网络架构搜索的方法，能够在多种搜索空间中发现优秀的模型候选项。