Deploying deep learning models requires taking into consideration neural
network metrics such as model size, inference latency, and #FLOPs, aside from
inference accuracy. This results in deep learning model designers leveraging
multi-objective optimization to design effective deep neural networks in
multiple criteria. However, applying multi-objective optimizations to neural
architecture search (NAS) is nontrivial because NAS tasks usually have a huge
search space, along with a non-negligible searching cost. This requires
effective multi-objective search algorithms to alleviate the GPU costs. In this
work, we implement a novel multi-objectives optimizer based on a recently
proposed meta-algorithm called LaMOO on NAS tasks. In a nutshell, LaMOO
speedups the search process by learning a model from observed samples to
partition the search space and then focusing on promising regions likely to
contain a subset of the Pareto frontier. Using LaMOO, we observe an improvement
of more than 200% sample efficiency compared to Bayesian optimization and
evolutionary-based multi-objective optimizers on different NAS datasets. For
example, when combined with LaMOO, qEHVI achieves a 225% improvement in sample
efficiency compared to using qEHVI alone in NasBench201. For real-world tasks,
LaMOO achieves 97.36% accuracy with only 1.62M #Params on CIFAR10 in only 600
search samples. On ImageNet, our large model reaches 80.4% top-1 accuracy with
only 522M #FLOPs.

使用 LaMOO 优化算法在神经架构搜索任务中取得了超过 200% 的样本效率提升，并在 CIFAR10 数据集上以仅 600 个搜索样本实现了 97.36% 的准确率和仅 1.62M 参数，以及在 ImageNet 数据集上仅使用 522M 浮点操作数实现了 80.4% 的 top-1 准确率。