Pareto front profiling in multi-objective optimization (MOO), i.e. finding a
diverse set of Pareto optimal solutions, is challenging, especially with
expensive objectives like neural network training. Typically, in MOO neural
architecture search (NAS), we aim to balance performance and hardware metrics
across devices. Prior NAS approaches simplify this task by incorporating
hardware constraints into the objective function, but profiling the Pareto
front necessitates a search for each constraint. In this work, we propose a
novel NAS algorithm that encodes user preferences for the trade-off between
performance and hardware metrics, and yields representative and diverse
architectures across multiple devices in just one search run. To this end, we
parameterize the joint architectural distribution across devices and multiple
objectives via a hypernetwork that can be conditioned on hardware features and
preference vectors, enabling zero-shot transferability to new devices.
Extensive experiments with up to 19 hardware devices and 3 objectives showcase
the effectiveness and scalability of our method. Finally, we show that, without
additional costs, our method outperforms existing MOO NAS methods across
qualitatively different search spaces and datasets, including MobileNetV3 on
ImageNet-1k and a Transformer space on machine translation.

我们提出了一种新的神经架构搜索算法，通过超网络对设备和多个目标进行参数化和条件化，实现了性能和硬件度量之间的权衡，并在一次搜索中获得多设备上的具有代表性和多样性的架构。在高达 19 个硬件设备和 3 个目标的大量实验证明了方法的有效性和可扩展性。最后，我们证明在不增加额外成本的情况下，我们的方法在不同搜索空间和数据集上优于现有的多目标优化神经架构搜索方法，包括 ImageNet-1k 上的 MobileNetV3 和机器翻译中的 Transformer 空间。