As one of the emerging challenges in Automated Machine Learning, the
Hardware-aware Neural Architecture Search (HW-NAS) tasks can be treated as
black-box multi-objective optimization problems (MOPs). An important
application of HW-NAS is real-time semantic segmentation, which plays a pivotal
role in autonomous driving scenarios. The HW-NAS for real-time semantic
segmentation inherently needs to balance multiple optimization objectives,
including model accuracy, inference speed, and hardware-specific
considerations. Despite its importance, benchmarks have yet to be developed to
frame such a challenging task as multi-objective optimization. To bridge the
gap, we introduce a tailored streamline to transform the task of HW-NAS for
real-time semantic segmentation into standard MOPs. Building upon the
streamline, we present a benchmark test suite, CitySeg/MOP, comprising fifteen
MOPs derived from the Cityscapes dataset. The CitySeg/MOP test suite is
integrated into the EvoXBench platform to provide seamless interfaces with
various programming languages (e.g., Python and MATLAB) for instant fitness
evaluations. We comprehensively assessed the CitySeg/MOP test suite on various
multi-objective evolutionary algorithms, showcasing its versatility and
practicality. Source codes are available at
this https URL

自动机器学习中的硬件感知神经架构搜索（HW-NAS）任务可视为黑盒多目标优化问题，其中一个重要应用是实时语义分割，通过一项新的基准测试套件 CitySeg/MOP 将 HW-NAS 任务转化为标准多目标优化问题，并在多种多目标进化算法上全面评估了该测试套件的多功能和实用性。

实时语义分割多目标优化基准测试套件

A Multi-objective Optimization Benchmark Test Suite for Real-time  Semantic Segmentation

In the evolutionary multi-objective optimization (EMO) field, the standard
practice is to present the final population of an EMO algorithm as the output.
However, it has been shown that the final population often includes solutions
which are dominated by other solutions generated and discarded in previous
generations. Recently, a new EMO framework has been proposed to solve this
issue by storing all the non-dominated solutions generated during the evolution
in an archive and selecting a subset of solutions from the archive as the
output. The key component in this framework is the subset selection from the
archive which usually stores a large number of candidate solutions. However,
most studies on subset selection focus on small candidate solution sets for
environmental selection. There is no benchmark test suite for large-scale
subset selection. This paper aims to fill this research gap by proposing a
benchmark test suite for subset selection from large candidate solution sets,
and comparing some representative methods using the proposed test suite. The
proposed test suite together with the benchmarking studies provides a baseline
for researchers to understand, use, compare, and develop subset selection
methods in the EMO field.

本文旨在填补演化多目标优化领域中子集选择策略研究缺口，提出基于大型候选解集的子集选择基准测试套件，并使用该测试套件比较一些典型方法。