Though numerous solvers have been proposed for the MaxSAT problem, and the
benchmark environment such as MaxSAT Evaluations provides a platform for the
comparison of the state-of-the-art solvers, existing assessments were usually
evaluated based on the quality, e.g., fitness, of the best-found solutions
obtained within a given running time budget. However, concerning solely the
final obtained solutions regarding specific time budgets may restrict us from
comprehending the behavior of the solvers along the convergence process. This
paper demonstrates that Empirical Cumulative Distribution Functions can be used
to compare MaxSAT local search solvers' anytime performance across multiple
problem instances and various time budgets. The assessment reveals distinctions
in solvers' performance and displays that the (dis)advantages of solvers adjust
along different running times. This work also exhibits that the quantitative
and high variance assessment of anytime performance can guide machines, i.e.,
automatic configurators, to search for better parameter settings. Our
experimental results show that the hyperparameter optimization tool, i.e.,
SMAC, generally achieves better parameter settings of local search when using
the anytime performance as the cost function, compared to using the fitness of
the best-found solutions.

本文介绍了一种使用经验累积分布函数来比较 MaxSAT 局部搜索求解器在多个问题实例和不同时间预算下的任意时刻性能的方法，实证评估结果显示求解器的性能存在差异，并且在不同的运行时间下求解器的优势和劣势会有所调整，同时，这项工作还证明了以任意时刻性能作为成本函数进行超参数优化的方法能够得到更好的局部搜索参数设置。