Relative Validity Indices (RVIs) such as the Silhouette Width Criterion,
Calinski-Harabasz and Davie's Bouldin indices are the most popular tools for
evaluating and optimising applications of clustering. Their ability to rank
collections of candidate partitions has been used to guide the selection of the
number of clusters, and to compare partitions from different clustering
algorithms. Beyond these more conventional tasks, many examples can be found in
the literature where RVIs have been used to compare and select other aspects of
clustering approaches such as data normalisation procedures, data
representation methods, and distance measures. The authors are not aware of any
studies that have attempted to establish the suitability of RVIs for such
comparisons. Moreover, given the impact of these aspects on pairwise
similarities, it is not even immediately obvious how RVIs should be implemented
when comparing these aspects. In this study, we conducted experiments with
seven common RVIs on over 2.7 million clustering partitions for both synthetic
and real-world datasets, encompassing feature-vector and time-series data. Our
findings suggest that RVIs are not well-suited to these unconventional tasks,
and that conclusions drawn from such applications may be misleading. It is
recommended that normalisation procedures, representation methods, and distance
measures instead be selected using external validation on high quality labelled
datasets or carefully designed outcome-oriented objective criteria, both of
which should be informed by relevant domain knowledge and clustering aims.

这项研究通过对超过 270 万个聚类分区进行实验，发现 RVIs 在非传统任务上并不适用，对此类应用得出的结论可能是误导性的，因此建议使用外部验证和相关领域知识来选择正规化程序、表示方法和距离度量。