Relative Validity Indices (RVIs) such as the Silhouette Width Criterion,
Calinski-Harabasz and Davie's Bouldin indices are the most popular tools for
evaluating and optimising applications of clustering. Their ability to rank
collections of candidate partitions has been used to guide the selection of the
number of clusters, and to compare partitions from different clustering
algorithms. Beyond these more conventional tasks, many examples can be found in
the literature where RVIs have been used to compare and select other aspects of
clustering approaches such as data normalisation procedures, data
representation methods, and distance measures. The authors are not aware of any
studies that have attempted to establish the suitability of RVIs for such
comparisons. Moreover, given the impact of these aspects on pairwise
similarities, it is not even immediately obvious how RVIs should be implemented
when comparing these aspects. In this study, we conducted experiments with
seven common RVIs on over 2.7 million clustering partitions for both synthetic
and real-world datasets, encompassing feature-vector and time-series data. Our
findings suggest that RVIs are not well-suited to these unconventional tasks,
and that conclusions drawn from such applications may be misleading. It is
recommended that normalisation procedures, representation methods, and distance
measures instead be selected using external validation on high quality labelled
datasets or carefully designed outcome-oriented objective criteria, both of
which should be informed by relevant domain knowledge and clustering aims.

这项研究通过对超过 270 万个聚类分区进行实验，发现 RVIs 在非传统任务上并不适用，对此类应用得出的结论可能是误导性的，因此建议使用外部验证和相关领域知识来选择正规化程序、表示方法和距离度量。

比较聚类方法使用相对有效性指数的研究

On the Use of Relative Validity Indices for Comparing Clustering  Approaches

Optimising deep neural networks is a challenging task due to complex training
dynamics, high computational requirements, and long training times. To address
this difficulty, we propose the framework of Generalisable Agents for Neural
Network Optimisation (GANNO) -- a multi-agent reinforcement learning (MARL)
approach that learns to improve neural network optimisation by dynamically and
responsively scheduling hyperparameters during training. GANNO utilises an
agent per layer that observes localised network dynamics and accordingly takes
actions to adjust these dynamics at a layerwise level to collectively improve
global performance. In this paper, we use GANNO to control the layerwise
learning rate and show that the framework can yield useful and responsive
schedules that are competitive with handcrafted heuristics. Furthermore, GANNO
is shown to perform robustly across a wide variety of unseen initial
conditions, and can successfully generalise to harder problems than it was
trained on. Our work presents an overview of the opportunities that this
paradigm offers for training neural networks, along with key challenges that
remain to be overcome.

我们提出了一种名为 GANNO（Generalisable Agents for Neural Network Optimisation）的多智能体强化学习框架，通过动态和响应性地调整超参数来改进神经网络优化，实验结果表明该框架可以与手工调整方法竞争，并能成功适应多种初始条件和更困难的问题。