Bayesian optimization is a principled optimization strategy for a black-box
objective function. It shows its effectiveness in a wide variety of real-world
applications such as scientific discovery and experimental design. In general,
the performance of Bayesian optimization is assessed by regret-based metrics
such as instantaneous, simple, and cumulative regrets. These metrics only rely
on function evaluations, so that they do not consider geometric relationships
between query points and global solutions, or query points themselves. Notably,
they cannot discriminate if multiple global solutions are successfully found.
Moreover, they do not evaluate Bayesian optimization's abilities to exploit and
explore a search space given. To tackle these issues, we propose four new
geometric metrics, i.e., precision, recall, average degree, and average
distance. These metrics allow us to compare Bayesian optimization algorithms
considering the geometry of both query points and global optima, or query
points. However, they are accompanied by an extra parameter, which needs to be
carefully determined. We therefore devise the parameter-free forms of the
respective metrics by integrating out the additional parameter. Finally, we
empirically validate that our proposed metrics can provide more convincing
interpretation and understanding of Bayesian optimization algorithms from
distinct perspectives, compared to the conventional metrics.

贝叶斯优化算法的效果评估依赖于遗憾度量，然而这些度量无法考虑查询点与全局解的几何关系，以及贝叶斯优化算法对搜索空间的开发和探索能力。为解决这些问题，本研究提出了四个新的几何度量，并通过集成额外参数来得到无需参数的指标形式。经实验证明，这些新的度量方法能更全面、有说服力地解释和理解贝叶斯优化算法。

超越遗憾：贝叶斯优化的几何度量

Beyond Regrets: Geometric Metrics for Bayesian Optimization

Many deep learning methods have successfully solved complex tasks in computer
vision and speech recognition applications. Nonetheless, the robustness of
these models has been found to be vulnerable to perturbed inputs or adversarial
examples, which are imperceptible to the human eye, but lead the model to
erroneous output decisions. In this study, we adapt and introduce two geometric
metrics, density and coverage, and evaluate their use in detecting adversarial
samples in batches of unseen data. We empirically study these metrics using
MNIST and two real-world biomedical datasets from MedMNIST, subjected to two
different adversarial attacks. Our experiments show promising results for both
metrics to detect adversarial examples. We believe that his work can lay the
ground for further study on these metrics' use in deployed machine learning
systems to monitor for possible attacks by adversarial examples or related
pathologies such as dataset shift.

本文研究通过引入几何度量法来检测深度学习模型中的对抗样本，并以 MNIST 数据集和两个医学数据集为例，对密度和覆盖度两种几何度量法进行了实证研究，结果表明这两种度量法能够检测到对抗样本并能在机器学习系统中使用来监控可能的对抗性例子或相关病理因素，为进一步研究机器学习系统的安全提供了基础。