Preferential Bayesian optimization (PBO) is a framework for optimizing a
decision-maker's latent preferences over available design choices. While
preferences often involve multiple conflicting objectives, existing work in PBO
assumes that preferences can be encoded by a single objective function. For
example, in robotic assistive devices, technicians often attempt to maximize
user comfort while simultaneously minimizing mechanical energy consumption for
longer battery life. Similarly, in autonomous driving policy design,
decision-makers wish to understand the trade-offs between multiple safety and
performance attributes before committing to a policy. To address this gap, we
propose the first framework for PBO with multiple objectives. Within this
framework, we present dueling scalarized Thompson sampling (DSTS), a
multi-objective generalization of the popular dueling Thompson algorithm, which
may be of interest beyond the PBO setting. We evaluate DSTS across four
synthetic test functions and two simulated exoskeleton personalization and
driving policy design tasks, showing that it outperforms several benchmarks.
Finally, we prove that DSTS is asymptotically consistent. As a direct
consequence, this result provides, to our knowledge, the first convergence
guarantee for dueling Thompson sampling in the PBO setting.

基于偏好的贝叶斯优化（PBO）框架用于优化决策者对可用设计选择的潜在偏好。本文提出了第一个具有多目标的 PBO 框架，并介绍了 dueling scalarized Thompson sampling（DSTS），它是一种流行的 dueling Thompson 算法的多目标推广。研究发现 DSTS 在多个测试函数和模拟的个性化外骨骼和驾驶政策设计任务中优于其他基准算法，并证明了 DSTS 是渐进一致的，为 PBO 中的 dueling Thompson 采样提供了首个收敛保证。