Robust optimisation is a well-established framework for optimising functions
in the presence of uncertainty. The inherent goal of this problem is to
identify a collection of inputs whose outputs are both desirable for the
decision maker, whilst also being robust to the underlying uncertainties in the
problem. In this work, we study the multi-objective extension of this problem
from a computational standpoint. We identify that the majority of all robust
multi-objective algorithms rely on two key operations: robustification and
scalarisation. Robustification refers to the strategy that is used to
marginalise over the uncertainty in the problem. Whilst scalarisation refers to
the procedure that is used to encode the relative importance of each objective.
As these operations are not necessarily commutative, the order that they are
performed in has an impact on the resulting solutions that are identified and
the final decisions that are made. This work aims to give an exposition on the
philosophical differences between these two operations and highlight when one
should opt for one ordering over the other. As part of our analysis, we
showcase how many existing risk concepts can be easily integrated into the
specification and solution of a robust multi-objective optimisation problem.
Besides this, we also demonstrate how one can principally define the notion of
a robust Pareto front and a robust performance metric based on our robustify
and scalarise methodology. To illustrate the efficacy of these new ideas, we
present two insightful numerical case studies which are based on real-world
data sets.

基于鲁棒优化的多目标问题，介绍了鲁棒化和标量化两种关键操作之间的哲学差异，展示了风险概念在鲁棒多目标优化问题中的集成，并通过两个基于真实数据集的案例研究验证了这些新思路的有效性。

基于标量化的风险概念的鲁棒多目标优化

Scalarisation-based risk concepts for robust multi-objective  optimisation

Multi-objective reinforcement learning (MORL) algorithms extend conventional
reinforcement learning (RL) to the more general case of problems with multiple,
conflicting objectives, represented by vector-valued rewards. Widely-used
scalar RL methods such as Q-learning can be modified to handle multiple
objectives by (1) learning vector-valued value functions, and (2) performing
action selection using a scalarisation or ordering operator which reflects the
user's utility with respect to the different objectives. However, as we
demonstrate here, if the user's utility function maps widely varying
vector-values to similar levels of utility, this can lead to interference in
the value-function learned by the agent, leading to convergence to sub-optimal
policies. This will be most prevalent in stochastic environments when
optimising for the Expected Scalarised Return criterion, but we present a
simple example showing that interference can also arise in deterministic
environments. We demonstrate empirically that avoiding the use of random
tie-breaking when identifying greedy actions can ameliorate, but not fully
overcome, the problems caused by value function interference.

多目标强化学习算法扩展了传统的强化学习方法以解决存在多个冲突目标的问题，通过向量值奖励进行表示。我们在此论文中展示了，如果用户的效用函数将各向量值映射到相似的效用级别，这可能导致代理学到的值函数受到干扰，从而收敛到次优策略。尽管在确定贪婪动作时避免使用随机打破关系，可以缓解由值函数干扰引起的问题，但并不能完全克服这个问题。

基于价值的多目标强化学习中的价值函数干扰和贪婪动作选择

Value function interference and greedy action selection in value-based  multi-objective reinforcement learning

Optimisation problems often have multiple conflicting objectives that can be
computationally and/or financially expensive. Mono-surrogate Bayesian
optimisation (BO) is a popular model-based approach for optimising such
black-box functions. It combines objective values via scalarisation and builds
a Gaussian process (GP) surrogate of the scalarised values. The location which
maximises a cheap-to-query acquisition function is chosen as the next location
to expensively evaluate. While BO is an effective strategy, the use of GPs is
limiting. Their performance decreases as the problem input dimensionality
increases, and their computational complexity scales cubically with the amount
of data. To address these limitations, we extend previous work on BO by
density-ratio estimation (BORE) to the multi-objective setting. BORE links the
computation of the probability of improvement acquisition function to that of
probabilistic classification. This enables the use of state-of-the-art
classifiers in a BO-like framework. In this work we present MBORE:
multi-objective Bayesian optimisation by density-ratio estimation, and compare
it to BO across a range of synthetic and real-world benchmarks. We find that
MBORE performs as well as or better than BO on a wide variety of problems, and
that it outperforms BO on high-dimensional and real-world problems.

这篇论文针对多目标优化问题，提出了一种名为 MBORE 的方法，它将 BORE 扩展到了多目标问题，并在一系列合成和真实世界基准测试中与 BO 进行了比较。结果表明，在许多问题上，MBORE 的表现要好于或优于 BO，在高维度和真实世界问题上优势更为明显。