Balancing multiple competing and conflicting objectives is an essential task for any artificial intelligence tasked with satisfying human values or preferences. Conflict arises both from misalignment between individuals with competing values, but also between conflicting value systems held by a single human. Starting with principle of loss-aversion, we designed a set of soft maximin function approaches to multi-objective decision-making. Bench-marking these functions in a set of previously-developed environments, we found that one new approach in particular, `split-function exp-log loss aversion' (SFELLA), learns faster than the state of the art thresholded alignment objective method \cite{vamplew_potential-based_2021} on three of four tasks it was tested on, and achieved the same optimal performance after learning. SFELLA also showed relative robustness improvements against changes in objective scale, which may highlight an advantage dealing with distribution shifts in the environment dynamics. Further work had to be omitted from the preprint, but in the final published version, we will further compare SFELLA to the multi-objective reward exponentials (MORE) approach, demonstrating that SFELLA performs similarly to MORE in a simple previously-described foraging task, but in a modified foraging environment with a new resource that was not depleted as the agent worked, SFELLA collected more of the new resource with very little cost incurred in terms of the old resource. Overall, we found SFELLA useful for avoiding problems that sometimes occur with a thresholded approach, and more reward-responsive than MORE while retaining its conservative, loss-averse incentive structure.

本文提出了一种新的多目标决策方法-SFELLA，它基于损失规避原则，避免了阈值导致的问题，并且相较于MORE方法更加回应奖励，但保留了它保守的、损失规避的激励结构。

运用软最小化方法提高瓶颈环境下多目标决策性能