When a robot autonomously performs a complex task, it frequently must balance
competing objectives while maintaining safety. This becomes more difficult in
uncertain environments with stochastic outcomes. Enhancing transparency in the
robot's behavior and aligning with user preferences are also crucial. This
paper introduces a novel framework for multi-objective reinforcement learning
that ensures safe task execution, optimizes trade-offs between objectives, and
adheres to user preferences. The framework has two main layers: a
multi-objective task planner and a high-level selector. The planning layer
generates a set of optimal trade-off plans that guarantee satisfaction of a
temporal logic task. The selector uses active inference to decide which
generated plan best complies with user preferences and aids learning. Operating
iteratively, the framework updates a parameterized learning model based on
collected data. Case studies and benchmarks on both manipulation and mobile
robots show that our framework outperforms other methods and (i) learns
multiple optimal trade-offs, (ii) adheres to a user preference, and (iii)
allows the user to adjust the balance between (i) and (ii).

本研究旨在引入一种新颖的多目标强化学习框架，确保任务的安全执行，优化目标之间的权衡，并符合用户的偏好。该框架包含两个主要层次：多目标任务规划器和高层选择器。通过案例研究和基准测试，证明了我们的框架在操作和移动机器人方面优于其他方法，并且能够学习多个最优权衡解决方案，符合用户的偏好，并允许用户调整优化权衡的平衡。