We focus on learning the desired objective function for a robot. Although trajectory demonstrations can be very informative of the desired objective, they can also be difficult for users to provide. Answers to comparison queries, asking which of two trajectories is preferable, are much easier for users, and have emerged as an effective alternative. Unfortunately, comparisons are far less informative. We propose that there is much richer information that users can easily provide and that robots ought to leverage. We focus on augmenting comparisons with feature queries, and introduce a unified formalism for treating all answers as observations about the true desired reward. We derive an active query selection algorithm, and test these queries in simulation and on real users. We find that richer, feature-augmented queries can extract more information faster, leading to robots that better match user preferences in their behavior.

本文针对机器人学习中的目标函数问题，提出了使用特征查询来增强比较查询的方法，并使用派生的主动查询选择算法，在仿真和真实用户测试中验证了指定特征增强查询比比较查询可获得更丰富的信息，可以更快地提取信息，从而更好地匹配用户行为偏好。

从更丰富的人类指导中学习：通过特征查询增强基于比较的学习