Inverse Reinforcement Learning (IRL) techniques deal with the problem of
deducing a reward function that explains the behavior of an expert agent who is
assumed to act optimally in an underlying unknown task. In several problems of
interest, however, it is possible to observe the behavior of multiple experts
with different degree of optimality (e.g., racing drivers whose skills ranges
from amateurs to professionals). For this reason, in this work, we extend the
IRL formulation to problems where, in addition to demonstrations from the
optimal agent, we can observe the behavior of multiple sub-optimal experts.
Given this problem, we first study the theoretical properties of the class of
reward functions that are compatible with a given set of experts, i.e., the
feasible reward set. Our results show that the presence of multiple sub-optimal
experts can significantly shrink the set of compatible rewards. Furthermore, we
study the statistical complexity of estimating the feasible reward set with a
generative model. To this end, we analyze a uniform sampling algorithm that
results in being minimax optimal whenever the sub-optimal experts' performance
level is sufficiently close to the one of the optimal agent.

给定一个包含多个次优专家行为的问题，我们将逆向强化学习（IRL）方法扩展到了这种情况，研究了与给定专家集兼容的奖励函数的理论性质，并分析了使用生成模型估计可行奖励集的统计复杂性，得到了一个具有极小极大最优性的均匀采样算法。