Practical Imitation Learning (IL) systems rely on large human demonstration
datasets for successful policy learning. However, challenges lie in maintaining
the quality of collected data and addressing the suboptimal nature of some
demonstrations, which can compromise the overall dataset quality and hence the
learning outcome. Furthermore, the intrinsic heterogeneity in human behavior
can produce equally successful but disparate demonstrations, further
exacerbating the challenge of discerning demonstration quality. To address
these challenges, this paper introduces Learning to Discern (L2D), an offline
imitation learning framework for learning from demonstrations with diverse
quality and style. Given a small batch of demonstrations with sparse quality
labels, we learn a latent representation for temporally embedded trajectory
segments. Preference learning in this latent space trains a quality evaluator
that generalizes to new demonstrators exhibiting different styles. Empirically,
we show that L2D can effectively assess and learn from varying demonstrations,
thereby leading to improved policy performance across a range of tasks in both
simulations and on a physical robot.

本文介绍了一种离线模仿学习框架，Learning to Discern (L2D)，用于从具有多样化质量和风格的演示中学习。通过在潜在空间中进行偏好学习，我们能够对不同风格的新演示进行评估和学习，从而提高各种任务的策略性能。