Learning diverse and qualified behaviors for utilization and adaptation
without supervision is a key ability of intelligent creatures. Ideal
unsupervised skill discovery methods are able to produce diverse and qualified
skills in the absence of extrinsic reward, while the discovered skill set can
efficiently adapt to downstream tasks in various ways. Maximizing the Mutual
Information (MI) between skills and visited states can achieve ideal
skill-conditioned behavior distillation in theory. However, it's difficult for
recent advanced methods to well balance behavioral quality (exploration) and
diversity (exploitation) in practice, which may be attributed to the
unreasonable MI estimation by their rigid intrinsic reward design. In this
paper, we propose Contrastive multi-objectives Skill Discovery (ComSD) which
tries to mitigate the quality-versus-diversity conflict of discovered behaviors
through a more reasonable MI estimation and a dynamically weighted intrinsic
reward. ComSD proposes to employ contrastive learning for a more reasonable
estimation of skill-conditioned entropy in MI decomposition. In addition, a
novel weighting mechanism is proposed to dynamically balance different entropy
(in MI decomposition) estimations into a novel multi-objective intrinsic
reward, to improve both skill diversity and quality. For challenging robot
behavior discovery, ComSD can produce a qualified skill set consisting of
diverse behaviors at different activity levels, which recent advanced methods
cannot. On numerical evaluations, ComSD exhibits state-of-the-art adaptation
performance, significantly outperforming recent advanced skill discovery
methods across all skill combination tasks and most skill finetuning tasks.
Codes will be released at this https URL

为了解决质量和多样性冲突的问题，论文提出了一种对比的多目标技能发现方法，通过更合理的互信息估计和动态加权的内在奖励来平衡发现行为的质量和多样性，并在挑战性机器人行为发现任务中展示了优秀的性能。