Adversarial Imitation Learning (AIL) allows the agent to reproduce expert
behavior with low-dimensional states and actions. However, challenges arise in
handling visual states due to their less distinguishable representation
compared to low-dimensional proprioceptive features. While existing methods
resort to adopt complex network architectures or separate the process of
learning representation and decision-making, they overlook valuable intra-agent
information within demonstrations. To address this problem, this paper proposes
a simple and effective solution by incorporating calibrated contrastive
representative learning into visual AIL framework. Specifically, we present an
image encoder in visual AIL, utilizing a combination of unsupervised and
supervised contrastive learning to extract valuable features from visual
states. Based on the fact that the improved agent often produces demonstrations
of varying quality, we propose to calibrate the contrastive loss by treating
each agent demonstrations as a mixed sample. The incorporation of contrastive
learning can be jointly optimized with the AIL framework, without modifying the
architecture or incurring significant computational costs. Experimental results
on DMControl Suite demonstrate our proposed method is sample efficient and can
outperform other compared methods from different aspects.

该研究提出了将校准对比学习应用于视觉对抗性模仿学习框架的简单有效解决方案，以从视觉状态中提取有价值的特征，在不修改架构或产生显著计算成本的情况下，可以与模仿学习框架共同优化。实验证明，该方法在 DMControl Suite 上能够以样本效率高、从多个方面超越其他方法。