In safe reinforcement learning, agent needs to balance between exploration actions and safety constraints. Following this paradigm, domain transfer approaches learn a prior Q-function from the related environments to prevent unsafe actions. However, because of the large number of false positives, some safe actions are never executed, leading to inadequate exploration in sparse-reward environments. In this work, we aim to learn an efficient state representation to balance the exploration and safety-prefer action in a sparse-reward environment. Firstly, the image input is mapped to latent representation by an auto-encoder. A further contrastive learning objective is employed to distinguish safe and unsafe states. In the learning phase, the latent distance is used to construct an additional safety check, which allows the agent to bias the exploration if it visits an unsafe state. To verify the effectiveness of our method, the experiment is carried out in three navigation-based MiniGrid environments. The result highlights that our method can explore the environment better while maintaining a good balance between safety and efficiency.

本研究解决了安全强化学习中探索与安全约束之间的平衡问题，提出了一种高效的状态表征学习方法，以应对稀疏奖励环境中的不充分探索。通过使用自编码器映射输入图像到隐层表示，并采用对比学习目标，研究显示该方法在保证安全性的同时，显著提高了探索效率。

通过对比表征学习增强安全强化学习中的探索