In Reinforcement Learning, the trade-off between exploration and exploitation
poses a complex challenge for achieving efficient learning from limited
samples. While recent works have been effective in leveraging past experiences
for policy updates, they often overlook the potential of reusing past
experiences for data collection. Independent of the underlying RL algorithm, we
introduce the concept of a Contrastive Initial State Buffer, which
strategically selects states from past experiences and uses them to initialize
the agent in the environment in order to guide it toward more informative
states. We validate our approach on two complex robotic tasks without relying
on any prior information about the environment: (i) locomotion of a quadruped
robot traversing challenging terrains and (ii) a quadcopter drone racing
through a track. The experimental results show that our initial state buffer
achieves higher task performance than the nominal baseline while also speeding
up training convergence.

强化学习中，探索和利用之间的权衡对于从有限样本中实现有效学习提出了复杂的挑战。我们引入了对比初始状态缓冲区的概念，独立于底层的强化学习算法，选择过去的经验中的状态并使用它们来初始化环境中的智能体，以引导它朝着更有信息量的状态发展。实验结果表明，我们的初始状态缓冲区在任务性能方面超过了基准线，同时还加速了训练收敛。