Curricula for goal-conditioned reinforcement learning agents typically rely on poor estimates of the agent's epistemic uncertainty or fail to consider the agents' epistemic uncertainty altogether, resulting in poor sample efficiency. We propose a novel algorithm, Query The Agent (QTA), which significantly improves sample efficiency by estimating the agent's epistemic uncertainty throughout the state space and setting goals in highly uncertain areas. Encouraging the agent to collect data in highly uncertain states allows the agent to improve its estimation of the value function rapidly. QTA utilizes a novel technique for estimating epistemic uncertainty, Predictive Uncertainty Networks (PUN), to allow QTA to assess the agent's uncertainty in all previously observed states. We demonstrate that QTA offers decisive sample efficiency improvements over preexisting methods.

本研究提出了一种名为“Query The Agent(QTA)”的新算法，它通过在状态空间中估计代理的认识不确定性并在高度不确定的区域中设定目标来显著提高样本效率。 QTA利用一种名为“Predictive Uncertainty Networks（PUN）”的新技术来估计认知不确定性，以便在所有先前观察到的状态下对代理的不确定性进行评估。我们证明QTA相对于现有方法具有决定性的样本效率提高。

查询智能体：通过认知不确定性估计提高样本利用效率