The success of reinforcement learning for real world robotics has been, in many cases limited to instrumented laboratory scenarios, often requiring arduous human effort and oversight to enable continuous learning. In this work, we discuss the elements that are needed for a robotic learning system that can continually and autonomously improve with data collected in the real world. We propose a particular instantiation of such a system, using dexterous manipulation as our case study. Subsequently, we investigate a number of challenges that come up when learning without instrumentation. In such settings, learning must be feasible without manually designed resets, using only on-board perception, and without hand-engineered reward functions. We propose simple and scalable solutions to these challenges, and then demonstrate the efficacy of our proposed system on a set of dexterous robotic manipulation tasks, providing an in-depth analysis of the challenges associated with this learning paradigm. We demonstrate that our complete system can learn without any human intervention, acquiring a variety of vision-based skills with a real-world three-fingered hand. Results and videos can be found at https://sites.google.com/view/realworld-rl/

本研究提出了一种基于强化学习的机器人学习系统，通过无需人工干预的自主学习，在真实世界环境下实现智能机器人的不断优化和提高。以灵巧操作为案例研究，探究了在无工具化监控和无手工奖励函数的情况下学习的挑战，并提出了简单、可扩展的解决方案，通过多次灵巧操作任务实验验证了该系统的高效性和优越性。

真实世界机器人强化学习的要素