With the widespread adoption of deep learning, reinforcement learning (RL)
has experienced a dramatic increase in popularity, scaling to previously
intractable problems, such as playing complex games from pixel observations,
sustaining conversations with humans, and controlling robotic agents. However,
there is still a wide range of domains inaccessible to RL due to the high cost
and danger of interacting with the environment. Offline RL is a paradigm that
learns exclusively from static datasets of previously collected interactions,
making it feasible to extract policies from large and diverse training
datasets. Effective offline RL algorithms have a much wider range of
applications than online RL, being particularly appealing for real-world
applications such as education, healthcare, and robotics. In this work, we
propose a unifying taxonomy to classify offline RL methods. Furthermore, we
provide a comprehensive review of the latest algorithmic breakthroughs in the
field, and a review of existing benchmarks' properties and shortcomings.
Finally, we provide our perspective on open problems and propose future
research directions for this rapidly growing field.

本论文提出一个在线学习和离线学习技术的归一化分类法，总结了离线 RL 领域的最新算法突破和现有基准的特性和不足，并提供了对未来研究方向的展望。