Offline reinforcement learning (RL) can be used to improve future performance
by leveraging historical data. There exist many different algorithms for
offline RL, and it is well recognized that these algorithms, and their
hyperparameter settings, can lead to decision policies with substantially
differing performance. This prompts the need for pipelines that allow
practitioners to systematically perform algorithm-hyperparameter selection for
their setting. Critically, in most real-world settings, this pipeline must only
involve the use of historical data. Inspired by statistical model selection
methods for supervised learning, we introduce a task- and method-agnostic
pipeline for automatically training, comparing, selecting, and deploying the
best policy when the provided dataset is limited in size. In particular, our
work highlights the importance of performing multiple data splits to produce
more reliable algorithm-hyperparameter selection. While this is a common
approach in supervised learning, to our knowledge, this has not been discussed
in detail in the offline RL setting. We show it can have substantial impacts
when the dataset is small. Compared to alternate approaches, our proposed
pipeline outputs higher-performing deployed policies from a broad range of
offline policy learning algorithms and across various simulation domains in
healthcare, education, and robotics. This work contributes toward the
development of a general-purpose meta-algorithm for automatic
algorithm-hyperparameter selection for offline RL.

提出一种用于离线强化学习中算法和超参数自动选择的元算法，通过使用多个数据拆分来产生更可靠的算法 - 超参数选择，并在医疗保健、教育和机器人技术等各种模拟领域中比较和选择最佳策略，并输出离线策略学习算法的更高性能的策略。