Recent work has shown that supervised learning alone, without temporal
difference (TD) learning, can be remarkably effective for offline RL. When does
this hold true, and which algorithmic components are necessary? Through
extensive experiments, we boil supervised learning for offline RL down to its
essential elements. In every environment suite we consider, simply maximizing
likelihood with a two-layer feedforward MLP is competitive with
state-of-the-art results of substantially more complex methods based on TD
learning or sequence modeling with Transformers. Carefully choosing model
capacity (e.g., via regularization or architecture) and choosing which
information to condition on (e.g., goals or rewards) are critical for
performance. These insights serve as a field guide for practitioners doing
Reinforcement Learning via Supervised Learning (which we coin "RvS learning").
They also probe the limits of existing RvS methods, which are comparatively
weak on random data, and suggest a number of open problems.

离线强化学习中的监督学习方法及其必要算法组件已被证明可以达到与基于 TD 学习或序列建模的复杂方法相当的高效性；如何选择模型容量和条件信息（例如目标或奖励）可以对性能产生至关重要的影响，这提供了对 RvS 学习从业者的指南。