Consistently checking the statistical significance of experimental results is
the first mandatory step towards reproducible science. This paper presents a
hitchhiker's guide to rigorous comparisons of reinforcement learning
algorithms. After introducing the concepts of statistical testing, we review
the relevant statistical tests and compare them empirically in terms of false
positive rate and statistical power as a function of the sample size (number of
seeds) and effect size. We further investigate the robustness of these tests to
violations of the most common hypotheses (normal distributions, same
distributions, equal variances). Beside simulations, we compare empirical
distributions obtained by running Soft-Actor Critic and Twin-Delayed Deep
Deterministic Policy Gradient on Half-Cheetah. We conclude by providing
guidelines and code to perform rigorous comparisons of RL algorithm
performances.

本文介绍了一份关于如何严格比较增强学习算法的指南，包括介绍了统计测试的概念、审查相关的统计测试以及比较它们在不同的样本量和效果量下的假阳性率和统计功效。除了通过模拟比较不同测试之外，我们还在 Half-Cheetah 上比较了 Soft-Actor Critic 和 Twin-Delayed Deep Deterministic Policy Gradient 算法的表现，并提供了指南和代码以进行 RL 算法性能的严格比较。