In this paper we present the first empirical study of the emphatic
temporal-difference learning algorithm (ETD), comparing it with conventional
temporal-difference learning, in particular, with linear TD(0), on on-policy
and off-policy variations of the Mountain Car problem. The initial motivation
for developing ETD was that it has good convergence properties under off-policy
training (Sutton, Mahmood and White 2016), but it is also a new algorithm for
the on-policy case. In both our on-policy and off-policy experiments, we found
that each method converged to a characteristic asymptotic level of error, with
ETD better than TD(0). TD(0) achieved a still lower error level temporarily
before falling back to its higher asymptote, whereas ETD never showed this kind
of "bounce". In the off-policy case (in which TD(0) is not guaranteed to
converge), ETD was significantly slower.

本文首次实证研究了强调时间差分学习算法（ETD）在 Mountain Car 问题的使用，对比标准的时间差分学习算法，比如线性 TD（0），研究表明在 on-policy 和 off-policy 两种情况下，ETD 算法的收敛效果更好。