We introduce off-policy distributional Q($\lambda$), a new addition to the
family of off-policy distributional evaluation algorithms. Off-policy
distributional Q($\lambda$) does not apply importance sampling for off-policy
learning, which introduces intriguing interactions with signed measures. Such
unique properties distributional Q($\lambda$) from other existing alternatives
such as distributional Retrace. We characterize the algorithmic properties of
distributional Q($\lambda$) and validate theoretical insights with tabular
experiments. We show how distributional Q($\lambda$)-C51, a combination of
Q($\lambda$) with the C51 agent, exhibits promising results on deep RL
benchmarks.

介绍了一种新的离线策略分布式 Q (λ) 评估算法，它与已有的算法不同之处在于不使用重要性抽样，并且在与符号测度的交互过程中具有独特的性质。通过表格实验验证了分布式 Q (λ) 的算法性质和理论洞见的有效性，并展示了将 Q (λ) 与 C51 代理相结合的分布式 Q (λ)-C51 在深度 RL 基准测试中取得的有希望的结果。