We introduce the $\gamma$-model, a predictive model of environment dynamics
with an infinite probabilistic horizon. Replacing standard single-step models
with $\gamma$-models leads to generalizations of the procedures central to
model-based control, including the model rollout and model-based value
estimation. The $\gamma$-model, trained with a generative reinterpretation of
temporal difference learning, is a natural continuous analogue of the successor
representation and a hybrid between model-free and model-based mechanisms. Like
a value function, it contains information about the long-term future; like a
standard predictive model, it is independent of task reward. We instantiate the
$\gamma$-model as both a generative adversarial network and normalizing flow,
discuss how its training reflects an inescapable tradeoff between training-time
and testing-time compounding errors, and empirically investigate its utility
for prediction and control.

本文介绍了 γ 模型 —— 一种具有无限概率时间的预测环境动态模型。它能够代替常规的单步模型，进一步通用模型控制、模型展开和基于模型价值评估等过程。该模型通过生成式时间差分学习进行训练，是状态衍生表示的连续自然模拟，并结合了无模型和基于模型的特点。我们将 γ 模型的实例作为生成式对抗网络和正则化流，并探讨它在预测和控制方面的实用性。