Neural density estimators are flexible families of parametric models which
have seen widespread use in unsupervised machine learning in recent years.
Maximum-likelihood training typically dictates that these models be constrained
to specify an explicit density. However, this limitation can be overcome by
instead using a neural network to specify an energy function, or unnormalized
density, which can subsequently be normalized to obtain a valid distribution.
The challenge with this approach lies in accurately estimating the normalizing
constant of the high-dimensional energy function. We propose the Autoregressive
Energy Machine, an energy-based model which simultaneously learns an
unnormalized density and computes an importance-sampling estimate of the
normalizing constant for each conditional in an autoregressive decomposition.
The Autoregressive Energy Machine achieves state-of-the-art performance on a
suite of density-estimation tasks.

Autoregressive Energy Machine 是一种高效的基于能量的神经网络模型，具备在无监督学习中广泛使用的灵活性，可以在不受概率密度限制的条件下计算归一化常数，实现在密度估计任务中的最优表现。

自回归能量机

Autoregressive Energy Machines

We consider the off-policy evaluation problem in Markov decision processes
with function approximation. We propose a generalization of the recently
introduced \emph{emphatic temporal differences} (ETD) algorithm
\citep{SuttonMW15}, which encompasses the original ETD($\lambda$), as well as
several other off-policy evaluation algorithms as special cases. We call this
framework \ETD, where our introduced parameter $\beta$ controls the decay rate
of an importance-sampling term. We study conditions under which the projected
fixed-point equation underlying \ETD\ involves a contraction operator, allowing
us to present the first asymptotic error bounds (bias) for \ETD. Our results
show that the original ETD algorithm always involves a contraction operator,
and its bias is bounded. Moreover, by controlling $\beta$, our proposed
generalization allows trading-off bias for variance reduction, thereby
achieving a lower total error.

本文提出了一种广义的基于强化学习的离线评估算法，该算法可以控制一个重要性采样因子的衰减速度，并研究了其收敛和精度属性。