We introduce the Generalized Energy Based Model (GEBM) for generative
modelling. These models combine two trained components: a base distribution
(generally an implicit model), which can learn the support of data with low
intrinsic dimension in a high dimensional space; and an energy function, to
refine the probability mass on the learned support. Both the energy function
and base jointly constitute the final model, unlike GANs, which retain only the
base distribution (the "generator"). GEBMs are trained by alternating between
learning the energy and the base. We show that both training stages are
well-defined: the energy is learned by maximising a generalized likelihood, and
the resulting energy-based loss provides informative gradients for learning the
base. Samples from the posterior on the latent space of the trained model can
be obtained via MCMC, thus finding regions in this space that produce better
quality samples. Empirically, the GEBM samples on image-generation tasks are of
much better quality than those from the learned generator alone, indicating
that all else being equal, the GEBM will outperform a GAN of the same
complexity. When using normalizing flows as base measures, GEBMs succeed on
density modelling tasks, returning comparable performance to direct maximum
likelihood of the same networks.

本文介绍了广义能量模型（GEBM）的产生式建模方法，指出了其在图像生成任务和密度建模任务中的应用和比 GANS 更好的性能表现。

广义能量基模型

Generalized Energy Based Models

Although exploration in reinforcement learning is well understood from a
theoretical point of view, provably correct methods remain impractical. In this
paper we study the interplay between exploration and approximation, what we
call approximate exploration. Our main goal is to further our theoretical
understanding of pseudo-count based exploration bonuses (Bellemare et al.,
2016), a practical exploration scheme based on density modelling. As a warm-up,
we quantify the performance of an exploration algorithm, MBIE-EB (Strehl and
Littman, 2008), when explicitly combined with state aggregation. This allows us
to confirm that, as might be expected, approximation allows the agent to trade
off between learning speed and quality of the learned policy. Next, we show how
a given density model can be related to an abstraction and that the
corresponding pseudo-count bonus can act as a substitute in MBIE-EB combined
with this abstraction, but may lead to either under- or over-exploration. Then,
we show that a given density model also defines an implicit abstraction, and
find a surprising mismatch between pseudo-counts derived either implicitly or
explicitly. Finally we derive a new pseudo-count bonus alleviating this issue.

研究强化学习中探索和近似之间的相互作用，提出一种基于密度建模的方法来改善探索，探讨伪计数奖励在此方法中的应用，发现了在其应用中可能存在的过度或不足探索问题，并提出一种新的伪计数奖励来缓解这些问题。

基于状态抽象的近似探索

Approximate Exploration through State Abstraction

State-of-the-art Deep Neural Networks can be easily fooled into providing
incorrect high-confidence predictions for images with small amounts of
adversarial noise. Does this expose a flaw with deep neural networks, or do we
simply need a better way to estimate confidence? In this paper we consider the
problem of accurately estimating predictive confidence. We formulate this
problem as that of density modelling, and show how traditional methods such as
softmax produce poor estimates. To address this issue, we propose a novel
confidence measure based on density modelling approaches. We test these
measures on images distorted by blur, JPEG compression, random noise and
adversarial noise. Experiments show that our confidence measure consistently
shows reduced confidence scores in the presence of such distortions - a
property which softmax often lacks.

本文提出了一种基于密度建模的新型置信度度量方法，用以解决 State-of-the-art Deep Neural Networks 存在的小量对抗噪音引起高置信度预测不正确问题，并验证其在图像失真、随机噪声和对抗噪声等方面具有鲁棒性，相比传统方法 softmax，置信度得分更低。