This paper studies the fundamental problem of multi-layer generator models in
learning hierarchical representations. The multi-layer generator model that
consists of multiple layers of latent variables organized in a top-down
architecture tends to learn multiple levels of data abstraction. However, such
multi-layer latent variables are typically parameterized to be Gaussian, which
can be less informative in capturing complex abstractions, resulting in limited
success in hierarchical representation learning. On the other hand, the
energy-based (EBM) prior is known to be expressive in capturing the data
regularities, but it often lacks the hierarchical structure to capture
different levels of hierarchical representations. In this paper, we propose a
joint latent space EBM prior model with multi-layer latent variables for
effective hierarchical representation learning. We develop a variational joint
learning scheme that seamlessly integrates an inference model for efficient
inference. Our experiments demonstrate that the proposed joint EBM prior is
effective and expressive in capturing hierarchical representations and
modelling data distribution.

本文研究多层生成模型在学习分层表示中的基本问题，并提出了一种联合潜在空间的基于能量的模型，通过多层潜在变量实现了有效的分层表示学习，并对数据分布建模。

使用联合潜在空间能量先验学习层次特征

Learning Hierarchical Features with Joint Latent Space Energy-Based  Prior

Variational autoencoders (VAEs) are one of the powerful likelihood-based
generative models with applications in many domains. However, they struggle to
generate high-quality images, especially when samples are obtained from the
prior without any tempering. One explanation for VAEs' poor generative quality
is the prior hole problem: the prior distribution fails to match the aggregate
approximate posterior. Due to this mismatch, there exist areas in the latent
space with high density under the prior that do not correspond to any encoded
image. Samples from those areas are decoded to corrupted images. To tackle this
issue, we propose an energy-based prior defined by the product of a base prior
distribution and a reweighting factor, designed to bring the base closer to the
aggregate posterior. We train the reweighting factor by noise contrastive
estimation, and we generalize it to hierarchical VAEs with many latent variable
groups. Our experiments confirm that the proposed noise contrastive priors
improve the generative performance of state-of-the-art VAEs by a large margin
on the MNIST, CIFAR-10, CelebA 64, and CelebA HQ 256 datasets. Our method is
simple and can be applied to a wide variety of VAEs to improve the expressivity
of their prior distribution.

针对变分自编码器的先验洞问题，我们提出了一种能源化先验，通过基础先验分布和重新加权因子的乘积来定义，以使基础先验更接近集体后验，并通过噪声对比估计来训练重新加权因子，实验证明此方法可以大幅提高 MNIST、CIFAR-10、CelebA 64 和 CelebA HQ 256 数据集上的最先进 VAE 的生成性能。