We propose a new generative hierarchical clustering model that learns a
flexible tree-based posterior distribution over latent variables. The proposed
Tree Variational Autoencoder (TreeVAE) hierarchically divides samples according
to their intrinsic characteristics, shedding light on hidden structure in the
data. It adapts its architecture to discover the optimal tree for encoding
dependencies between latent variables. The proposed tree-based generative
architecture permits lightweight conditional inference and improves generative
performance by utilizing specialized leaf decoders. We show that TreeVAE
uncovers underlying clusters in the data and finds meaningful hierarchical
relations between the different groups on a variety of datasets, including
real-world imaging data. We present empirically that TreeVAE provides a more
competitive log-likelihood lower bound than the sequential counterparts.
Finally, due to its generative nature, TreeVAE is able to generate new samples
from the discovered clusters via conditional sampling.

提出了一种新的生成分层聚类模型 ——Tree Variational Autoencoder（TreeVAE），它能够分层地划分样本并发现数据中潜在结构，并利用树形结构中的叶子节点解码器提高了生成性能。TreeVAE 在不同的数据集上表现出了较高的对数似然下界，并且通过条件采样能够从聚类中生成新的样本。

树形变分自编码器

Tree Variational Autoencoders

Conditional inference on arbitrary subsets of variables is a core problem in
probabilistic inference with important applications such as masked language
modeling and image inpainting. In recent years, the family of Any-Order
Autoregressive Models (AO-ARMs) -- closely related to popular models such as
BERT and XLNet -- has shown breakthrough performance in arbitrary conditional
tasks across a sweeping range of domains. But, in spite of their success, in
this paper we identify significant improvements to be made to previous
formulations of AO-ARMs. First, we show that AO-ARMs suffer from redundancy in
their probabilistic model, i.e., they define the same distribution in multiple
different ways. We alleviate this redundancy by training on a smaller set of
univariate conditionals that still maintains support for efficient arbitrary
conditional inference. Second, we upweight the training loss for univariate
conditionals that are evaluated more frequently during inference. Our method
leads to improved performance with no compromises on tractability, giving
state-of-the-art likelihoods in arbitrary conditional modeling on text (Text8),
image (CIFAR10, ImageNet32), and continuous tabular data domains.

本文针对任意条件建模中的核心问题，提出了一种针对 Any-Order 自回归模型（AO-ARMs）的新的、有效的训练方法，通过训练一组更小的一元条件分布同时提升性能并保证可计算性，在文本、图像和连续数据等领域中取得了最先进的结果。

正确地训练和推断任意阶自回归模型

Training and Inference on Any-Order Autoregressive Models the Right Way

Given an inverse problem with a normalizing flow prior, we wish to estimate
the distribution of the underlying signal conditioned on the observations. We
approach this problem as a task of conditional inference on the pre-trained
unconditional flow model. We first establish that this is computationally hard
for a large class of flow models. Motivated by this, we propose a framework for
approximate inference that estimates the target conditional as a composition of
two flow models. This formulation leads to a stable variational inference
training procedure that avoids adversarial training. Our method is evaluated on
a variety of inverse problems and is shown to produce high-quality samples with
uncertainty quantification. We further demonstrate that our approach can be
amortized for zero-shot inference.

本文提出了一种基于流模型的方法来估计条件密度并解决逆问题，并通过实验验证其在产生高质量样本和不确定性量化方面的有效性。