We present a hierarchical VAE that, for the first time, outperforms the PixelCNN in log-likelihood on all natural image benchmarks. We begin by observing that VAEs can actually implement autoregressive models, and other, more efficient generative models, if made sufficiently deep. Despite this, autoregressive models have traditionally outperformed VAEs. We test if insufficient depth explains the performance gap by by scaling a VAE to greater stochastic depth than previously explored and evaluating it on CIFAR-10, ImageNet, and FFHQ. We find that, in comparison to the PixelCNN, these very deep VAEs achieve higher likelihoods, use fewer parameters, generate samples thousands of times faster, and are more easily applied to high-resolution images. We visualize the generative process and show the VAEs learn efficient hierarchical visual representations. We release our source code and models at https://github.com/openai/vdvae.

本文提出了具有层次结构的VAE，并且在所有自然图像基准上，它比PixelCNN更快地生成样本的同时在对数似然上表现更好。我们通过将VAE的随机深度比以前探索的更深来测试不足深度是否会导致VAE性能下降。与PixelCNN相比，这些非常深的VAE可以获得更高的似然度，使用更少的参数，更快地生成样本，并且对高分辨率图像更容易应用，这是因为VAE学习了高效的分层视觉表示。

极深的VAE可以推广自回归模型并在图像上优于它们