Vector quantization (VQ) is a technique to deterministically learn features
with discrete codebook representations. It is commonly performed with a
variational autoencoding model, VQ-VAE, which can be further extended to
hierarchical structures for making high-fidelity reconstructions. However, such
hierarchical extensions of VQ-VAE often suffer from the codebook/layer collapse
issue, where the codebook is not efficiently used to express the data, and
hence degrades reconstruction accuracy. To mitigate this problem, we propose a
novel unified framework to stochastically learn hierarchical discrete
representation on the basis of the variational Bayes framework, called
hierarchically quantized variational autoencoder (HQ-VAE). HQ-VAE naturally
generalizes the hierarchical variants of VQ-VAE, such as VQ-VAE-2 and
residual-quantized VAE (RQ-VAE), and provides them with a Bayesian training
scheme. Our comprehensive experiments on image datasets show that HQ-VAE
enhances codebook usage and improves reconstruction performance. We also
validated HQ-VAE in terms of its applicability to a different modality with an
audio dataset.

基于变分贝叶斯框架，提出一种新颖的统一框架，即分层量化变分自动编码器（HQ-VAE），其以离散表示形式随机学习层次化的离散特征，并解决了分层 VQ-VAE 中的代码本 / 层级坍塌问题，从而提高重建性能。