One noted issue of vector-quantized variational autoencoder (VQ-VAE) is that
the learned discrete representation uses only a fraction of the full capacity
of the codebook, also known as codebook collapse. We hypothesize that the
training scheme of VQ-VAE, which involves some carefully designed heuristics,
underlies this issue. In this paper, we propose a new training scheme that
extends the standard VAE via novel stochastic dequantization and quantization,
called stochastically quantized variational autoencoder (SQ-VAE). In SQ-VAE, we
observe a trend that the quantization is stochastic at the initial stage of the
training but gradually converges toward a deterministic quantization, which we
call self-annealing. Our experiments show that SQ-VAE improves codebook
utilization without using common heuristics. Furthermore, we empirically show
that SQ-VAE is superior to VAE and VQ-VAE in vision- and speech-related tasks.

本文提出了一种名为 SQ-VAE 的新型训练方法，通过引入一种名为自退火的随机量化方法，扩展标准变分自编码器来提高编码本的利用率，并在视觉和语音任务中优于 VAE 和 VQ-VAE