Vector Quantisation (VQ) is experiencing a comeback in machine learning,
where it is increasingly used in representation learning. However, optimizing
the codevectors in existing VQ-VAE is not entirely trivial. A problem is
codebook collapse, where only a small subset of codevectors receive gradients
useful for their optimisation, whereas a majority of them simply ``dies off''
and is never updated or used. This limits the effectiveness of VQ for learning
larger codebooks in complex computer vision tasks that require high-capacity
representations. In this paper, we present a simple alternative method for
online codebook learning, Clustering VQ-VAE (CVQ-VAE). Our approach selects
encoded features as anchors to update the ``dead'' codevectors, while
optimising the codebooks which are alive via the original loss. This strategy
brings unused codevectors closer in distribution to the encoded features,
increasing the likelihood of being chosen and optimized. We extensively
validate the generalization capability of our quantiser on various datasets,
tasks (e.g. reconstruction and generation), and architectures (e.g. VQ-VAE,
VQGAN, LDM). Our CVQ-VAE can be easily integrated into the existing models with
just a few lines of code.

参考文献通过在线聚类学习方法 Clustering VQ-VAE（CVQ-VAE）解决了向量量化（VQ）中的代码书坍塌问题，提高了在复杂的计算机视觉任务中学习更大的代码书的效果。