Weight-sharing quantization has emerged as a technique to reduce energy
expenditure during inference in large neural networks by constraining their
weights to a limited set of values. However, existing methods for
weight-sharing quantization often make assumptions about the treatment of
weights based on value alone that neglect the unique role weight position
plays. This paper proposes a probabilistic framework based on Bayesian neural
networks (BNNs) and a variational relaxation to identify which weights can be
moved to which cluster centre and to what degree based on their individual
position-specific learned uncertainty distributions. We introduce a new
initialisation setting and a regularisation term which allow for the training
of BNNs under complex dataset-model combinations. By leveraging the flexibility
of weight values captured through a probability distribution, we enhance noise
resilience and downstream compressibility. Our iterative clustering procedure
demonstrates superior compressibility and higher accuracy compared to
state-of-the-art methods on both ResNet models and the more complex
transformer-based architectures. In particular, our method outperforms the
state-of-the-art quantization method top-1 accuracy by 1.6% on ImageNet using
DeiT-Tiny, with its 5 million+ weights now represented by only 296 unique
values.

该论文提出了一种基于贝叶斯神经网络（BNNs）和变分松弛的概率性框架，用于确定可以将哪些权重转移到哪个聚类中心以及在多大程度上，以提高权重共享量化方法的压缩性能和准确性。