Post-training Neural Network (NN) model compression is an attractive approach
for deploying large, memory-consuming models on devices with limited memory
resources. In this study, we investigate the rate-distortion tradeoff for NN
model compression. First, we suggest a Rotation-Invariant Quantization (RIQ)
technique that utilizes a single parameter to quantize the entire NN model,
yielding a different rate at each layer, i.e., mixed-precision quantization.
Then, we prove that our rotation-invariant approach is optimal in terms of
compression. We rigorously evaluate RIQ and demonstrate its capabilities on
various models and tasks. For example, RIQ facilitates $\times 19.4$ and
$\times 52.9$ compression ratios on pre-trained VGG dense and pruned models,
respectively, with $<0.4\%$ accuracy degradation. Code:
https://github.com/ehaleva/RIQ.

通过提出 RIQ 技术，我们证明了其在压缩神经网络模型方面的最优性，并在各种模型和任务上证明了其优异性能，包括在预训练的 VGG dense 和 pruned 模型上能导致高达 x52.9 的压缩比。

旋转不变量量化用于模型压缩

Rotation Invariant Quantization for Model Compression

We propose and study the problem of distribution-preserving lossy
compression. Motivated by recent advances in extreme image compression which
allow to maintain artifact-free reconstructions even at very low bitrates, we
propose to optimize the rate-distortion tradeoff under the constraint that the
reconstructed samples follow the distribution of the training data. The
resulting compression system recovers both ends of the spectrum: On one hand,
at zero bitrate it learns a generative model of the data, and at high enough
bitrates it achieves perfect reconstruction. Furthermore, for intermediate
bitrates it smoothly interpolates between learning a generative model of the
training data and perfectly reconstructing the training samples. We study
several methods to approximately solve the proposed optimization problem,
including a novel combination of Wasserstein GAN and Wasserstein Autoencoder,
and present an extensive theoretical and empirical characterization of the
proposed compression systems.

本文提出并研究了分布保持有损压缩问题，旨在优化速率失真间的平衡，使重建样本符合训练数据的分布，并通过采用 Wasserstein GAN 和 Wasserstein 自编码器的组合等方法，进行了理论和实证分析。