In this paper, we address the problem of reducing the memory footprint of
convolutional network architectures. We introduce a vector quantization method
that aims at preserving the quality of the reconstruction of the network
outputs rather than its weights. The principle of our approach is that it
minimizes the loss reconstruction error for in-domain inputs. Our method only
requires a set of unlabelled data at quantization time and allows for efficient
inference on CPU by using byte-aligned codebooks to store the compressed
weights. We validate our approach by quantizing a high performing ResNet-50
model to a memory size of 5MB (20x compression factor) while preserving a top-1
accuracy of 76.1% on ImageNet object classification and by compressing a Mask
R-CNN with a 26x factor.

本文提出一种矢量量化方法，以减小卷积神经网络架构的存储占用，能以较小的内存占用提供高精度的图像识别。

And the Bit Goes Down: 重新审视神经网络量化

And the Bit Goes Down: Revisiting the Quantization of Neural Networks

In this work we introduce a differentiable version of the Compositional
Pattern Producing Network, called the DPPN. Unlike a standard CPPN, the
topology of a DPPN is evolved but the weights are learned. A Lamarckian
algorithm, that combines evolution and learning, produces DPPNs to reconstruct
an image. Our main result is that DPPNs can be evolved/trained to compress the
weights of a denoising autoencoder from 157684 to roughly 200 parameters, while
achieving a reconstruction accuracy comparable to a fully connected network
with more than two orders of magnitude more parameters. The regularization
ability of the DPPN allows it to rediscover (approximate) convolutional network
architectures embedded within a fully connected architecture. Such
convolutional architectures are the current state of the art for many computer
vision applications, so it is satisfying that DPPNs are capable of discovering
this structure rather than having to build it in by design. DPPNs exhibit
better generalization when tested on the Omniglot dataset after being trained
on MNIST, than directly encoded fully connected autoencoders. DPPNs are
therefore a new framework for integrating learning and evolution.

介绍了可微分的 “组合式模式生成网络”（DPPN），并利用 Lamarckian 算法组合进化和学习，通过学习产生重建图像，并创造性地应用于压缩 denoising autoencoder 的权重和 rediscove 全连接的卷积神经网络结构，具有更好的泛化能力。