Quantization has emerged as a promising direction for model compression.
Recently, data-free quantization has been widely studied as a promising method
to avoid privacy concerns, which synthesizes images as an alternative to real
training data. Existing methods use classification loss to ensure the
reliability of the synthesized images. Unfortunately, even if these images are
well-classified by the pre-trained model, they still suffer from low semantics
and homogenization issues. Intuitively, these low-semantic images are sensitive
to perturbations, and the pre-trained model tends to have inconsistent output
when the generator synthesizes an image with poor semantics. To this end, we
propose Robustness-Guided Image Synthesis (RIS), a simple but effective method
to enrich the semantics of synthetic images and improve image diversity,
further boosting the performance of downstream data-free compression tasks.
Concretely, we first introduce perturbations on input and model weight, then
define the inconsistency metrics at feature and prediction levels before and
after perturbations. On the basis of inconsistency on two levels, we design a
robustness optimization objective to enhance the semantics of synthetic images.
Moreover, we also make our approach diversity-aware by forcing the generator to
synthesize images with small correlations in the label space. With RIS, we
achieve state-of-the-art performance for various settings on data-free
quantization and can be extended to other data-free compression tasks.

我们提出了一种简单但有效的方法，即稳健导向的图像合成（RIS），通过在输入和模型权重上引入扰动，并在扰动前后在特征和预测水平上定义不一致性指标，设计一个稳健优化目标来增强合成图像的语义，进而提高图像多样性，从而在无数据压缩任务中取得了最先进的性能。