Generative Adversarial Networks (GANs) have been widely-used in image translation, but their high computational and storage costs impede the deployment on mobile devices. Prevalent methods for CNN compression cannot be directly applied to GANs due to the complicated generator architecture and the unstable adversarial training. To solve these, in this paper, we introduce a novel GAN compression method, termed DMAD, by proposing a Differentiable Mask and a co-Attention Distillation. The former searches for a light-weight generator architecture in a training-adaptive manner. To overcome channel inconsistency when pruning the residual connections, an adaptive cross-block group sparsity is further incorporated. The latter simultaneously distills informative attention maps from both the generator and discriminator of a pre-trained model to the searched generator, effectively stabilizing the adversarial training of our light-weight model. Experiments show that DMAD can reduce the Multiply Accumulate Operations (MACs) of CycleGAN by 13x and that of Pix2Pix by 4x while retaining a comparable performance against the full model. Code is available at https://github.com/SJLeo/DMAD.

本研究提出了一种新颖的GAN压缩方法DMAD，通过使用可微分的掩模和协同注意力熔炼来搜索轻量级发生器体系结构，同时从预训练模型的生成器和判别器中提取有用的注意力图，成功稳定轻量级模型的对抗训练，而在保持可比余白性能的前提下，DMAD能够将CycleGAN的MACs降低13倍，Pix2Pix的MACs降低4倍。

通过可微分掩码和共同注意力蒸馏学习图像翻译的高效GAN