Generative Adversarial Networks (GANs) have achieved huge success in
generating high-fidelity images, however, they suffer from low efficiency due
to tremendous computational cost and bulky memory usage. Recent efforts on
compression GANs show noticeable progress in obtaining smaller generators by
sacrificing image quality or involving a time-consuming searching process. In
this work, we aim to address these issues by introducing a teacher network that
provides a search space in which efficient network architectures can be found,
in addition to performing knowledge distillation. First, we revisit the search
space of generative models, introducing an inception-based residual block into
generators. Second, to achieve target computation cost, we propose a one-step
pruning algorithm that searches a student architecture from the teacher model
and substantially reduces searching cost. It requires no l1 sparsity
regularization and its associated hyper-parameters, simplifying the training
procedure. Finally, we propose to distill knowledge through maximizing feature
similarity between teacher and student via an index named Global Kernel
Alignment (GKA). Our compressed networks achieve similar or even better image
fidelity (FID, mIoU) than the original models with much-reduced computational
cost, e.g., MACs. Code will be released at
this https URL

通过引入教师网络，在搜索空间中查找高效的网络架构并进行知识蒸馏，该方法可以在更小的计算成本下实现类似于原始模型的图像质量，从而可能解决生成对抗网络的计算成本高的问题。