Learning portable neural networks is very essential for computer vision for
the purpose that pre-trained heavy deep models can be well applied on edge
devices such as mobile phones and micro sensors. Most existing deep neural
network compression and speed-up methods are very effective for training
compact deep models, when we can directly access the training dataset. However,
training data for the given deep network are often unavailable due to some
practice problems (e.g. privacy, legal issue, and transmission), and the
architecture of the given network are also unknown except some interfaces. To
this end, we propose a novel framework for training efficient deep neural
networks by exploiting generative adversarial networks (GANs). To be specific,
the pre-trained teacher networks are regarded as a fixed discriminator and the
generator is utilized for derivating training samples which can obtain the
maximum response on the discriminator. Then, an efficient network with smaller
model size and computational complexity is trained using the generated data and
the teacher network, simultaneously. Efficient student networks learned using
the proposed Data-Free Learning (DAFL) method achieve 92.22% and 74.47%
accuracies using ResNet-18 without any training data on the CIFAR-10 and
CIFAR-100 datasets, respectively. Meanwhile, our student network obtains an
80.56% accuracy on the CelebA benchmark.

提出基于生成对抗网络（GANs）的深度神经网络的无数据训练方法，将预训练的教师网络视为固定的鉴别器，利用生成器派生的训练样本和教师网络一起训练出模型较小和复杂度较低的有效网络。通过 Data-Free Learning（DAFL）方法学习，使用 ResNet-18 在 CIFAR-10 和 CIFAR-100 数据集上实现了 92.22％ 和 74.47％ 的精度，并在 CelebA 基准测试上获得了 80.56％ 的精度。