In this era of digital information explosion, an abundance of data from
numerous modalities is being generated as well as archived everyday. However,
most problems associated with training Deep Neural Networks still revolve
around lack of data that is rich enough for a given task. Data is required not
only for training an initial model, but also for future learning tasks such as
Model Compression and Incremental Learning. A diverse dataset may be used for
training an initial model, but it may not be feasible to store it throughout
the product life cycle due to data privacy issues or memory constraints. We
propose to bridge the gap between the abundance of available data and lack of
relevant data, for the future learning tasks of a given trained network. We use
the available data, that may be an imbalanced subset of the original training
dataset, or a related domain dataset, to retrieve representative samples from a
trained classifier, using a novel Data-enriching GAN (DeGAN) framework. We
demonstrate that data from a related domain can be leveraged to achieve
state-of-the-art performance for the tasks of Data-free Knowledge Distillation
and Incremental Learning on benchmark datasets. We further demonstrate that our
proposed framework can enrich any data, even from unrelated domains, to make it
more useful for the future learning tasks of a given network.

本论文提出了一种名为 Data-enriching GAN 的框架，可以利用来自不同领域的数据来提高已经训练好的深度神经网络在无数据和增量学习任务上的表现。