A unique cognitive capability of humans consists in their ability to acquire
new knowledge and skills from a sequence of experiences. Meanwhile, artificial
intelligence systems are good at learning only the last given task without
being able to remember the databases learnt in the past. We propose a novel
lifelong learning methodology by employing a Teacher-Student network framework.
While the Student module is trained with a new given database, the Teacher
module would remind the Student about the information learnt in the past. The
Teacher, implemented by a Generative Adversarial Network (GAN), is trained to
preserve and replay past knowledge corresponding to the probabilistic
representations of previously learn databases. Meanwhile, the Student module is
implemented by a Variational Autoencoder (VAE) which infers its latent variable
representation from both the output of the Teacher module as well as from the
newly available database. Moreover, the Student module is trained to capture
both continuous and discrete underlying data representations across different
domains. The proposed lifelong learning framework is applied in supervised,
semi-supervised and unsupervised training. The code is available~:
https://github.com/dtuzi123/Lifelong-Teacher-Student-Network-Learning

提出了一种基于教师 - 学生网络框架的新型终身学习方法，利用生成对抗网络（GAN）实现教师模块，保留和回放以往学习数据库的概率表示，利用变分自编码器（VAE）实现学生模块，学习捕捉不同领域的连续和离散数据表征，并应用于监督、半监督和无监督培训。