Data is one of the most important factors in machine learning. However, even if we have high-quality data, there is a situation in which access to the data is restricted. For example, access to the medical data from outside is strictly limited due to the privacy issues. In this case, we have to learn a model sequentially only with the data accessible in the corresponding stage. In this work, we propose a new method for preserving learned knowledge by modeling the high-level feature space and the output space to be mutually informative, and constraining feature vectors to lie in the modeled space during training. The proposed method is easy to implement as it can be applied by simply adding a reconstruction loss to an objective function. We evaluate the proposed method on CIFAR-10/100 and a chest X-ray dataset, and show benefits in terms of knowledge preservation compared to previous approaches.

为了解决在数据访问受限的情况下学习模型的问题，本文提出了一种简单易实现的方法，通过建模高层次特征空间和输出空间来相互信息，并在训练期间约束特征向量位于建模空间中，以保护已学到的知识，并在CIFAR-10/100数据集和胸透数据集上验证了其有效性。

约束潜空间的保留知识神经网络连续学习