Intelligent behaviour in the real-world requires the ability to acquire new knowledge from an ongoing sequence of experiences while preserving and reusing past knowledge. We propose a novel algorithm for unsupervised representation learning from piece-wise stationary visual data: Variational Autoencoder with Shared Embeddings (VASE). Based on the Minimum Description Length principle, VASE automatically detects shifts in the data distribution and allocates spare representational capacity to new knowledge, while simultaneously protecting previously learnt representations from catastrophic forgetting. Our approach encourages the learnt representations to be disentangled, which imparts a number of desirable properties: VASE can deal sensibly with ambiguous inputs, it can enhance its own representations through imagination-based exploration, and most importantly, it exhibits semantically meaningful sharing of latents between different datasets. Compared to baselines with entangled representations, our approach is able to reason beyond surface-level statistics and perform semantically meaningful cross-domain inference.

本文提出了一种基于最小描述长度原则的变分自编码器算法，可以实现无监督表示学习，并可以自动检测数据分布的变化并为新知识分配足够的表示空间，同时保护以前学习的表示免于灾难性遗忘，该方法可以实现表示的分离，具有共享潜在特征的语义上有意义的特征，同时可以跨领域共享经验。

跨域潜在同源的终身解缠结表示学习