We present an end-to-end trained memory system that quickly adapts to new data and generates samples like them. Inspired by Kanerva's sparse distributed memory, it has a robust distributed reading and writing mechanism. The memory is analytically tractable, which enables optimal on-line compression via a Bayesian update-rule. We formulate it as a hierarchical conditional generative model, where memory provides a rich data-dependent prior distribution. Consequently, the top-down memory and bottom-up perception are combined to produce the code representing an observation. Empirically, we demonstrate that the adaptive memory significantly improves generative models trained on both the Omniglot and CIFAR datasets. Compared with the Differentiable Neural Computer (DNC) and its variants, our memory model has greater capacity and is significantly easier to train.

本研究提出了一种端到端训练的内存系统，可以快速适应新数据并生成类似的样本。它具有强大的分布式读写机制和可分析的内存，可以通过贝叶斯更新规则进行最优在线压缩，以内存作为数据依赖的先验概率分布的层次性条件生成模型，将自适应内存与自底向上感知相结合产生表示观察的代码。经验证明，与Differentiable Neural Computer (DNC)及其变种相比，我们的内存模型具有更高的容量和更容易训练。

Kanerva 机器：一种生成式分布式记忆