We study the problem of learning associative memory -- a system which is able
to retrieve a remembered pattern based on its distorted or incomplete version.
Attractor networks provide a sound model of associative memory: patterns are
stored as attractors of the network dynamics and associative retrieval is
performed by running the dynamics starting from a query pattern until it
converges to an attractor. In such models the dynamics are often implemented as
an optimization procedure that minimizes an energy function, such as in the
classical Hopfield network. In general it is difficult to derive a writing rule
for a given dynamics and energy that is both compressive and fast. Thus, most
research in energy-based memory has been limited either to tractable energy
models not expressive enough to handle complex high-dimensional objects such as
natural images, or to models that do not offer fast writing. We present a novel
meta-learning approach to energy-based memory models (EBMM) that allows one to
use an arbitrary neural architecture as an energy model and quickly store
patterns in its weights. We demonstrate experimentally that our EBMM approach
can build compressed memories for synthetic and natural data, and is capable of
associative retrieval that outperforms existing memory systems in terms of the
reconstruction error and compression rate.

本文提出了一种基于元学习的能量记忆模型，通过使用任意神经结构作为能量模型并快速将图案存储在其权重中，实现了对合成和自然数据的压缩记忆，并在重构误差和压缩率方面优于现有的记忆系统。

元学习深度能量型记忆模型

Meta-Learning Deep Energy-Based Memory Models

People learn throughout life. However, incrementally updating conventional
neural networks leads to catastrophic forgetting. A common remedy is replay,
which is inspired by how the brain consolidates memory. Replay involves
fine-tuning a network on a mixture of new and old instances. While there is
neuroscientific evidence that the brain replays compressed memories, existing
methods for convolutional networks replay raw images. Here, we propose REMIND,
a brain-inspired approach that enables efficient replay with compressed
representations. REMIND is trained in an online manner, meaning it learns one
example at a time, which is closer to how humans learn. Under the same
constraints, REMIND outperforms other methods for incremental class learning on
the ImageNet ILSVRC-2012 dataset. We probe REMIND's robustness to data ordering
schemes known to induce catastrophic forgetting. We demonstrate REMIND's
generality by pioneering online learning for Visual Question Answering (VQA).

提出 REMIND 方法，用于神经网络的在线学习。该方法以压缩的记忆方式实现了网络的更新，达到了降低过去学习内容遗忘的目的，并在图像分类和视觉问答任务中表现出了优越性。