We introduce a novel approach to endowing neural networks with emergent,
long-term, large-scale memory. Distinct from strategies that connect neural
networks to external memory banks via intricately crafted controllers and
hand-designed attentional mechanisms, our memory is internal, distributed,
co-located alongside computation, and implicitly addressed, while being
drastically simpler than prior efforts. Architecting networks with multigrid
structure and connectivity, while distributing memory cells alongside
computation throughout this topology, we observe the emergence of coherent
memory subsystems. Our hierarchical spatial organization, parameterized
convolutionally, permits efficient instantiation of large-capacity memories,
while multigrid topology provides short internal routing pathways, allowing
convolutional networks to efficiently approximate the behavior of fully
connected networks. Such networks have an implicit capacity for internal
attention; augmented with memory, they learn to read and write specific memory
locations in a dynamic data-dependent manner. We demonstrate these capabilities
on exploration and mapping tasks, where our network is able to self-organize
and retain long-term memory for trajectories of thousands of time steps. On
tasks decoupled from any notion of spatial geometry: sorting, associative
recall, and question answering, our design functions as a truly generic memory
and yields excellent results.

本文提出了一种新颖的方法，为神经网络赋予出现、长期、大规模的内存，该方法针对神经网络与外部存储器库通过精心设计的控制器和手动设计的注意机制进行连接的策略进行了区分。我们的内存是内部、分布式、与计算共存、隐式寻址，而且比之前的方法明显更简单。经过层次化空间组织，参数化卷积，并分布在整个拓扑中的计算的存储单元，我们观察到了一种连贯记忆子系统的出现。