Recent research developing neural network architectures with external memory
have often used the benchmark bAbI question and answering dataset which
provides a challenging number of tasks requiring reasoning. Here we employed a
classic associative inference task from the memory-based reasoning neuroscience
literature in order to more carefully probe the reasoning capacity of existing
memory-augmented architectures. This task is thought to capture the essence of
reasoning -- the appreciation of distant relationships among elements
distributed across multiple facts or memories. Surprisingly, we found that
current architectures struggle to reason over long distance associations.
Similar results were obtained on a more complex task involving finding the
shortest path between nodes in a path. We therefore developed MEMO, an
architecture endowed with the capacity to reason over longer distances. This
was accomplished with the addition of two novel components. First, it
introduces a separation between memories (facts) stored in external memory and
the items that comprise these facts in external memory. Second, it makes use of
an adaptive retrieval mechanism, allowing a variable number of "memory hops"
before the answer is produced. MEMO is capable of solving our novel reasoning
tasks, as well as match state of the art results in bAbI.

发展神经网络结构与外部存储器的最近研究，为了更仔细地探究现有记忆增强框架的推理能力，我们使用了以前认知神经科学文献中的经典关联推断任务，其捕捉跨越多个事实或记忆分布的元素之间的远距离关系，结果表明当前架构在处理长距离关联推理时存在困难，因此，我们开发了 MEMO，这是一种能够处理长距离关联推理的体系结构，它包括两个新的组件：一是在外部存储器中存储记忆体（事实）和组成这些记忆体的项之间的分离，二是采用自适应检索机制，在产生答案之前允许可变数量的 “记忆体跳跃”，MEMO 能够解决我们的新推理任务，并且匹配 bAbI 的最新结果。