Stack-augmented recurrent neural networks (RNNs) have been of interest to the
deep learning community for some time. However, the difficulty of training
memory models remains a problem obstructing the widespread use of such models.
In this paper, we propose the Ordered Memory architecture. Inspired by Ordered
Neurons (Shen et al., 2018), we introduce a new attention-based mechanism and
use its cumulative probability to control the writing and erasing operation of
the memory. We also introduce a new Gated Recursive Cell to compose lower-level
representations into higher-level representation. We demonstrate that our model
achieves strong performance on the logical inference task (Bowman et al.,
2015)and the ListOps (Nangia and Bowman, 2018) task. We can also interpret the
model to retrieve the induced tree structure, and find that these induced
structures align with the ground truth. Finally, we evaluate our model on the
Stanford SentimentTreebank tasks (Socher et al., 2013), and find that it
performs comparatively with the state-of-the-art methods in the literature.

本研究提出了一种命令式记忆网络结构（Ordered Memory architecture），通过引入一种新的基于注意力机制的累计概率控制内存的写入和擦除操作，并使用一种新的门控递归单元将低级表示组合成高级表示。在逻辑推理和列表组合任务中获得了强大的表现，并证明其表现与现有方法相当。