While machine learning is currently very successful in several application domains, we are still very far from a real Artificial Intelligence. In this paper, we study basic sequence prediction problems that are beyond the scope of what is learnable with popular methods such as recurrent networks. We show that simple algorithms can be learnt from sequential data with a recurrent network associated with trainable stacks. We focus our study on algorithmically generated sequences such as $a^n b^{n}$, that can only be learnt by models which have the capacity to count. Our study highlights certain topics in machine learning that deserve more attention, such as addressing the shortcomings of purely gradient based training of non-convex models. We achieve progress in this direction by incorporating search based strategy. Once trained, we show that our method is able generalize to sequences up to an arbitrary size.

本文讨论了标准深度学习方法的局限性，并展示了如何通过以结构化方式增加模型的复杂性来克服这些限制，具体地，研究了仅适用于具有计数和记忆序列能力模型的算法生成序列的最简单序列预测问题，证明了可以使用与可训练内存相关的循环网络从序列数据中学习一些基本算法。

使用堆栈增强循环网络推断算法模式