It is widely believed that deep neural networks contain layer specialization,
wherein neural networks extract hierarchical features representing edges and
patterns in shallow layers and complete objects in deeper layers. Unlike common
feed-forward models that have distinct filters at each layer, recurrent
networks reuse the same parameters at various depths. In this work, we observe
that recurrent models exhibit the same hierarchical behaviors and the same
performance benefits with depth as feed-forward networks despite reusing the
same filters at every recurrence. By training models of various feed-forward
and recurrent architectures on several datasets for image classification as
well as maze solving, we show that recurrent networks have the ability to
closely emulate the behavior of non-recurrent deep models, often doing so with
far fewer parameters.

本文研究了深度神经网络的层次结构特征，发现递归神经网络与前馈网络在提取特征和处理深层次信息方面表现出类似的表现和性能优越，且递归模型能够模拟前馈模型的行为并使用更少的参数来实现图像分类和迷宫问题的求解。