It is widely believed that deep neural networks contain layer specialization,
wherein neural networks extract hierarchical features representing edges and
patterns in shallow layers and complete objects in deeper layers. Unlike common
feed-forward models that have distinct filters at each layer, recurrent
networks reuse the same parameters at various depths. In this work, we observe
that recurrent models exhibit the same hierarchical behaviors and the same
performance benefits with depth as feed-forward networks despite reusing the
same filters at every recurrence. By training models of various feed-forward
and recurrent architectures on several datasets for image classification as
well as maze solving, we show that recurrent networks have the ability to
closely emulate the behavior of non-recurrent deep models, often doing so with
far fewer parameters.

本文研究了深度神经网络的层次结构特征，发现递归神经网络与前馈网络在提取特征和处理深层次信息方面表现出类似的表现和性能优越，且递归模型能够模拟前馈模型的行为并使用更少的参数来实现图像分类和迷宫问题的求解。

递归和深度的惊人相似性

The Uncanny Similarity of Recurrence and Depth

Do convolutional networks really need a fixed feed-forward structure? What
if, after identifying the high-level concept of an image, a network could move
directly to a layer that can distinguish fine-grained differences? Currently, a
network would first need to execute sometimes hundreds of intermediate layers
that specialize in unrelated aspects. Ideally, the more a network already knows
about an image, the better it should be at deciding which layer to compute
next. In this work, we propose convolutional networks with adaptive inference
graphs (ConvNet-AIG) that adaptively define their network topology conditioned
on the input image. Following a high-level structure similar to residual
networks (ResNets), ConvNet-AIG decides for each input image on the fly which
layers are needed. In experiments on ImageNet we show that ConvNet-AIG learns
distinct inference graphs for different categories. Both ConvNet-AIG with 50
and 101 layers outperform their ResNet counterpart, while using 20% and 38%
less computations respectively. By grouping parameters into layers for related
classes and only executing relevant layers, ConvNet-AIG improves both
efficiency and overall classification quality. Lastly, we also study the effect
of adaptive inference graphs on the susceptibility towards adversarial
examples. We observe that ConvNet-AIG shows a higher robustness than ResNets,
complementing other known defense mechanisms.

该文提出了一种自适应推理图的卷积网络 (CNN)，根据输入图像动态定义其网络拓扑结构，通过将相关类别的参数分组到一起并仅执行相关层，从而提高了效率和分类质量，并对自适应推理图对抗情况的鲁棒性进行了研究。