Models based on deep convolutional networks have dominated recent image
interpretation tasks; we investigate whether models which are also recurrent,
or "temporally deep", are effective for tasks involving sequences, visual and
otherwise. We develop a novel recurrent convolutional architecture suitable for
large-scale visual learning which is end-to-end trainable, and demonstrate the
value of these models on benchmark video recognition tasks, image description
and retrieval problems, and video narration challenges. In contrast to current
models which assume a fixed spatio-temporal receptive field or simple temporal
averaging for sequential processing, recurrent convolutional models are "doubly
deep"' in that they can be compositional in spatial and temporal "layers". Such
models may have advantages when target concepts are complex and/or training
data are limited. Learning long-term dependencies is possible when
nonlinearities are incorporated into the network state updates. Long-term RNN
models are appealing in that they directly can map variable-length inputs
(e.g., video frames) to variable length outputs (e.g., natural language text)
and can model complex temporal dynamics; yet they can be optimized with
backpropagation. Our recurrent long-term models are directly connected to
modern visual convnet models and can be jointly trained to simultaneously learn
temporal dynamics and convolutional perceptual representations. Our results
show such models have distinct advantages over state-of-the-art models for
recognition or generation which are separately defined and/or optimized.

研究比较深度卷积网络和带有循环结构的深度卷积神经网络的效果，针对视频识别、图像描述、检索以及视频叙事方面的问题，开发出一种新颖的循环卷积架构，该架构可以训练端到端，可以同时学习时间动态和卷积感知表示，并具有学习长期依赖性的能力。实验结果证明，循环卷积模型在识别或生成方面与现有的模型相比具有明显的优势。