Solving the visual symbol grounding problem has long been a goal of artificial intelligence. The field appears to be advancing closer to this goal with recent breakthroughs in deep learning for natural language grounding in static images. In this paper, we propose to translate videos directly to sentences using a unified deep neural network with both convolutional and recurrent structure. Described video datasets are scarce, and most existing methods have been applied to toy domains with a small vocabulary of possible words. By transferring knowledge from 1.2M+ images with category labels and 100,000+ images with captions, our method is able to create sentence descriptions of open-domain videos with large vocabularies. We compare our approach with recent work using language generation metrics, subject, verb, and object prediction accuracy, and a human evaluation.

本篇论文提出直接利用统一深度神经网络将视频转换为句子的方法，并通过将知识从含标记种类超过120万张图像和带字幕的超过100,000张图像中迁移而创建具有大型词汇库的开放域视频句子描述。通过与语言生成度量，主语，动词和宾语预测准确度以及人的评估进行比较，证明本方法的有效性。

使用深度递归神经网络将视频翻译为自然语言