Typical techniques for video captioning follow the encoder-decoder framework,
which can only focus on one source video being processed. A potential
disadvantage of such design is that it cannot capture the multiple visual
context information of a word appearing in more than one relevant videos in
training data. To tackle this limitation, we propose the Memory-Attended
Recurrent Network (MARN) for video captioning, in which a memory structure is
designed to explore the full-spectrum correspondence between a word and its
various similar visual contexts across videos in training data. Thus, our model
is able to achieve a more comprehensive understanding for each word and yield
higher captioning quality. Furthermore, the built memory structure enables our
method to model the compatibility between adjacent words explicitly instead of
asking the model to learn implicitly, as most existing models do. Extensive
validation on two real-word datasets demonstrates that our MARN consistently
outperforms state-of-the-art methods.

提出了一种记忆注意力循环网络用于视频字幕生成，可以在训练数据中探索词与其各种类似视觉上下文的全谱对应关系，从而实现对每个单词的更全面理解，并提高字幕生成质量。