Temporal localization remains an important challenge in video understanding.
In this work, we present our solution to the 3rd YouTube-8M Video Understanding
Challenge organized by Google Research. Participants were required to build a
segment-level classifier using a large-scale training data set with noisy
video-level labels and a relatively small-scale validation data set with
accurate segment-level labels. We formulated the problem as a multiple instance
multi-label learning and developed an attention-based mechanism to selectively
emphasize the important frames by attention weights. The model performance is
further improved by constructing multiple sets of attention networks. We
further fine-tuned the model using the segment-level data set. Our final model
consists of an ensemble of attention/multi-attention networks, deep bag of
frames models, recurrent neural networks and convolutional neural networks. It
ranked 13th on the private leader board and stands out for its efficient usage
of resources.

本文提出一种多注意力机制的模型来解决视频理解中的时间定位问题，模型结合了多个注意力网络、深度帧模型、循环神经网络和卷积神经网络，并基于多实例多标签学习和 attention 权重来加强对视频中重要帧的关注，从而在 YouTube-8M Video Understanding Challenge 中取得了较好的成绩。