In this paper, we propose to learn temporal embeddings of video frames for complex video analysis. Large quantities of unlabeled video data can be easily obtained from the Internet. These videos possess the implicit weak label that they are sequences of temporally and semantically coherent images. We leverage this information to learn temporal embeddings for video frames by associating frames with the temporal context that they appear in. To do this, we propose a scheme for incorporating temporal context based on past and future frames in videos, and compare this to other contextual representations. In addition, we show how data augmentation using multi-resolution samples and hard negatives helps to significantly improve the quality of the learned embeddings. We evaluate various design decisions for learning temporal embeddings, and show that our embeddings can improve performance for multiple video tasks such as retrieval, classification, and temporal order recovery in unconstrained Internet video.

本文提出一种学习视频帧临时嵌入的方法，以便进行复杂的视频分析，利用互联网上大量的未标记的视频数据，通过将帧与其所处的时间上下文相关联来学习视频帧的时间嵌入，并通过使用多分辨率采样和困难负样本的数据增广来显著提高所学嵌入的质量，证明嵌入可以提高多个视频任务的性能，例如在无约束的互联网视频中检索、分类和时间顺序恢复。

为复杂视频分析学习时间嵌入