This paper presents TCE: Temporally Coherent Embeddings for self-supervised video representation learning. The proposed method exploits inherent structure of unlabeled video data to explicitly enforce temporal coherency in the embedding space, rather than indirectly learning it through ranking or predictive pretext tasks. In the same way that high-level visual information in the world changes smoothly, we believe that nearby frames in learned representations should demonstrate similar properties. Using this assumption, we train the TCE model to encode videos such that adjacent frames exist close to each other and videos are separated from one another. Using TCE we learn robust representations from large quantities of unlabeled video data. We evaluate our self-supervised trained TCE model by adding a classification layer and finetuning the learned representation on the downstream task of video action recognition on the UCF101 dataset. We obtain 68.7% accuracy and outperform the state-of-the-art self-supervised methods despite using a significantly smaller dataset for pre-training. Notably, we demonstrate results competitive with more complex 3D-CNN based networks while training with a 2D-CNN network backbone on action recognition tasks.

TCE是一种自监督学习方法，利用无标签视频数据的内在结构在嵌入空间中明确强制执行时间上的连贯性，以便从大量无标签视频数据中学习出稳健的表示，该方法经过了对多个数据集的评估，能够继续改进视频动作识别的性能。

自监督视频表示学习的时间连续嵌入