This work presents a self-supervised learning framework named TeG to explore
Temporal Granularity in learning video representations. In TeG, we sample a
long clip from a video and a short clip that lies inside the long clip. We then
extract their dense temporal embeddings. The training objective consists of two
parts: a fine-grained temporal learning objective to maximize the similarity
between corresponding temporal embeddings in the short clip and the long clip,
and a persistent temporal learning objective to pull together global embeddings
of the two clips. Our study reveals the impact of temporal granularity with
three major findings. 1) Different video tasks may require features of
different temporal granularities. 2) Intriguingly, some tasks that are widely
considered to require temporal awareness can actually be well addressed by
temporally persistent features. 3) The flexibility of TeG gives rise to
state-of-the-art results on 8 video benchmarks, outperforming supervised
pre-training in most cases.

该研究提出了一种名为 TeG 的自监督学习框架来探索学习视频表示中的时间粒度，其在 8 个视频基准测试中显示出了超越有监督预训练的最新成果。