Self-supervised video representation learning aimed at maximizing similarity between different temporal segments of one video, in order to enforce feature persistence over time. This leads to loss of pertinent information related to temporal relationships, rendering actions such as `enter' and `leave' to be indistinguishable. To mitigate this limitation, we propose Latent Time Navigation (LTN), a time-parameterized contrastive learning strategy that is streamlined to capture fine-grained motions. Specifically, we maximize the representation similarity between different video segments from one video, while maintaining their representations time-aware along a subspace of the latent representation code including an orthogonal basis to represent temporal changes. Our extensive experimental analysis suggests that learning video representations by LTN consistently improves performance of action classification in fine-grained and human-oriented tasks (e.g., on Toyota Smarthome dataset). In addition, we demonstrate that our proposed model, when pre-trained on Kinetics-400, generalizes well onto the unseen real world video benchmark datasets UCF101 and HMDB51, achieving state-of-the-art performance in action recognition.

本文提出了一种时间参数对比学习策略Latent Time Navigation (LTN)，以捕捉细微的运动，从而增强视频表示学习模型中时间感知特性，使模型在fine-grained和面向人类的任务中的分类性能得到了显著提高。同时，在预训练于Kinetics-400数据集的基础上，该模型在UCF101和HMDB51数据集上也取得了state-of-the-art的行动识别性能。

通过潜在时间导航进行自监督视频表示学习