The crux of self-supervised video representation learning is to build general
features from unlabeled videos. However, most recent works have mainly focused
on high-level semantics and neglected lower-level representations and their
temporal relationship which are crucial for general video understanding. To
address these challenges, this paper proposes a multi-level feature
optimization framework to improve the generalization and temporal modeling
ability of learned video representations. Concretely, high-level features
obtained from naive and prototypical contrastive learning are utilized to build
distribution graphs, guiding the process of low-level and mid-level feature
learning. We also devise a simple temporal modeling module from multi-level
features to enhance motion pattern learning. Experiments demonstrate that
multi-level feature optimization with the graph constraint and temporal
modeling can greatly improve the representation ability in video understanding.
Code is available at
this https URL

本文提出了一种多层次特征优化框架，利用高级别特征构建分布图并指导低中级别特征学习。同时，设计了简单的时间建模模块以增强运动模式学习，并证明了该方法可以大大提高视频理解中的表示能力。