Semi-Supervised Learning can be more beneficial for the video domain compared
to images because of its higher annotation cost and dimensionality. Besides,
any video understanding task requires reasoning over both spatial and temporal
dimensions. In order to learn both the static and motion related features for
the semi-supervised action recognition task, existing methods rely on hard
input inductive biases like using two-modalities (RGB and Optical-flow) or
two-stream of different playback rates. Instead of utilizing unlabeled videos
through diverse input streams, we rely on self-supervised video
representations, particularly, we utilize temporally-invariant and
temporally-distinctive representations. We observe that these representations
complement each other depending on the nature of the action. Based on this
observation, we propose a student-teacher semi-supervised learning framework,
TimeBalance, where we distill the knowledge from a temporally-invariant and a
temporally-distinctive teacher. Depending on the nature of the unlabeled video,
we dynamically combine the knowledge of these two teachers based on a novel
temporal similarity-based reweighting scheme. Our method achieves
state-of-the-art performance on three action recognition benchmarks: UCF101,
HMDB51, and Kinetics400. Code: this https URL

文章通过自监督学习方法，提出了一个基于时间均衡的学生 - 教师半监督学习框架，动态地结合具有时变不变性和时变区别性的两个教师的知识，从而取得了 UCF101，HMDB51 和 Kinetics400 三个动作识别基准数据集的最新性能。