Online Action Detection (OAD) in videos is proposed as a per-frame labeling
task to address the real-time prediction tasks that can only obtain the
previous and current video frames. This paper presents a novel
learning-with-privileged based framework for online action detection where the
future frames only observable at the training stages are considered as a form
of privileged information. Knowledge distillation is employed to transfer the
privileged information from the offline teacher to the online student. We note
that this setting is different from conventional KD because the difference
between the teacher and student models mostly lies in input data rather than
the network architecture. We propose Privileged Knowledge Distillation (PKD)
which (i) schedules a curriculum learning procedure and (ii) inserts auxiliary
nodes to the student model, both for shrinking the information gap and
improving learning performance. Compared to other OAD methods that explicitly
predict future frames, our approach avoids learning unpredictable unnecessary
yet inconsistent visual contents and achieves state-of-the-art accuracy on two
popular OAD benchmarks, TVSeries and THUMOS14.

本文提出了一种基于特权信息学习的在线动作检测框架，其中包括知识蒸馏方法和 Privileged Knowledge Distillation (PKD) 方法，以缩小信息差并提高学习性能，并在两个常用的 OAD 基准测试 TVSeries 和 THUMOS14 上取得了当前最佳性能.