The CNN-encoding of features from entire videos for the representation of
human actions has rarely been addressed. Instead, CNN work has focused on
approaches to fuse spatial and temporal networks, but these were typically
limited to processing shorter sequences. We present a new video representation,
called temporal linear encoding (TLE) and embedded inside of CNNs as a new
layer, which captures the appearance and motion throughout entire videos. It
encodes this aggregated information into a robust video feature representation,
via end-to-end learning. Advantages of TLEs are: (a) they encode the entire
video into a compact feature representation, learning the semantics and a
discriminative feature space; (b) they are applicable to all kinds of networks
like 2D and 3D CNNs for video classification; and (c) they model feature
interactions in a more expressive way and without loss of information. We
conduct experiments on two challenging human action datasets: HMDB51 and
UCF101. The experiments show that TLE outperforms current state-of-the-art
methods on both datasets.

该研究介绍了一种新的视频表示方法 —— 时间线性编码 (TLE)，通过端到端学习的方式，将视频中的外观和动作全局编码到一种紧凑的特征表示中，可以应用于 2D 和 3D CNNs，并且在人类动作识别任务上表现更好。