In this paper, we propose a convolutional layer inspired by optical flow
algorithms to learn motion representations. Our representation flow layer is a
fully-differentiable layer designed to capture the `flow' of any representation
channel within a convolutional neural network for action recognition. Its
parameters for iterative flow optimization are learned in an end-to-end fashion
together with the other CNN model parameters, maximizing the action recognition
performance. Furthermore, we newly introduce the concept of learning `flow of
flow' representations by stacking multiple representation flow layers. We
conducted extensive experimental evaluations, confirming its advantages over
previous recognition models using traditional optical flows in both
computational speed and performance. Code/models available here:
this https URL

本文提出了一种卷积层，它能够学习动作表示，我们称之为表示流层，它是一种完全可微分的层，能够捕捉卷积神经网络中任何表示通道的 “流动”，通过迭代流优化过程学习参数，并与其他 CNN 模型参数一起最大化动作识别性能。我们还引入了多个表示流层学习 “流之流” 表示的概念，并通过广泛的实验评估，证实了其优于使用传统光流的先前识别模型在计算速度和性能方面的优势。