Many time-series classification problems involve developing metrics that are
invariant to temporal misalignment. In human activity analysis, temporal
misalignment arises due to various reasons including differing initial phase,
sensor sampling rates, and elastic time-warps due to subject-specific
biomechanics. Past work in this area has only looked at reducing intra-class
variability by elastic temporal alignment. In this paper, we propose a hybrid
model-based and data-driven approach to learn warping functions that not just
reduce intra-class variability, but also increase inter-class separation. We
call this a temporal transformer network (TTN). TTN is an interpretable
differentiable module, which can be easily integrated at the front end of a
classification network. The module is capable of reducing intra-class variance
by generating input-dependent warping functions which lead to rate-robust
representations. At the same time, it increases inter-class variance by
learning warping functions that are more discriminative. We show improvements
over strong baselines in 3D action recognition on challenging datasets using
the proposed framework. The improvements are especially pronounced when
training sets are smaller.

提出了一种基于混合模型和数据驱动的时间变形网络 (TTN)，可用于降低类内差异并提高类间差异，在 3D 动作识别中获得了显著的改进。

时间变换网络：不变性和区分性时间弯曲的联合学习

Temporal Transformer Networks: Joint Learning of Invariant and  Discriminative Time Warping

Recent approaches in depth-based human activity analysis achieved outstanding
performance and proved the effectiveness of 3D representation for
classification of action classes. Currently available depth-based and
RGB+D-based action recognition benchmarks have a number of limitations,
including the lack of training samples, distinct class labels, camera views and
variety of subjects. In this paper we introduce a large-scale dataset for RGB+D
human action recognition with more than 56 thousand video samples and 4 million
frames, collected from 40 distinct subjects. Our dataset contains 60 different
action classes including daily, mutual, and health-related actions. In
addition, we propose a new recurrent neural network structure to model the
long-term temporal correlation of the features for each body part, and utilize
them for better action classification. Experimental results show the advantages
of applying deep learning methods over state-of-the-art hand-crafted features
on the suggested cross-subject and cross-view evaluation criteria for our
dataset. The introduction of this large scale dataset will enable the community
to apply, develop and adapt various data-hungry learning techniques for the
task of depth-based and RGB+D-based human activity analysis.

本文介绍了一个包含超过 56,000 个视频样本和 4 百万帧的大规模 RGB + D 人体动作识别数据集，其中包括 60 种不同的行为类别，并提出了一种新的循环神经网络结构来更好地对每个身体部位的特征进行长期时间相关性建模。实验结果表明，在跨主体和跨视图评估标准上，应用深度学习方法比最先进的手工制作特征具有优势。