Human activity recognition based on video streams has received numerous
attentions in recent years. Due to lack of depth information, RGB video based
activity recognition performs poorly compared to RGB-D video based solutions.
On the other hand, acquiring depth information, inertia etc. is costly and
requires special equipment, whereas RGB video streams are available in ordinary
cameras. Hence, our goal is to investigate whether similar or even higher
accuracy can be achieved with RGB-only modality. In this regard, we propose a
novel framework that couples skeleton data extracted from RGB video and deep
Bidirectional Long Short Term Memory (BLSTM) model for activity recognition. A
big challenge of training such a deep network is the limited training data, and
exploring RGB-only stream significantly exaggerates the difficulty. We
therefore propose a set of algorithmic techniques to train this model
effectively, e.g., data augmentation, dynamic frame dropout and gradient
injection. The experiments demonstrate that our RGB-only solution surpasses the
state-of-the-art approaches that all exploit RGB-D video streams by a notable
margin. This makes our solution widely deployable with ordinary cameras.

该研究提出了一种基于 RGB 视频的人体活动识别深度学习模型，配合骨架数据以及算法技术（如数据增强，动态帧随机失活和梯度注入）来训练模型，实验结果表明该模型能够超越现有采用 RGB-D 视频流所采用的解决方案。