3D action recognition has broad applications in human-computer interaction
and intelligent surveillance. However, recognizing similar actions remains
challenging since previous literature fails to capture motion and shape cues
effectively from noisy depth data. In this paper, we propose a novel two-layer
Bag-of-Visual-Words (BoVW) model, which suppresses the noise disturbances and
jointly encodes both motion and shape cues. First, background clutter is
removed by a background modeling method that is designed for depth data. Then,
motion and shape cues are jointly used to generate robust and distinctive
spatial-temporal interest points (STIPs): motion-based STIPs and shape-based
STIPs. In the first layer of our model, a multi-scale 3D local steering kernel
(M3DLSK) descriptor is proposed to describe local appearances of cuboids around
motion-based STIPs. In the second layer, a spatial-temporal vector (STV)
descriptor is proposed to describe the spatial-temporal distributions of
shape-based STIPs. Using the Bag-of-Visual-Words (BoVW) model, motion and shape
cues are combined to form a fused action representation. Our model performs
favorably compared with common STIP detection and description methods. Thorough
experiments verify that our model is effective in distinguishing similar
actions and robust to background clutter, partial occlusions and pepper noise.

本文针对噪声深度数据难以捕捉运动和形状线索的现状，提出了一种新颖的两层 “视觉词袋”（BoVW）模型。该模型通过背景建模和引入运动和形状线索生成稳健和独特的时空兴趣点，利用多尺度 3D 局部方向核 (M3DLSK) 描述算法和空间 - 时间向量 (STV) 描述算法，提出融合运动和形状线索的行动表示。实验验证了该模型在识别类似动作和抵抗背景杂波、部分遮挡和掩蔽噪声等情况中的有效性。