Dance experts often view dance as a hierarchy of information, spanning
low-level (raw images, image sequences), mid-levels (human poses and bodypart
movements), and high-level (dance genre). We propose a Hierarchical Dance Video
Recognition framework (HDVR). HDVR estimates 2D pose sequences, tracks dancers,
and then simultaneously estimates corresponding 3D poses and 3D-to-2D imaging
parameters, without requiring ground truth for 3D poses. Unlike most methods
that work on a single person, our tracking works on multiple dancers, under
occlusions. From the estimated 3D pose sequence, HDVR extracts body part
movements, and therefrom dance genre. The resulting hierarchical dance
representation is explainable to experts. To overcome noise and interframe
correspondence ambiguities, we enforce spatial and temporal motion smoothness
and photometric continuity over time. We use an LSTM network to extract 3D
movement subsequences from which we recognize the dance genre. For experiments,
we have identified 154 movement types, of 16 body parts, and assembled a new
University of Illinois Dance (UID) Dataset, containing 1143 video clips of 9
genres covering 30 hours, annotated with movement and genre labels. Our
experimental results demonstrate that our algorithms outperform the
state-of-the-art 3D pose estimation methods, which also enhances our dance
recognition performance.

提出了一种分层舞蹈视频识别框架 (HDVR)，它通过估计 2D 姿势序列，跟踪舞者，并同时估计相应的 3D 姿势和 3D 到 2D 成像参数来提取舞蹈的层次结构。我们在多人跟踪和通过 LSTM 网络识别舞蹈类型方面优于现有的 3D 姿势估计方法。