Dance challenges are going viral in video communities like TikTok nowadays.
Once a challenge becomes popular, thousands of short-form videos will be
uploaded in merely a couple of days. Therefore, virality prediction from dance
challenges is of great commercial value and has a wide range of applications,
such as smart recommendation and popularity promotion. In this paper, a novel
multi-modal framework which integrates skeletal, holistic appearance, facial
and scenic cues is proposed for comprehensive dance virality prediction. To
model body movements, we propose a pyramidal skeleton graph convolutional
network (PSGCN) which hierarchically refines spatio-temporal skeleton graphs.
Meanwhile, we introduce a relational temporal convolutional network (RTCN) to
exploit appearance dynamics with non-local temporal relations. An attentive
fusion approach is finally proposed to adaptively aggregate predictions from
different modalities. To validate our method, we introduce a large-scale viral
dance video (VDV) dataset, which contains over 4,000 dance clips of eight viral
dance challenges. Extensive experiments on the VDV dataset demonstrate the
efficacy of our model. Extensive experiments on the VDV dataset well
demonstrate the effectiveness of our approach. Furthermore, we show that short
video applications like multi-dimensional recommendation and action feedback
can be derived from our model.

本研究使用多种视觉和场景信息，提出了一个新的多模态框架用于全面预测舞蹈挑战的传播趋势。使用金字塔骨骼图卷积网络（PSGCN）模拟身体动作，并引入关系时间卷积网络（RTCN）来利用非局部时间关系进行外观动态建模。提出了一种关注融合方法，最终从不同模态自适应聚合预测。对大规模病毒舞蹈视频数据集进行了实验验证，证明了我们模型的有效性。另外，展示了该模型的多维度推荐和行动反馈等应用。