Automatic choreography generation is a challenging task because it often
requires an understanding of two abstract concepts - music and dance - which
are realized in the two different modalities, namely audio and video,
respectively. In this paper, we propose a music-driven choreography generation
system using an auto-regressive encoder-decoder network. To this end, we first
collect a set of multimedia clips that include both music and corresponding
dance motion. We then extract the joint coordinates of the dancer from video
and the mel-spectrogram of music from audio, and train our network using
music-choreography pairs as input. Finally, a novel dance motion is generated
at the inference time when only music is given as an input. We performed a user
study for a qualitative evaluation of the proposed method, and the results show
that the proposed model is able to generate musically meaningful and natural
dance movements given an unheard song.

通过使用自回归编码解码网络设计了一种音乐驱动舞蹈编排生成系统，该网络利用音乐和对应的舞蹈运动，运用多媒体片段进行训练，能够在只有音乐输入的情况下生成新的舞蹈运动。经过用户研究，结果表明该方法可以生成富有音乐感和自然的新舞蹈动作。