Choreographers determine what the dances look like, while cameramen determine
the final presentation of dances. Recently, various methods and datasets have
showcased the feasibility of dance synthesis. However, camera movement
synthesis with music and dance remains an unsolved challenging problem due to
the scarcity of paired data. Thus, we present DCM, a new multi-modal 3D
dataset, which for the first time combines camera movement with dance motion
and music audio. This dataset encompasses 108 dance sequences (3.2 hours) of
paired dance-camera-music data from the anime community, covering 4 music
genres. With this dataset, we uncover that dance camera movement is
multifaceted and human-centric, and possesses multiple influencing factors,
making dance camera synthesis a more challenging task compared to camera or
dance synthesis alone. To overcome these difficulties, we propose
DanceCamera3D, a transformer-based diffusion model that incorporates a novel
body attention loss and a condition separation strategy. For evaluation, we
devise new metrics measuring camera movement quality, diversity, and dancer
fidelity. Utilizing these metrics, we conduct extensive experiments on our DCM
dataset, providing both quantitative and qualitative evidence showcasing the
effectiveness of our DanceCamera3D model. Code and video demos are available at
this https URL

通过提出多模态三维数据集 DCM 和基于 Transformer 的扩散模型 DanceCamera3D，综合定量和定性证据显示了 DanceCamera3D 模型的有效性，以解决舞蹈摄像机运动与音乐和舞蹈的综合问题。