We present a novel framework that brings the 3D motion retargeting task from controlled environments to in-the-wild scenarios. In particular, our method is capable of retargeting body motion from a character in a 2D monocular video to a 3D character without using any motion capture system or 3D reconstruction procedure. It is designed to leverage massive online videos for unsupervised training, needless of 3D annotations or motion-body pairing information. The proposed method is built upon two novel canonicalization operations, structure canonicalization and view canonicalization. Trained with the canonicalization operations and the derived regularizations, our method learns to factorize a skeleton sequence into three independent semantic subspaces, i.e., motion, structure, and view angle. The disentangled representation enables motion retargeting from 2D to 3D with high precision. Our method achieves superior performance on motion transfer benchmarks with large body variations and challenging actions. Notably, the canonicalized skeleton sequence could serve as a disentangled and interpretable representation of human motion that benefits action analysis and motion retrieval.

该研究提出了一种新的框架，通过结构和视图标准化操作，将3D动作重定位任务从控制环境转移到野外场景。该方法利用在线视频进行非监督训练，不需要使用任何运动捕捉系统或3D重建程序，从2D视频中重定位身体动作到3D角色，具有高精度的2D到3D动作转换性能，并且产生的规范化骨架序列可用于人类运动的可解释表示。

通过规范化网络实现野外动作再定位的MoCaNet