We present Neural Marionette, an unsupervised approach that discovers the
skeletal structure from a dynamic sequence and learns to generate diverse
motions that are consistent with the observed motion dynamics. Given a video
stream of point cloud observation of an articulated body under arbitrary
motion, our approach discovers the unknown low-dimensional skeletal
relationship that can effectively represent the movement. Then the discovered
structure is utilized to encode the motion priors of dynamic sequences in a
latent structure, which can be decoded to the relative joint rotations to
represent the full skeletal motion. Our approach works without any prior
knowledge of the underlying motion or skeletal structure, and we demonstrate
that the discovered structure is even comparable to the hand-labeled ground
truth skeleton in representing a 4D sequence of motion. The skeletal structure
embeds the general semantics of possible motion space that can generate motions
for diverse scenarios. We verify that the learned motion prior is generalizable
to the multi-modal sequence generation, interpolation of two poses, and motion
retargeting to a different skeletal structure.

Neural Marionette 是一种无监督学习方法，可以从动态序列中发现骨架结构，学会生成与观察到的运动动态一致的多样化运动，无需先验知识，可泛化到多模态序列生成、两个姿势的插值和运动重新定向。