We introduce Ponymation, a new method for learning a generative model of
articulated 3D animal motions from raw, unlabeled online videos. Unlike
existing approaches for motion synthesis, our model does not require any pose
annotations or parametric shape models for training, and is learned purely from
a collection of raw video clips obtained from the Internet. We build upon a
recent work, MagicPony, which learns articulated 3D animal shapes purely from
single image collections, and extend it on two fronts. First, instead of
training on static images, we augment the framework with a video training
pipeline that incorporates temporal regularizations, achieving more accurate
and temporally consistent reconstructions. Second, we learn a generative model
of the underlying articulated 3D motion sequences via a spatio-temporal
transformer VAE, simply using 2D reconstruction losses without relying on any
explicit pose annotations. At inference time, given a single 2D image of a new
animal instance, our model reconstructs an articulated, textured 3D mesh, and
generates plausible 3D animations by sampling from the learned motion latent
space.

我们引入了 Ponymation，一种从原始、未标记的在线视频中学习生成性模型的方法，该方法不需要任何姿势注释或参数化形状模型进行训练，并且纯粹是从从互联网获得的一系列原始视频片段中学习的。