We propose a novel unsupervised method to autoregressively generate videos
from a single frame and a sparse motion input. Our trained model can generate
realistic object-to-object interactions and separate the dynamics and the
extents of multiple objects despite only observing them under correlated motion
activities. Key components in our method are the randomized conditioning
scheme, the encoding of the input motion control, and the randomized and sparse
sampling to break correlations. Our model, which we call YODA, has the ability
to move objects without physically touching them. We show both qualitatively
and quantitatively that YODA accurately follows the user control, while
yielding a video quality that is on par with or better than state of the art
video generation prior work on several datasets. For videos, visit our project
website this https URL

我们提出了一种新颖的无监督方法，可以根据单个帧和稀疏动作输入自回归地生成视频，该方法能够生成逼真的物体间相互作用，并能在只观察到它们在相关动作活动下运动时分离多个物体的动力学和幅度；我们的模型 YODA 能够移动物体而不用直接接触它们，定性和定量实验表明，YODA 在几个数据集上的视频生成效果不仅与现有领先工作相当，而且表现更优秀。