Recent techniques for text-to-4D generation synthesize dynamic 3D scenes
using supervision from pre-trained text-to-video models. However, existing
representations for motion, such as deformation models or time-dependent neural
representations, are limited in the amount of motion they can generate-they
cannot synthesize motion extending far beyond the bounding box used for volume
rendering. The lack of a more flexible motion model contributes to the gap in
realism between 4D generation methods and recent, near-photorealistic video
generation models. Here, we propose TC4D: trajectory-conditioned text-to-4D
generation, which factors motion into global and local components. We represent
the global motion of a scene's bounding box using rigid transformation along a
trajectory parameterized by a spline. We learn local deformations that conform
to the global trajectory using supervision from a text-to-video model. Our
approach enables the synthesis of scenes animated along arbitrary trajectories,
compositional scene generation, and significant improvements to the realism and
amount of generated motion, which we evaluate qualitatively and through a user
study. Video results can be viewed on our website:
this https URL

通过轨迹条件下的文本到 4D 生成，我们提出了一种将运动分解为全局和局部组件的方法，该方法通过文本到视频模型的监督来学习全局轨迹参数化的边界框的刚性变换以及适应于全局轨迹的局部变形，并在合成场景的任意轨迹上产生动画场景，从而提高了合成场景的逼真度和生成运动的数量。