Recent techniques for text-to-4D generation synthesize dynamic 3D scenes
using supervision from pre-trained text-to-video models. However, existing
representations for motion, such as deformation models or time-dependent neural
representations, are limited in the amount of motion they can generate-they
cannot synthesize motion extending far beyond the bounding box used for volume
rendering. The lack of a more flexible motion model contributes to the gap in
realism between 4D generation methods and recent, near-photorealistic video
generation models. Here, we propose TC4D: trajectory-conditioned text-to-4D
generation, which factors motion into global and local components. We represent
the global motion of a scene's bounding box using rigid transformation along a
trajectory parameterized by a spline. We learn local deformations that conform
to the global trajectory using supervision from a text-to-video model. Our
approach enables the synthesis of scenes animated along arbitrary trajectories,
compositional scene generation, and significant improvements to the realism and
amount of generated motion, which we evaluate qualitatively and through a user
study. Video results can be viewed on our website:
this https URL

通过轨迹条件下的文本到 4D 生成，我们提出了一种将运动分解为全局和局部组件的方法，该方法通过文本到视频模型的监督来学习全局轨迹参数化的边界框的刚性变换以及适应于全局轨迹的局部变形，并在合成场景的任意轨迹上产生动画场景，从而提高了合成场景的逼真度和生成运动的数量。

TC4D：轨迹条件的文本到 4D 生成

TC4D: Trajectory-Conditioned Text-to-4D Generation

Recent breakthroughs in text-to-4D generation rely on pre-trained
text-to-image and text-to-video models to generate dynamic 3D scenes. However,
current text-to-4D methods face a three-way tradeoff between the quality of
scene appearance, 3D structure, and motion. For example, text-to-image models
and their 3D-aware variants are trained on internet-scale image datasets and
can be used to produce scenes with realistic appearance and 3D structure -- but
no motion. Text-to-video models are trained on relatively smaller video
datasets and can produce scenes with motion, but poorer appearance and 3D
structure. While these models have complementary strengths, they also have
opposing weaknesses, making it difficult to combine them in a way that
alleviates this three-way tradeoff. Here, we introduce hybrid score
distillation sampling, an alternating optimization procedure that blends
supervision signals from multiple pre-trained diffusion models and incorporates
benefits of each for high-fidelity text-to-4D generation. Using hybrid SDS, we
demonstrate synthesis of 4D scenes with compelling appearance, 3D structure,
and motion.

使用混合得分蒸馏取样的方法，我们展示了具有引人注目外观、3D 结构和动态的 4D 场景的合成。