A recent frontier in computer vision has been the task of 3D video
generation, which consists of generating a time-varying 3D representation of a
scene. To generate dynamic 3D scenes, current methods explicitly model 3D
temporal dynamics by jointly optimizing for consistency across both time and
views of the scene. In this paper, we instead investigate whether it is
necessary to explicitly enforce multiview consistency over time, as current
approaches do, or if it is sufficient for a model to generate 3D
representations of each timestep independently. We hence propose a model,
Vid3D, that leverages 2D video diffusion to generate 3D videos by first
generating a 2D "seed" of the video's temporal dynamics and then independently
generating a 3D representation for each timestep in the seed video. We evaluate
Vid3D against two state-of-the-art 3D video generation methods and find that
Vid3D is achieves comparable results despite not explicitly modeling 3D
temporal dynamics. We further ablate how the quality of Vid3D depends on the
number of views generated per frame. While we observe some degradation with
fewer views, performance degradation remains minor. Our results thus suggest
that 3D temporal knowledge may not be necessary to generate high-quality
dynamic 3D scenes, potentially enabling simpler generative algorithms for this
task.

该研究讨论了计算机视觉中的一个前沿课题，3D 视频生成，以生成场景的时变 3D 表示为目标。通过提出 Vid3D 模型，研究表明可以通过生成视频的二维 “种子” 和独立生成每个时间步的三维表示，来实现高质量的动态 3D 场景生成，从而不再需要显式建模多视角一致性或三维时间动态，为此任务提供更简单的生成算法。