We study the problem of synthesizing a long-term dynamic video from only a single image. This is challenging since it requires consistent visual content movements given large camera motions. Existing methods either hallucinate inconsistent perpetual views or struggle with long camera trajectories. To address these issues, it is essential to estimate the underlying 4D (including 3D geometry and scene motion) and fill in the occluded regions. To this end, we present Make-It-4D, a novel method that can generate a consistent long-term dynamic video from a single image. On the one hand, we utilize layered depth images (LDIs) to represent a scene, and they are then unprojected to form a feature point cloud. To animate the visual content, the feature point cloud is displaced based on the scene flow derived from motion estimation and the corresponding camera pose. Such 4D representation enables our method to maintain the global consistency of the generated dynamic video. On the other hand, we fill in the occluded regions by using a pretrained diffusion model to inpaint and outpaint the input image. This enables our method to work under large camera motions. Benefiting from our design, our method can be training-free which saves a significant amount of training time. Experimental results demonstrate the effectiveness of our approach, which showcases compelling rendering results.

通过估计4D（包括3D几何和场景运动）并填充遮挡区域，本文提出了Make-It-4D方法，从单个图像中生成一致的长期动态视频。通过使用分层深度图像（LDIs）表示场景，并利用运动估计和相应的相机姿势引导的场景流来位移特征点云，我们的方法能够保持生成动态视频的全局一致性，并利用预训练扩散模型来修补输入图像的遮挡区域，从而适用于大幅度相机运动。实验结果证明了我们方法的有效性，展示了引人注目的渲染结果。

Make-It-4D: 从单张图像综合生成一致的长期动态场景视频