Recent advances in the diffusion models have significantly improved
text-to-image generation. However, generating videos from text is a more
challenging task than generating images from text, due to the much larger
dataset and higher computational cost required. Most existing video generation
methods use either a 3D U-Net architecture that considers the temporal
dimension or autoregressive generation. These methods require large datasets
and are limited in terms of computational costs compared to text-to-image
generation. To tackle these challenges, we propose a simple but effective novel
grid diffusion for text-to-video generation without temporal dimension in
architecture and a large text-video paired dataset. We can generate a
high-quality video using a fixed amount of GPU memory regardless of the number
of frames by representing the video as a grid image. Additionally, since our
method reduces the dimensions of the video to the dimensions of the image,
various image-based methods can be applied to videos, such as text-guided video
manipulation from image manipulation. Our proposed method outperforms the
existing methods in both quantitative and qualitative evaluations,
demonstrating the suitability of our model for real-world video generation.

通过使用网格扩散模型生成视频，我们可以通过固定数量的 GPU 内存生成高质量的视频，减小视频维度从而可以应用各种基于图像的方法，如基于文本的视频操作。我们的方法在定量和定性评估中表现优于现有方法，证明了我们的模型在真实世界中进行视频生成的适用性。