We introduce the MAsked Generative VIdeo Transformer, MAGVIT, to tackle
various video synthesis tasks with a single model. We introduce a 3D tokenizer
to quantize a video into spatial-temporal visual tokens and propose an
embedding method for masked video token modeling to facilitate multi-task
learning. We conduct extensive experiments to demonstrate the quality,
efficiency, and flexibility of MAGVIT. Our experiments show that (i) MAGVIT
performs favorably against state-of-the-art approaches and establishes the
best-published FVD on three video generation benchmarks, including the
challenging Kinetics-600. (ii) MAGVIT outperforms existing methods in inference
time by two orders of magnitude against diffusion models and by 60x against
autoregressive models. (iii) A single MAGVIT model supports ten diverse
generation tasks and generalizes across videos from different visual domains.
The source code and trained models will be released to the public at
this https URL

本研究提出了 MAgVIT，通过 3D 分词器对视频进行量化并通过视频掩码令牌建模进行多任务学习，展示了 MAgVIT 在视频生成任务中的质量、效率和灵活性，且支持多种生成任务。