In this paper, we present NUWA-Infinity, a generative model for infinite
visual synthesis, which is defined as the task of generating arbitrarily-sized
high-resolution images or long-duration videos. An autoregressive over
autoregressive generation mechanism is proposed to deal with this variable-size
generation task, where a global patch-level autoregressive model considers the
dependencies between patches, and a local token-level autoregressive model
considers dependencies between visual tokens within each patch. A Nearby
Context Pool (NCP) is introduced to cache-related patches already generated as
the context for the current patch being generated, which can significantly save
computation costs without sacrificing patch-level dependency modeling. An
Arbitrary Direction Controller (ADC) is used to decide suitable generation
orders for different visual synthesis tasks and learn order-aware positional
embeddings. Compared to DALL-E, Imagen and Parti, NUWA-Infinity can generate
high-resolution images with arbitrary sizes and support long-duration video
generation additionally. Compared to NUWA, which also covers images and videos,
NUWA-Infinity has superior visual synthesis capabilities in terms of resolution
and variable-size generation. The GitHub link is
this https URL The homepage link is
this https URL

该论文提出了 NUWA-Infinity，这是一个无限生成视觉图像或视频的生成模型，它利用基于补丁级别的自回归模型和基于标记级别的自回归模型并引入了近邻上下文池和任意方向控制器来提高图像和视频的生成质量。