Diffusion models excel at generating photo-realistic images but come with
significant computational costs in both training and sampling. While various
techniques address these computational challenges, a less-explored issue is
designing an efficient and adaptable network backbone for iterative refinement.
Current options like U-Net and Vision Transformer often rely on
resource-intensive deep networks and lack the flexibility needed for generating
images at variable resolutions or with a smaller network than used in training.
This study introduces LEGO bricks, which seamlessly integrate Local-feature
Enrichment and Global-content Orchestration. These bricks can be stacked to
create a test-time reconfigurable diffusion backbone, allowing selective
skipping of bricks to reduce sampling costs and generate higher-resolution
images than the training data. LEGO bricks enrich local regions with an MLP and
transform them using a Transformer block while maintaining a consistent
full-resolution image across all bricks. Experimental results demonstrate that
LEGO bricks enhance training efficiency, expedite convergence, and facilitate
variable-resolution image generation while maintaining strong generative
performance. Moreover, LEGO significantly reduces sampling time compared to
other methods, establishing it as a valuable enhancement for diffusion models.

这项研究引入了乐高积木，通过无缝地整合局部特征增强和全局内容编排，设计了一种高效灵活的网络主干，用于迭代改进。乐高积木可以在测试时重新配置扩散主干，通过选择性跳过积木来减少采样成本并生成比训练数据分辨率更高的图像。实验结果表明，乐高积木提高了训练效率，加快了收敛速度，并在保持强大生成性能的同时促进了可变分辨率图像的生成。此外，与其他方法相比，乐高显著减少了采样时间，使其成为扩散模型的有价值增强。