Nature is infinitely resolution-free. In the context of this reality,
existing diffusion models, such as Diffusion Transformers, often face
challenges when processing image resolutions outside of their trained domain.
To overcome this limitation, we present the Flexible Vision Transformer (FiT),
a transformer architecture specifically designed for generating images with
unrestricted resolutions and aspect ratios. Unlike traditional methods that
perceive images as static-resolution grids, FiT conceptualizes images as
sequences of dynamically-sized tokens. This perspective enables a flexible
training strategy that effortlessly adapts to diverse aspect ratios during both
training and inference phases, thus promoting resolution generalization and
eliminating biases induced by image cropping. Enhanced by a meticulously
adjusted network structure and the integration of training-free extrapolation
techniques, FiT exhibits remarkable flexibility in resolution extrapolation
generation. Comprehensive experiments demonstrate the exceptional performance
of FiT across a broad range of resolutions, showcasing its effectiveness both
within and beyond its training resolution distribution. Repository available at
this https URL

为了克服传统图像处理方法中目标域以外的图像分辨率挑战，本文引入一种名为 Flexible Vision Transformer (FiT) 的变压器架构，它专门用于生成具有无限制分辨率和宽高比的图像。与传统的静态分辨率网格方式不同，FiT 将图像构想为动态尺寸令牌序列，从而实现了灵活的训练策略，适应了不同宽高比的图像。经过精心调整的网络结构和训练无关的外推技术的集成增强了 FiT 在分辨率外推生成方面的灵活性。综合实验证明 FiT 在广泛分辨率范围内表现出色，展示了它在训练分辨率分布内外的有效性。