This paper introduces PipeFusion, a novel approach that harnesses multi-GPU
parallelism to address the high computational and latency challenges of
generating high-resolution images with diffusion transformers (DiT) models.
PipeFusion splits images into patches and distributes the network layers across
multiple devices. It employs a pipeline parallel manner to orchestrate
communication and computations. By leveraging the high similarity between the
input from adjacent diffusion steps, PipeFusion eliminates the waiting time in
the pipeline by reusing the one-step stale feature maps to provide context for
the current step. Our experiments demonstrate that it can generate higher image
resolution where existing DiT parallel approaches meet OOM. PipeFusion
significantly reduces the required communication bandwidth, enabling DiT
inference to be hosted on GPUs connected via PCIe rather than the more costly
NVLink infrastructure, which substantially lowers the overall operational
expenses for serving DiT models. Our code is publicly available at
this https URL

介绍了 PipeFusion 的一种新方法，利用多 GPU 并行性来解决扩散变压器（DiT）模型生成高分辨率图像的高计算和延迟挑战。通过分割图像并在多个设备上分布网络层，以管道并行的方式编排通信和计算，利用相邻扩散步骤输入的高相似性，通过重用一步陈旧的特征图为当前步骤提供上下文，实验结果表明，PipeFusion 能够在现有 DiT 并行方法无法满足 OOM 的情况下生成更高的图像分辨率。PipeFusion 显著降低了所需的通信带宽，使 DiT 推理能够在通过 PCIe 连接的 GPU 上托管，而不是更昂贵的 NVLink 基础结构，从而大幅降低了为服务 DiT 模型的整体运营费用。我们的代码公开可用于 https:// 所提供的 URL。