Diffusion models have garnered significant interest from the community for
their great generative ability across various applications. However, their
typical multi-step sequential-denoising nature gives rise to high cumulative
latency, thereby precluding the possibilities of parallel computation. To
address this, we introduce AsyncDiff, a universal and plug-and-play
acceleration scheme that enables model parallelism across multiple devices. Our
approach divides the cumbersome noise prediction model into multiple
components, assigning each to a different device. To break the dependency chain
between these components, it transforms the conventional sequential denoising
into an asynchronous process by exploiting the high similarity between hidden
states in consecutive diffusion steps. Consequently, each component is
facilitated to compute in parallel on separate devices. The proposed strategy
significantly reduces inference latency while minimally impacting the
generative quality. Specifically, for the Stable Diffusion v2.1, AsyncDiff
achieves a 2.7x speedup with negligible degradation and a 4.0x speedup with
only a slight reduction of 0.38 in CLIP Score, on four NVIDIA A5000 GPUs. Our
experiments also demonstrate that AsyncDiff can be readily applied to video
diffusion models with encouraging performances. The code is available at
this https URL

利用多设备之间的模型并行性，通过对多个组件进行异步处理，AsyncDiff 显著缩短推理延迟，同时最小程度地影响生成质量。