Recent advances in Generative Artificial Intelligence have fueled numerous applications, particularly those involving Generative Adversarial Networks (GANs), which are essential for synthesizing realistic photos and videos. However, efficiently training GANs remains a critical challenge due to their computationally intensive and numerically unstable nature. Existing methods often require days or even weeks for training, posing significant resource and time constraints. In this work, we introduce ParaGAN, a scalable distributed GAN training framework that leverages asynchronous training and an asymmetric optimization policy to accelerate GAN training. ParaGAN employs a congestion-aware data pipeline and hardware-aware layout transformation to enhance accelerator utilization, resulting in over 30% improvements in throughput. With ParaGAN, we reduce the training time of BigGAN from 15 days to 14 hours while achieving 91% scaling efficiency. Additionally, ParaGAN enables unprecedented high-resolution image generation using BigGAN.

本研究解决了生成对抗网络（GAN）在训练过程中面临的效率低下和数值不稳定性的问题。我们提出的ParaGAN框架通过异步训练和不对称优化策略加快GAN的训练速度，显著降低了BigGAN的训练时间，从15天缩短至14小时，并实现了91%的扩展效率，同时能够生成前所未有的高分辨率图像。

ParaGAN：一个可扩展的分布式生成对抗网络训练框架