Generative Adversarial Networks (GANs) are one of the most recent deep
learning models that generate synthetic data from limited genuine datasets.
GANs are on the frontier as further extension of deep learning into many
domains (e.g., medicine, robotics, content synthesis) requires massive sets of
labeled data that is generally either unavailable or prohibitively costly to
collect. Although GANs are gaining prominence in various fields, there are no
accelerators for these new models. In fact, GANs leverage a new operator,
called transposed convolution, that exposes unique challenges for hardware
acceleration. This operator first inserts zeros within the multidimensional
input, then convolves a kernel over this expanded array to add information to
the embedded zeros. Even though there is a convolution stage in this operator,
the inserted zeros lead to underutilization of the compute resources when a
conventional convolution accelerator is employed. We propose the GANAX
architecture to alleviate the sources of inefficiency associated with the
acceleration of GANs using conventional convolution accelerators, making the
first GAN accelerator design possible. We propose a reorganization of the
output computations to allocate compute rows with similar patterns of zeros to
adjacent processing engines, which also avoids inconsequential multiply-adds on
the zeros. This compulsory adjacency reclaims data reuse across these
neighboring processing engines, which had otherwise diminished due to the
inserted zeros. The reordering breaks the full SIMD execution model, which is
prominent in convolution accelerators. Therefore, we propose a unified
MIMD-SIMD design for GANAX that leverages repeated patterns in the computation
to create distinct microprograms that execute concurrently in SIMD mode.

本论文提出了一种名为 GANAX 的新型加速器设计，旨在解决深度生成对抗网络中的卷积精度和硬件加速效率问题，利用重新组织输出计算以及 MIMD-SIMD 统一设计等策略，有效加速了 GAN 的训练和运行。