Over the past years, great progress has been made in improving the computing
power of general-purpose graphics processing units (GPGPUs), which facilitates
the prosperity of deep neural networks (DNNs) in multiple fields like computer
vision and natural language processing. A typical DNN training process
repeatedly updates tens of millions of parameters, which not only requires huge
computing resources but also consumes significant energy. In order to train
DNNs in a more energy-efficient way, we empirically investigate the impact of
GPU Dynamic Voltage and Frequency Scaling (DVFS) on the energy consumption and
performance of deep learning. Our experiments cover a wide range of GPU
architectures, DVFS settings, and DNN configurations. We observe that, compared
to the default core frequency settings of three tested GPUs, the optimal core
frequency can help conserve 8.7%$\sim$23.1% energy consumption for different
DNN training cases. Regarding the inference, the benefits vary from
19.6%$\sim$26.4%. Our findings suggest that GPU DVFS has great potentials to
help develop energy efficient DNN training/inference schemes.

研究探讨了 GPU 动态电压频率调节对深度学习的影响，发现与默认核心频率相比，最优核心频率可以在不同的 DNN 训练情况下节省 8.7%~23.1% 能源消耗，推断时收益范围为 19.6%~26.4%。这一研究表明，GPU DVFS 有助于开发能源高效的 DNN 训练 / 推断方案。