Training deep neural networks (DNNs) is becoming increasingly more resource-
and energy-intensive every year. Unfortunately, existing works primarily focus
on optimizing DNN training for faster completion, often without considering the
impact on energy efficiency.
In this paper, we observe that common practices to improve training
performance can often lead to inefficient energy usage. More importantly, we
demonstrate that there is a tradeoff between energy consumption and performance
optimization. To this end, we propose Zeus, an optimization framework to
navigate this tradeoff by automatically finding optimal job- and GPU-level
configurations for recurring DNN training jobs. Zeus uses an online
exploration-exploitation approach in conjunction with just-in-time energy
profiling, averting the need for expensive offline measurements, while adapting
to data drifts over time. Our evaluation shows that Zeus can improve the energy
efficiency of DNN training by 15.3%-75.8% for diverse workloads.

本文提出了一种名为 Zeus 的优化框架，它可以自动找到适合于经常出现的深度神经网络训练任务的最佳作业和 GPU 级别配置，从而在考虑到能源消耗和性能优化之间的权衡时，可以提高 DNN 训练的能源效率达到 15.3% 至 75.8%。