Training large-scale artificial intelligence (AI) models demands significant
computational power and energy, leading to increased carbon footprint with
potential environmental repercussions. This paper delves into the challenges of
training AI models across geographically distributed (geo-distributed) data
centers, emphasizing the balance between learning performance and carbon
footprint. We consider Federated Learning (FL) as a solution, which prioritizes
model parameter exchange over raw data, ensuring data privacy and compliance
with local regulations. Given the variability in carbon intensity across
regions, we propose a new framework called CAFE (short for Carbon-Aware
Federated Learning) to optimize training within a fixed carbon footprint
budget. Our approach incorporates coreset selection to assess learning
performance, employs the Lyapunov drift-plus-penalty framework to address the
unpredictability of future carbon intensity, and devises an efficient algorithm
to address the combinatorial complexity of the data center selection. Through
extensive simulations using real-world carbon intensity data, we demonstrate
the efficacy of our algorithm, highlighting its superiority over existing
methods in optimizing learning performance while minimizing environmental
impact.

通过提出一个名为 CAFE 的框架，同时优化学习性能和减少环境影响，本研究重点探讨了在各地数据中心分布的条件下训练大规模人工智能模型所面临的挑战。