Training large AI models on numerous GPUs consumes a massive amount of
energy. We observe that not all energy consumed during training directly
contributes to end-to-end training throughput, and a significant portion can be
removed without slowing down training, which we call energy bloat.
In this work, we identify two independent sources of energy bloat in large
model training, intrinsic and extrinsic, and propose Perseus, a unified
optimization framework that mitigates both. Perseus obtains the "iteration
time-energy" Pareto frontier of any large model training job using an efficient
iterative graph cut-based algorithm and schedules energy consumption of its
forward and backward computations across time to remove intrinsic and extrinsic
energy bloat. Evaluation on large models like GPT-3 and Bloom shows that
Perseus reduces energy consumption of large model training by up to 30%,
enabling savings otherwise unobtainable before.

通过使用 Perseus 框架，我们能够减少大型 AI 模型训练的能量消耗，提高训练效率，进而节省能源。

Perseus: 从大型模型训练中消除能量浪费

Perseus: Removing Energy Bloat from Large Model Training

Cross-Lingual Summarization (CLS) aims at generating summaries in one
language for the given documents in another language. CLS has attracted wide
research attention due to its practical significance in the multi-lingual
world. Though great contributions have been made, existing CLS works typically
focus on short documents, such as news articles, short dialogues and guides.
Different from these short texts, long documents such as academic articles and
business reports usually discuss complicated subjects and consist of thousands
of words, making them non-trivial to process and summarize. To promote CLS
research on long documents, we construct Perseus, the first long-document CLS
dataset which collects about 94K Chinese scientific documents paired with
English summaries. The average length of documents in Perseus is more than two
thousand tokens. As a preliminary study on long-document CLS, we build and
evaluate various CLS baselines, including pipeline and end-to-end methods.
Experimental results on Perseus show the superiority of the end-to-end
baseline, outperforming the strong pipeline models equipped with sophisticated
machine translation systems. Furthermore, to provide a deeper understanding, we
manually analyze the model outputs and discuss specific challenges faced by
current approaches. We hope that our work could benchmark long-document CLS and
benefit future studies.

本文介绍了长文档跨语言摘要的研究现状，并构建了 Perseus 数据集，评估了不同的从结构和端到端方案，结果表明端到端方案卓越地胜过利用复杂机器翻译系统配备的管道模型。