As modern DNN models grow ever larger, collective communications between the
accelerators (allreduce, etc.) emerge as a significant performance bottleneck.
Designing efficient communication schedules is challenging given today's highly
diverse and heterogeneous network fabrics. In this paper, we present
ForestColl, a tool that generates efficient schedules for any network topology.
ForestColl constructs broadcast/aggregation spanning trees as the communication
schedule, achieving theoretically minimum network congestion. Its schedule
generation runs in strongly polynomial time and is highly scalable. ForestColl
supports any network fabrics, including both switching fabrics and direct
connections, as well as any network graph structure. We evaluated ForestColl on
multi-cluster AMD MI250 and NVIDIA A100 platforms. ForestColl's schedules
achieved up to 52\% higher performance compared to the vendors' own optimized
communication libraries, RCCL and NCCL. ForestColl also outperforms other
state-of-the-art schedule generation techniques with both up to 61\% more
efficient generated schedules and orders of magnitude faster schedule
generation speed.

在现代 DNN 模型不断增大的同时，加速器之间的集体通信（allreduce 等）成为一个重要的性能瓶颈。在今天高度多样化和异构的网络结构下，设计高效的通信调度是具有挑战性的。本文提出了 ForestColl，一种为任何网络拓扑生成高效调度的工具。ForestColl 构建广播 / 聚合的生成树作为通信调度，实现了理论上的最小网络拥塞。其调度生成在强多项式时间内运行，并具有高度可扩展性。在多集群 AMD MI250 和 NVIDIA A100 平台上评估了 ForestColl。与供应商自己优化的通信库 RCCL 和 NCCL 相比，ForestColl 的调度性能提高了多达 52％。ForestColl 还在生成调度效率上超过其他最先进的调度生成技术，生成的调度更加高效，在调度生成速度上有数量级的提升。

ForestColl：异构网络纺织品上的高效集体通信

ForestColl: Efficient Collective Communications on Heterogeneous Network  Fabrics

Large language models such as ChatGPT have the potential to revolutionize the
construction industry by automating repetitive and time-consuming tasks. This
paper presents a study in which ChatGPT was used to generate a construction
schedule for a simple construction project. The output from ChatGPT was
evaluated by a pool of participants that provided feedback regarding their
overall interaction experience and the quality of the output. The results show
that ChatGPT can generate a coherent schedule that follows a logical approach
to fulfill the requirements of the scope indicated. The participants had an
overall positive interaction experience and indicated the great potential of
such a tool to automate many preliminary and time-consuming tasks. However, the
technology still has limitations, and further development is needed before it
can be widely adopted in the industry. Overall, this study highlights the
potential of using large language models in the construction industry and the
need for further research.

本研究探讨了使用 ChatGPT 自动生成建筑进度表的能力及其在建筑业自动化中的潜力，并得出如下结论： ChatGPT 能够生成符合逻辑并满足指示范围要求的进度表，具有显著的潜力用于自动化建筑行业相关任务，但技术仍有局限性需进一步发展以满足广泛的业界需求。