This paper introduces Helix, a distributed system for high-throughput,
low-latency large language model (LLM) serving on heterogeneous GPU clusters. A
key idea behind Helix is to formulate inference computation of LLMs over
heterogeneous GPUs and network connections as a max-flow problem for a
directed, weighted graph, whose nodes represent GPU instances and edges capture
both GPU and network heterogeneity through their capacities. Helix then uses a
mixed integer linear programming (MILP) algorithm to discover highly optimized
strategies to serve LLMs. This approach allows Helix to jointly optimize model
placement and request scheduling, two highly entangled tasks in heterogeneous
LLM serving. Our evaluation on several heterogeneous cluster settings ranging
from 24 to 42 GPU nodes shows that Helix improves serving throughput by up to
2.7$\times$ and reduces prompting and decoding latency by up to 2.8$\times$ and
1.3$\times$, respectively, compared to best existing approaches.

介绍了一种名为 Helix 的分布式系统，用于在异构 GPU 集群上提供高吞吐量和低延迟的大语言模型（LLM）服务。通过将 LLMs 的推理计算以最大流问题形式表示为有向加权图，使用混合整数线性规划（MILP）算法来发现高度优化的策略，从而联合优化模型放置和请求调度，在几个异构集群设置上的评估结果表明，与现有的最佳方法相比，Helix 将服务吞吐量提高了 2.7 倍，并将提示和解码延迟分别降低了 2.8 倍和 1.3 倍。

Helix: 基于异构 GPU 上的最大流进行大型语言模型的分布式服务

Helix: Distributed Serving of Large Language Models via Max-Flow on  Heterogeneous GPUs

Development of machine learning (ML) workflows is a tedious process of
iterative experimentation: developers repeatedly make changes to workflows
until the desired accuracy is attained. We describe our vision for a
"human-in-the-loop" ML system that accelerates this process: by intelligently
tracking changes and intermediate results over time, such a system can enable
rapid iteration, quick responsive feedback, introspection and debugging, and
background execution and automation. We finally describe Helix, our preliminary
attempt at such a system that has already led to speedups of up to 10x on
typical iterative workflows against competing systems.

该研究描述了一种 “人机互动” 机器学习系统的愿景，它通过智能跟踪更改和中间结果，实现了工作流加速，反馈及时，内省和调试，背景执行和自动化，并描述了 Helix，初步尝试这样一个系统已经在典型的迭代工作流中实现了最多 10 倍的提速。