We consider ML query processing in distributed systems where GPU-enabled
workers coordinate to execute complex queries: a computing style often seen in
applications that interact with users in support of image processing and
natural language processing. In such systems, coscheduling of GPU memory
management and task placement represents a promising opportunity. We propose
Navigator, a novel framework that unifies these functions to reduce job latency
while using resources efficiently, placing tasks where data dependencies will
be satisfied, collocating tasks from the same job (when this will not overload
the host or its GPU), and efficiently managing GPU memory. Comparison with
other state of the art schedulers shows a significant reduction in completion
times while requiring the same amount or even fewer resources. In one case,
just half the servers were needed for processing the same workload.

我们提出了 Navigator，这是一个将 GPU 内存管理和任务调度统一起来的新框架，它在保证资源利用效率的同时，将任务放置在满足数据依赖关系的位置，将来自同一作业的任务放置在一起（在不过载主机或其 GPU 的情况下），并有效地管理 GPU 内存，与其他领先的调度程序相比，完成时间明显缩短，同时所需的资源量相同甚至更少，在一个案例中，处理相同工作负载只需要一半的服务器。