Many applications such as autonomous driving and augmented reality, require
the concurrent running of multiple deep neural networks (DNN) that poses
different levels of real-time performance requirements. However, coordinating
multiple DNN tasks with varying levels of criticality on edge GPUs remains an
area of limited study. Unlike server-level GPUs, edge GPUs are resource-limited
and lack hardware-level resource management mechanisms for avoiding resource
contention. Therefore, we propose Miriam, a contention-aware task coordination
framework for multi-DNN inference on edge GPU. Miriam consolidates two main
components, an elastic-kernel generator, and a runtime dynamic kernel
coordinator, to support mixed critical DNN inference. To evaluate Miriam, we
build a new DNN inference benchmark based on CUDA with diverse representative
DNN workloads. Experiments on two edge GPU platforms show that Miriam can
increase system throughput by 92% while only incurring less than 10\% latency
overhead for critical tasks, compared to state of art baselines.

提出了 Miriam，一种用于多个边缘 GPU 上的深度神经网络任务协调的计划，通过弹性内核生成器和运行时动态内核协调器，可以实现混合关键性 DNN 推理，并且在关键任务上只产生不到 10％的延迟开销，相比现有技术基准，系统吞吐量可提高 92％。