Deep learning inference on streaming media data, such as object detection in
video or LiDAR feeds and text extraction from audio waves, is now ubiquitous.
To achieve high inference accuracy, these applications typically require
significant network bandwidth to gather high-fidelity data and extensive GPU
resources to run deep neural networks (DNNs). While the high demand for network
bandwidth and GPU resources could be substantially reduced by optimally
adapting the configuration knobs, such as video resolution and frame rate,
current adaptation techniques fail to meet three requirements simultaneously:
adapt configurations (i) with minimum extra GPU or bandwidth overhead; (ii) to
reach near-optimal decisions based on how the data affects the final DNN's
accuracy, and (iii) do so for a range of configuration knobs. This paper
presents OneAdapt, which meets these requirements by leveraging a
gradient-ascent strategy to adapt configuration knobs. The key idea is to
embrace DNNs' differentiability to quickly estimate the accuracy's gradient to
each configuration knob, called AccGrad. Specifically, OneAdapt estimates
AccGrad by multiplying two gradients: InputGrad (i.e. how each configuration
knob affects the input to the DNN) and DNNGrad (i.e. how the DNN input affects
the DNN inference output). We evaluate OneAdapt across five types of
configurations, four analytic tasks, and five types of input data. Compared to
state-of-the-art adaptation schemes, OneAdapt cuts bandwidth usage and GPU
usage by 15-59% while maintaining comparable accuracy or improves accuracy by
1-5% while using equal or fewer resources.

通过优化配置选项以提高深度学习推理的准确性和降低网络带宽和 GPU 资源的使用量，本文提出了一种名为 OneAdapt 的方法，通过利用深度神经网络的可微性来快速估计准确度梯度，从而达到三个要求：最小化额外的 GPU 或带宽开销，基于数据对最终深度神经网络的准确性的影响达到接近最优的决策，并在一系列配置选项范围内实现这些目标。与现有技术相比，OneAdapt 在减少带宽和 GPU 使用量方面可达 15-59%，同时准确度保持相当或提高 1-5% 且使用相同或更少的资源。