Gathering knowledge about surroundings and generating situational awareness
for IoT devices is of utmost importance for systems developed for smart urban
and uncontested environments. For example, a large-area surveillance system is
typically equipped with multi-modal sensors such as cameras and LIDARs and is
required to execute deep learning algorithms for action, face, behavior, and
object recognition. However, these systems face power and memory constraints
due to their ubiquitous nature, making it crucial to optimize data processing,
deep learning algorithm input, and model inference communication. In this
paper, we propose a self-adaptive optimization framework for a testbed
comprising two Unmanned Ground Vehicles (UGVs) and two NVIDIA Jetson devices.
This framework efficiently manages multiple tasks (storage, processing,
computation, transmission, inference) on heterogeneous nodes concurrently. It
involves compressing and masking input image frames, identifying similar
frames, and profiling devices to obtain boundary conditions for optimization..
Finally, we propose and optimize a novel parameter split-ratio, which indicates
the proportion of the data required to be offloaded to another device while
considering the networking bandwidth, busy factor, memory (CPU, GPU, RAM), and
power constraints of the devices in the testbed. Our evaluations captured while
executing multiple tasks (e.g., PoseNet, SegNet, ImageNet, DetectNet, DepthNet)
simultaneously, reveal that executing 70% (split-ratio=70%) of the data on the
auxiliary node minimizes the offloading latency by approx. 33% (18.7 ms/image
to 12.5 ms/image) and the total operation time by approx. 47% (69.32s to
36.43s) compared to the baseline configuration (executing on the primary node).

本文提出了一个用于优化深度学习算法输入、模型推理通信、深度学习算法的自适应优化框架，并通过评估表明，将数据的 70% 分配给辅助结点可以最小化离线延迟，节约总操作时间。