Distributed inference is a popular approach for efficient DNN inference at
the edge. However, traditional Static and Dynamic DNNs are not
distribution-friendly, causing system reliability and adaptability issues. In
this paper, we introduce Fluid Dynamic DNNs (Fluid DyDNNs), tailored for
distributed inference. Distinct from Static and Dynamic DNNs, Fluid DyDNNs
utilize a novel nested incremental training algorithm to enable independent and
combined operation of its sub-networks, enhancing system reliability and
adaptability. Evaluation on embedded Arm CPUs with a DNN model and the MNIST
dataset, shows that in scenarios of single device failure, Fluid DyDNNs ensure
continued inference, whereas Static and Dynamic DNNs fail. When devices are
fully operational, Fluid DyDNNs can operate in either a High-Accuracy mode and
achieve comparable accuracy with Static DNNs, or in a High-Throughput mode and
achieve 2.5x and 2x throughput compared with Static and Dynamic DNNs,
respectively.

分布推理是边缘上高效 DNN 推理的一种流行方法。本文引入了专为分布推理设计的流体动态 DNNs（Fluid DyDNNs），通过使用新颖的嵌套增量训练算法来提高其子网络的独立和组合操作能力，从而增强系统的可靠性和适应性。在嵌入式 Arm CPU 上评估了一个 DNN 模型和 MNIST 数据集，在单设备故障的情况下，Fluid DyDNNs 确保推理的持续性，而 Static DNNs 和 Dynamic DNNs 则失败。当设备全部正常运行时，Fluid DyDNNs 可以在高精度模式下达到与 Static DNNs 相当的准确性，或者在高吞吐量模式下分别比 Static 和 Dynamic DNNs 提高 2.5 倍和 2 倍的吞吐量。