We propose distributed deep neural networks (DDNNs) over distributed
computing hierarchies, consisting of the cloud, the edge (fog) and end devices.
While being able to accommodate inference of a deep neural network (DNN) in the
cloud, a DDNN also allows fast and localized inference using shallow portions
of the neural network at the edge and end devices. When supported by a scalable
distributed computing hierarchy, a DDNN can scale up in neural network size and
scale out in geographical span. Due to its distributed nature, DDNNs enhance
sensor fusion, system fault tolerance and data privacy for DNN applications. In
implementing a DDNN, we map sections of a DNN onto a distributed computing
hierarchy. By jointly training these sections, we minimize communication and
resource usage for devices and maximize usefulness of extracted features which
are utilized in the cloud. The resulting system has built-in support for
automatic sensor fusion and fault tolerance. As a proof of concept, we show a
DDNN can exploit geographical diversity of sensors to improve object
recognition accuracy and reduce communication cost. In our experiment, compared
with the traditional method of offloading raw sensor data to be processed in
the cloud, DDNN locally processes most sensor data on end devices while
achieving high accuracy and is able to reduce the communication cost by a
factor of over 20x.

分布式深度神经网络 (DDNN) 可以在云端、边缘设备和终端中适应深度神经网络的推断，由于其分布式特性，DDNN 增强了 DNN 应用程序的传感器融合、系统容错和数据隐私，通过将 DNN 映射到分布式计算层次结构中的不同组成部分并联合训练这些部分，我们最小化设备通信和资源使用并最大化提取特征的实用性。实验结果表明，DDNN 可以利用传感器的地理多样性来提高物体识别精度并减少通信成本。