The last few years have brought advances in computer vision at an amazing
pace, grounded on new findings in deep neural network construction and training
as well as the availability of large labeled datasets. Applying these networks
to images demands a high computational effort and pushes the use of
state-of-the-art networks on real-time video data out of reach of embedded
platforms. Many recent works focus on reducing network complexity for real-time
inference on embedded computing platforms. We adopt an orthogonal viewpoint and
propose a novel algorithm exploiting the spatio-temporal sparsity of pixel
changes. This optimized inference procedure resulted in an average speed-up of
9.1x over cuDNN on the Tegra X2 platform at a negligible accuracy loss of <0.1%
and no retraining of the network for a semantic segmentation application.
Similarly, an average speed-up of 7.0x has been achieved for a pose detection
DNN and a reduction of 5x of the number of arithmetic operations to be
performed for object detection on static camera video surveillance data. These
throughput gains combined with a lower power consumption result in an energy
efficiency of 511 GOp/s/W compared to 70 GOp/s/W for the baseline.

采用新算法利用空间 - 时间稀疏性在计算机视觉中取得了高能效，即使在嵌入式设备上不需要重新训练网络，也可在语义分割应用中达到快速且准确的实时推断

CBinfer：利用帧间局部性加速卷积神经网络对视频流的推断

CBinfer: Exploiting Frame-to-Frame Locality for Faster Convolutional  Network Inference on Video Streams

Extracting per-frame features using convolutional neural networks for
real-time processing of video data is currently mainly performed on powerful
GPU-accelerated workstations and compute clusters. However, there are many
applications such as smart surveillance cameras that require or would benefit
from on-site processing. To this end, we propose and evaluate a novel algorithm
for change-based evaluation of CNNs for video data recorded with a static
camera setting, exploiting the spatio-temporal sparsity of pixel changes. We
achieve an average speed-up of 8.6x over a cuDNN baseline on a realistic
benchmark with a negligible accuracy loss of less than 0.1% and no retraining
of the network. The resulting energy efficiency is 10x higher than that of
per-frame evaluation and reaches an equivalent of 328 GOp/s/W on the Tegra X1
platform.

本论文提出一种新的算法，利用像素变化的时空稀疏性，对静态摄像机录制的视频数据进行卷积神经网络的基于变化的评估，从而实现本地视频数据的处理，其速度比 cuDNN 基准测试平均快 8.6 倍，精度损失不到 0.1％，不需要对网络进行重新训练，而能源效率可达 328 GOp /s/ W。