Convolutional neural network inference on video data requires powerful
hardware for real-time processing. Given the inherent coherence across
consecutive frames, large parts of a video typically change little. By skipping
identical image regions and truncating insignificant pixel updates,
computational redundancy can in theory be reduced significantly. However, these
theoretical savings have been difficult to translate into practice, as sparse
updates hamper computational consistency and memory access coherence; which are
key for efficiency on real hardware. With DeltaCNN, we present a sparse
convolutional neural network framework that enables sparse frame-by-frame
updates to accelerate video inference in practice. We provide sparse
implementations for all typical CNN layers and propagate sparse feature updates
end-to-end - without accumulating errors over time. DeltaCNN is applicable to
all convolutional neural networks without retraining. To the best of our
knowledge, we are the first to significantly outperform the dense reference,
cuDNN, in practical settings, achieving speedups of up to 7x with only marginal
differences in accuracy.

DeltaCNN 是一个在视频数据上加速 CNN 网络推断的框架，通过对非重要像素进行截断和跳过相似图像区域以减少计算冗余，使得在实际设置中能够显著地提高效率并在准确度上只有少量差异。