The computation of convolution layers in deep neural networks typically rely
on high performance routines that trade space for time by using additional
memory (either for packing purposes or required as part of the algorithm) to
improve performance. The problems with such an approach are two-fold. First,
these routines incur additional memory overhead which reduces the overall size
of the network that can fit on embedded devices with limited memory capacity.
Second, these high performance routines were not optimized for performing
convolution, which means that the performance obtained is usually less than
conventionally expected. In this paper, we demonstrate that direct convolution,
when implemented correctly, eliminates all memory overhead, and yields
performance that is between 10% to 400% times better than existing high
performance implementations of convolution layers on conventional and embedded
CPU architectures. We also show that a high performance direct convolution
exhibits better scaling performance, i.e. suffers less performance drop, when
increasing the number of threads.

本篇论文研究并证明了，当直接卷积实现正确时，消除了所有的内存开销，且效率在传统和嵌入式 CPU 架构上比现有的高性能卷积实现提高了 10% 到 400% 不等，并可以更好地扩展性能，即增加线程数时的性能下降更少。

高性能零内存开销直接卷积

High Performance Zero-Memory Overhead Direct Convolutions

Convolution layers are prevalent in many classes of deep neural networks,
including Convolutional Neural Networks (CNNs) which provide state-of-the-art
results for tasks like image recognition, neural machine translation and speech
recognition. The computationally expensive nature of a convolution operation
has led to the proliferation of implementations including matrix-matrix
multiplication formulation, and direct convolution primarily targeting GPUs. In
this paper, we introduce direct convolution kernels for x86 architectures, in
particular for Xeon and XeonPhi systems, which are implemented via a dynamic
compilation approach. Our JIT-based implementation shows close to theoretical
peak performance, depending on the setting and the CPU architecture at hand. We
additionally demonstrate how these JIT-optimized kernels can be integrated into
a lightweight multi-node graph execution model. This illustrates that single-
and multi-node runs yield high efficiencies and high image-throughputs when
executing state-of-the-art image recognition tasks on CPUs.

本文介绍了针对 x86 体系结构的直接卷积核和动态编译方法实现的 JIT 优化内核，该内核可在多节点下高效执行最新的图像识别任务，使单机和多节点运行时高效地通过 CPU 执行任务的高吞吐量。