High-performance computing systems are moving towards 2.5D and 3D memory
hierarchies, based on High Bandwidth Memory (HBM) and Hybrid Memory Cube (HMC)
to mitigate the main memory bottlenecks. This trend is also creating new
opportunities to revisit near-memory computation. In this paper, we propose a
flexible processor-in-memory (PIM) solution for scalable and energy-efficient
execution of deep convolutional networks (ConvNets), one of the fastest-growing
workloads for servers and high-end embedded systems. Our codesign approach
consists of a network of Smart Memory Cubes (modular extensions to the standard
HMC) each augmented with a many-core PIM platform called NeuroCluster.
NeuroClusters have a modular design based on NeuroStream coprocessors (for
Convolution-intensive computations) and general-purpose RISCV cores. In
addition, a DRAM-friendly tiling mechanism and a scalable computation paradigm
are presented to efficiently harness this computational capability with a very
low programming effort. NeuroCluster occupies only 8% of the total logic-base
(LoB) die area in a standard HMC and achieves an average performance of 240
GFLOPS for complete execution of full-featured state-of-the-art (SoA) ConvNets
within a power budget of 2.5W. Overall 11 W is consumed in a single SMC device,
with 22.5 GFLOPS/W energy-efficiency which is 3.5X better than the best GPU
implementations in similar technologies. The minor increase in system-level
power and the negligible area increase make our PIM system a cost-effective and
energy efficient solution, easily scalable to 955 GFLOPS with a small network
of just four SMCs.

本文提出了一种基于高带宽内存和处理器内存的柔性解决方案，名为 NeuroCluster，可用于深度卷积神经网络的可扩展和节能执行，并实现了最好的能源效率，平均性能为 240 GFLOPS，能源效率为 22.5 GFLOPS/W。