Specialized hardware accelerators have been extensively used for Deep Neural
Networks (DNNs) to provide power/performance benefits. These accelerators
contain specialized hardware that supports DNN operators, and scratchpad memory
for storing the tensor operands. Often, the size of the scratchpad is
insufficient to store all the tensors needed for the computation, and
additional data accesses are needed to move tensors back and forth from host
memory during the computation with significant power/performance overhead. The
volume of these additional data accesses depends on the operator schedule, and
memory allocation (specific locations selected for the tensors in the
scratchpad). We propose an optimization framework, named COSMA, for mapping
DNNs to an accelerator that finds the optimal operator schedule, memory
allocation and tensor replacement that minimizes the additional data accesses.
COSMA provides an Integer Linear Programming (ILP) formulation to generate the
optimal solution for mapping a DNN to the accelerator for a given scratchpad
size. We demonstrate that, using an off-the-shelf ILP solver, COSMA obtains the
optimal solution in seconds for a wide-range of state-of-the-art DNNs for
different applications. Further, it out-performs existing methods by reducing
on average 84% of the non-compulsory data accesses. We further propose a
divide-and-conquer heuristic to scale up to certain complex DNNs generated by
Neural Architecture Search, and this heuristic solution reduces on average 85%
data accesses compared with other works.

通过一个名为 COSMA 的优化框架，我们能够在专用硬件加速器上为 DNNs 找到最佳的操作调度、内存分配和张量替换方式，从而最小化额外数据访问，同时通过分而治之的启发式方法可以将复杂 DNNs 的数据访问降低 85% 以上。