Neural Architecture Search (NAS) has demonstrated its power on various AI
accelerating platforms such as Field Programmable Gate Arrays (FPGAs) and
Graphic Processing Units (GPUs). However, it remains an open problem, how to
integrate NAS with Application-Specific Integrated Circuits (ASICs), despite
them being the most powerful AI accelerating platforms. The major bottleneck
comes from the large design freedom associated with ASIC designs. Moreover,
with the consideration that multiple DNNs will run in parallel for different
workloads with diverse layer operations and sizes, integrating heterogeneous
ASIC sub-accelerators for distinct DNNs in one design can significantly boost
performance, and at the same time further complicate the design space. To
address these challenges, in this paper we build ASIC template set based on
existing successful designs, described by their unique dataflows, so that the
design space is significantly reduced. Based on the templates, we further
propose a framework, namely NASAIC, which can simultaneously identify multiple
DNN architectures and the associated heterogeneous ASIC accelerator design,
such that the design specifications (specs) can be satisfied, while the
accuracy can be maximized. Experimental results show that compared with
successive NAS and ASIC design optimizations which lead to design spec
violations, NASAIC can guarantee the results to meet the design specs with
17.77%, 2.49x, and 2.32x reductions on latency, energy, and area and with 0.76%
accuracy loss. To the best of the authors' knowledge, this is the first work on
neural architecture and ASIC accelerator design co-exploration.

该论文提出了一种名为 NASAIC 的框架来同时确定多个 DNN 架构和相关的异构 ASIC 加速器设计，以满足设计规范并最大限度地提高准确性。