The unprecedented accuracy of convolutional neural networks (CNNs) across a
broad range of AI tasks has led to their widespread deployment in mobile and
embedded settings. In a pursuit for high-performance and energy-efficient
inference, significant research effort has been invested in the design of
FPGA-based CNN accelerators. In this context, single computation engines
constitute a popular approach to support diverse CNN modes without the overhead
of fabric reconfiguration. Nevertheless, this flexibility often comes with
significantly degraded performance on memory-bound layers and resource
underutilisation due to the suboptimal mapping of certain layers on the
engine's fixed configuration. In this work, we investigate the implications in
terms of CNN engine design for a class of models that introduce a
pre-convolution stage to decompress the weights at run time. We refer to these
approaches as on-the-fly. This paper presents unzipFPGA, a novel CNN inference
system that counteracts the limitations of existing CNN engines. The proposed
framework comprises a novel CNN hardware architecture that introduces a weights
generator module that enables the on-chip on-the-fly generation of weights,
alleviating the negative impact of limited bandwidth on memory-bound layers. We
further enhance unzipFPGA with an automated hardware-aware methodology that
tailors the weights generation mechanism to the target CNN-device pair, leading
to an improved accuracy-performance balance. Finally, we introduce an input
selective processing element (PE) design that balances the load between PEs in
suboptimally mapped layers. The proposed framework yields hardware designs that
achieve an average of 2.57x performance efficiency gain over highly optimised
GPU designs for the same power constraints and up to 3.94x higher performance
density over a diverse range of state-of-the-art FPGA-based CNN accelerators.

本文介绍了一种名为 unzipFPGA 的 CNN 推断系统，它通过引入权重生成模块实现了芯片上的实时权重生成，为小容量内存限制的层提供了更好的支持；此外，还介绍了一种自动化硬件感知的方法，将权重生成机制与目标 CNN 设备进行搭配，从而实现了更好的准确性和性能平衡；最后，还引入了一种输入选择性处理单元（PE）设计，平衡了子优化映射层的负载。该提议的框架在相同功耗约束下，与高度优化的 GPU 设计相比，实现了平均 2.57 倍的性能效率提升，同时在各种最先进的基于 FPGA 的 CNN 加速器中具有高达 3.94 倍的性能密度。